RollingRestartArgs/RollingRestartITest.TestWorkloads/3: /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:753: Failure
Failed
Bad status: Timed out: cluster didn't become healthy: Running 'cluster ksck 127.16.221.126:44487' did not succeed in 60 seconds: Runtime error: error running 'cluster ksck 127.16.221.126:44487': stdout: Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
b182c77196c04b549d02ed8a4be16054 | 127.16.221.126:44487 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.16.221.84:34517 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server States
Server | State
----------------------------------+------------------
6cbd668c726a4b1298e789270a0031e2 | MAINTENANCE_MODE
8150d3eeb1c84234a13484098d17ce8f | MAINTENANCE_MODE
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
1c0cb4e9f1d94dfbbfbaf9138aa4e655 | 127.16.221.65:45529 | HEALTHY | /L0 | 0 | 0
1fe60cc8e38948b694506041d7734713 | 127.16.221.71:44909 | HEALTHY | /L1 | 0 | 0
439cf2177bdb4c19aa8449b6cbd440a7 | 127.16.221.67:41189 | HEALTHY | /L0 | 0 | 3
4f219d2e8b954461973a8ca8680a6383 | 127.16.221.69:35369 | HEALTHY | /L1 | 1 | 0
6cbd668c726a4b1298e789270a0031e2 | 127.16.221.73:38265 | HEALTHY | /L2 | 0 | 0
8150d3eeb1c84234a13484098d17ce8f | 127.16.221.72:40741 | HEALTHY | /L2 | 0 | 0
bede5abcb6174794bd2283843157bcef | 127.16.221.70:41185 | HEALTHY | /L1 | 0 | 0
d33fd96ef7d44ad8805718466ca9447f | 127.16.221.68:42243 | HEALTHY | /L1 | 0 | 0
e1bf694ea1724daaa49f0f0746a03eeb | 127.16.221.66:42593 | HEALTHY | /L0 | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
/L2 | 2
/L0 | 3
/L1 | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.16.221.65 | experimental | 127.16.221.65:45529
local_ip_for_outbound_sockets | 127.16.221.66 | experimental | 127.16.221.66:42593
local_ip_for_outbound_sockets | 127.16.221.67 | experimental | 127.16.221.67:41189
local_ip_for_outbound_sockets | 127.16.221.68 | experimental | 127.16.221.68:42243
local_ip_for_outbound_sockets | 127.16.221.69 | experimental | 127.16.221.69:35369
local_ip_for_outbound_sockets | 127.16.221.70 | experimental | 127.16.221.70:41185
local_ip_for_outbound_sockets | 127.16.221.71 | experimental | 127.16.221.71:44909
local_ip_for_outbound_sockets | 127.16.221.72 | experimental | 127.16.221.72:40741
local_ip_for_outbound_sockets | 127.16.221.73 | experimental | 127.16.221.73:38265
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb | hidden | 127.16.221.65:45529
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb | hidden | 127.16.221.66:42593
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb | hidden | 127.16.221.67:41189
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb | hidden | 127.16.221.68:42243
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb | hidden | 127.16.221.69:35369
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb | hidden | 127.16.221.70:41185
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb | hidden | 127.16.221.71:44909
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb | hidden | 127.16.221.72:40741
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb | hidden | 127.16.221.73:38265
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.16.221.84:34517 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
Tablet 5ca5ea09d51f4eb390c4433fec8ed9fa of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): RUNNING
4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): RUNNING [LEADER]
6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap replayed 2/3 log segments. Stats: ops{read=9245 overwritten=0 applied=9244 ignored=0} inserts{seen=462100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
All reported replicas are:
A = 439cf2177bdb4c19aa8449b6cbd440a7
B = 4f219d2e8b954461973a8ca8680a6383
C = 6cbd668c726a4b1298e789270a0031e2
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
, stderr: W20250624 07:58:09.168478 21569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:58:09.168990 21569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:58:09.187798 21569 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Runtime error: ksck discovered errors
: /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu: process exited with non-zero status 1
W20250624 07:58:11.427768 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 526: this message will repeat every 5th retry.
W20250624 07:58:11.926002 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 531: this message will repeat every 5th retry.
I20250624 07:58:12.205101 21576 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 23 similar messages]
I20250624 07:58:12.376014 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 3/3 (572.4K/3.09M this segment, stats: ops{read=9572 overwritten=0 applied=9572 ignored=0} inserts{seen=478500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:58:12.446153 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 536: this message will repeat every 5th retry.
W20250624 07:58:12.963755 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 541: this message will repeat every 5th retry.
W20250624 07:58:13.516028 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 546: this message will repeat every 5th retry.
W20250624 07:58:14.070739 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 551: this message will repeat every 5th retry.
W20250624 07:58:14.551173 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 556: this message will repeat every 5th retry.
W20250624 07:58:15.063230 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 561: this message will repeat every 5th retry.
I20250624 07:58:15.232630 21628 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:58:15.606241 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 566: this message will repeat every 5th retry.
W20250624 07:58:16.074327 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 571: this message will repeat every 5th retry.
W20250624 07:58:16.568455 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 576: this message will repeat every 5th retry.
W20250624 07:58:17.070832 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 581: this message will repeat every 5th retry.
I20250624 07:58:17.376118 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 3/3 (2.36M/3.09M this segment, stats: ops{read=10626 overwritten=0 applied=10625 ignored=0} inserts{seen=531150 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:58:17.632210 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 586: this message will repeat every 5th retry.
W20250624 07:58:18.142153 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 591: this message will repeat every 5th retry.
I20250624 07:58:18.269800 21631 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 29 similar messages]
W20250624 07:58:18.641099 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 596: this message will repeat every 5th retry.
W20250624 07:58:19.116403 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 601: this message will repeat every 5th retry.
I20250624 07:58:19.437165 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replayed 3/3 log segments. Stats: ops{read=11055 overwritten=0 applied=11055 ignored=0} inserts{seen=552650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250624 07:58:19.438315 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap complete.
I20250624 07:58:19.445492 20980 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent bootstrapping tablet: real 68.600s user 46.265s sys 1.569s
I20250624 07:58:19.451423 20980 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.452657 20980 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Initialized, Role: FOLLOWER
I20250624 07:58:19.453598 20980 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11055, Last appended: 2.11055, Last appended by leader: 11055, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.462325 20980 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
I20250624 07:58:19.818593 21632 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:19.819025 21632 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.821497 21632 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:19.856235 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 11480 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:19.873847 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 11480 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:19.875397 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:20.170615 21632 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:21.375931 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:21.376569 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:21.378225 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:21.379071 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 13605 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:21.380018 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 13605 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:21.381042 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:21.381672 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:21.389463 21631 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 1841 ops behind the committed index [suppressed 17 similar messages]
I20250624 07:58:21.731043 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:21.731282 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:21.732007 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:21.732952 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14030 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:21.732983 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14030 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:21.735353 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:21.736313 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:22.102583 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:22.102818 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:22.103601 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:22.104065 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14455 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:22.104283 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14455 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:22.104905 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:22.106537 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:22.742956 21648 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:22.743311 21648 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:22.744582 21648 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:22.745646 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15305 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:22.750190 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15305 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:22.751720 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:22.755040 21648 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:23.308735 21648 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:23.308986 21648 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:23.309844 21648 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:23.311044 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:23.312011 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:23.312705 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:23.313453 21648 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:23.394874 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 37.790s user 24.589s sys 0.426s Metrics: {"bytes_written":20043962,"cfile_init":1,"dirs.queue_time_us":14808,"dirs.run_cpu_time_us":274,"dirs.run_wall_time_us":1030,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":172663,"lbm_writes_1-10_ms":14,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":859,"peak_mem_usage":0,"rows_written":722800,"thread_start_us":21608,"threads_started":2,"wal-append.queue_time_us":6873}
I20250624 07:58:23.397328 19772 maintenance_manager.cc:419] P 439cf2177bdb4c19aa8449b6cbd440a7: Scheduling UndoDeltaBlockGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa): 8294353 bytes on disk
I20250624 07:58:23.398746 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: UndoDeltaBlockGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20250624 07:58:26.406147 19772 maintenance_manager.cc:419] P 439cf2177bdb4c19aa8449b6cbd440a7: Scheduling LogGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa): free 25168025 bytes of WAL
I20250624 07:58:26.406726 19659 log_reader.cc:385] T 5ca5ea09d51f4eb390c4433fec8ed9fa: removed 3 log segments from log reader
I20250624 07:58:26.406890 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000001 (ops 1-4937)
I20250624 07:58:26.407029 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000002 (ops 4938-9696)
I20250624 07:58:26.407121 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000003 (ops 9697-14400)
I20250624 07:58:26.416723 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: LogGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 0.010s user 0.000s sys 0.007s Metrics: {}
I20250624 07:58:26.475273 17269 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250624 07:58:26.475557 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 1c0cb4e9f1d94dfbbfbaf9138aa4e655 and pid 19496
************************ BEGIN STACKS **************************
I20250624 07:58:27.180711 20211 maintenance_manager.cc:419] P 4f219d2e8b954461973a8ca8680a6383: Scheduling FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa): perf score=0.080383
[New LWP 19498]
[New LWP 19499]
[New LWP 19500]
[New LWP 19501]
[New LWP 19508]
[New LWP 19509]
[New LWP 19510]
[New LWP 19513]
[New LWP 19514]
[New LWP 19515]
[New LWP 19517]
[New LWP 19518]
[New LWP 19519]
[New LWP 19520]
[New LWP 19521]
[New LWP 19522]
[New LWP 19523]
[New LWP 19524]
[New LWP 19525]
[New LWP 19526]
[New LWP 19527]
[New LWP 19528]
[New LWP 19529]
[New LWP 19530]
[New LWP 19531]
[New LWP 19532]
[New LWP 19533]
[New LWP 19534]
[New LWP 19535]
[New LWP 19536]
[New LWP 19537]
[New LWP 19538]
[New LWP 19539]
[New LWP 19540]
[New LWP 19541]
[New LWP 19542]
[New LWP 19543]
[New LWP 19544]
[New LWP 19545]
[New LWP 19546]
[New LWP 19547]
[New LWP 19548]
[New LWP 19549]
[New LWP 19550]
[New LWP 19551]
[New LWP 19552]
[New LWP 19553]
[New LWP 19554]
[New LWP 19555]
[New LWP 19556]
[New LWP 19557]
[New LWP 19558]
[New LWP 19559]
[New LWP 19560]
[New LWP 19561]
[New LWP 19562]
[New LWP 19563]
[New LWP 19564]
[New LWP 19565]
[New LWP 19566]
[New LWP 19567]
[New LWP 19568]
[New LWP 19569]
[New LWP 19570]
[New LWP 19571]
[New LWP 19572]
[New LWP 19573]
[New LWP 19574]
[New LWP 19575]
[New LWP 19576]
[New LWP 19577]
[New LWP 19578]
[New LWP 19579]
[New LWP 19580]
[New LWP 19581]
[New LWP 19582]
[New LWP 19583]
[New LWP 19584]
[New LWP 19585]
[New LWP 19586]
[New LWP 19587]
[New LWP 19588]
[New LWP 19589]
[New LWP 19590]
[New LWP 19591]
[New LWP 19592]
[New LWP 19593]
[New LWP 19594]
[New LWP 19595]
[New LWP 19596]
[New LWP 19597]
[New LWP 19598]
[New LWP 19599]
[New LWP 19600]
[New LWP 19601]
[New LWP 19602]
[New LWP 19603]
[New LWP 19604]
[New LWP 19605]
[New LWP 19606]
[New LWP 19607]
[New LWP 19608]
[New LWP 19609]
[New LWP 19610]
[New LWP 19611]
[New LWP 19612]
[New LWP 19613]
[New LWP 19614]
[New LWP 19615]
[New LWP 19616]
[New LWP 19617]
[New LWP 19618]
[New LWP 19619]
[New LWP 19620]
[New LWP 19621]
[New LWP 19622]
[New LWP 19623]
[New LWP 19624]
[New LWP 19625]
[New LWP 19626]
[New LWP 19634]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f544da40d50 in ?? ()
Id Target Id Frame
* 1 LWP 19496 "kudu" 0x00007f544da40d50 in ?? ()
2 LWP 19498 "kudu" 0x00007f544da3cfb9 in ?? ()
3 LWP 19499 "kudu" 0x00007f544da3cfb9 in ?? ()
4 LWP 19500 "kudu" 0x00007f544da3cfb9 in ?? ()
5 LWP 19501 "kernel-watcher-" 0x00007f544da3cfb9 in ?? ()
6 LWP 19508 "ntp client-1950" 0x00007f544da409e2 in ?? ()
7 LWP 19509 "file cache-evic" 0x00007f544da3cfb9 in ?? ()
8 LWP 19510 "sq_acceptor" 0x00007f5446762cb9 in ?? ()
9 LWP 19513 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
10 LWP 19514 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
11 LWP 19515 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
12 LWP 19517 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
13 LWP 19518 "MaintenanceMgr " 0x00007f544da3cad3 in ?? ()
14 LWP 19519 "txn-status-mana" 0x00007f544da3cfb9 in ?? ()
15 LWP 19520 "collect_and_rem" 0x00007f544da3cfb9 in ?? ()
16 LWP 19521 "tc-session-exp-" 0x00007f544da3cfb9 in ?? ()
17 LWP 19522 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
18 LWP 19523 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
19 LWP 19524 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
20 LWP 19525 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
21 LWP 19526 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
22 LWP 19527 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
23 LWP 19528 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
24 LWP 19529 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
25 LWP 19530 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
26 LWP 19531 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
27 LWP 19532 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
28 LWP 19533 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
29 LWP 19534 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
30 LWP 19535 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
31 LWP 19536 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
32 LWP 19537 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
33 LWP 19538 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
34 LWP 19539 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
35 LWP 19540 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
36 LWP 19541 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
37 LWP 19542 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
38 LWP 19543 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
39 LWP 19544 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
40 LWP 19545 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
41 LWP 19546 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
42 LWP 19547 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
43 LWP 19548 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
44 LWP 19549 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
45 LWP 19550 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
46 LWP 19551 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
47 LWP 19552 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
48 LWP 19553 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
49 LWP 19554 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
50 LWP 19555 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
51 LWP 19556 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
52 LWP 19557 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
53 LWP 19558 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
54 LWP 19559 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
55 LWP 19560 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
56 LWP 19561 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
57 LWP 19562 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
58 LWP 19563 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
59 LWP 19564 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
60 LWP 19565 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
61 LWP 19566 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
62 LWP 19567 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
63 LWP 19568 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
64 LWP 19569 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
65 LWP 19570 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
66 LWP 19571 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
67 LWP 19572 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
68 LWP 19573 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
69 LWP 19574 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
70 LWP 19575 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
71 LWP 19576 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
72 LWP 19577 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
73 LWP 19578 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
74 LWP 19579 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
75 LWP 19580 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
76 LWP 19581 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
77 LWP 19582 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
78 LWP 19583 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
79 LWP 19584 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
80 LWP 19585 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
81 LWP 19586 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
82 LWP 19587 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
83 LWP 19588 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
84 LWP 19589 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
85 LWP 19590 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
86 LWP 19591 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
87 LWP 19592 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
88 LWP 19593 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
89 LWP 19594 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
90 LWP 19595 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
91 LWP 19596 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
92 LWP 19597 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
93 LWP 19598 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
94 LWP 19599 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
95 LWP 19600 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
96 LWP 19601 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
97 LWP 19602 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
98 LWP 19603 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
99 LWP 19604 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
100 LWP 19605 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
101 LWP 19606 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
102 LWP 19607 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
103 LWP 19608 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
104 LWP 19609 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
105 LWP 19610 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
106 LWP 19611 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
107 LWP 19612 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
108 LWP 19613 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
109 LWP 19614 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
110 LWP 19615 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
111 LWP 19616 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
112 LWP 19617 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
113 LWP 19618 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
114 LWP 19619 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
115 LWP 19620 "rpc worker-1962" 0x00007f544da3cad3 in ?? ()
116 LWP 19621 "rpc worker-1962" 0x00007f544da3cad3 in ?? ()
117 LWP 19622 "diag-logger-196" 0x00007f544da3cfb9 in ?? ()
118 LWP 19623 "result-tracker-" 0x00007f544da3cfb9 in ?? ()
119 LWP 19624 "excess-log-dele" 0x00007f544da3cfb9 in ?? ()
120 LWP 19625 "acceptor-19625" 0x00007f54467710c7 in ?? ()
121 LWP 19626 "heartbeat-19626" 0x00007f54513041e1 in ?? ()
122 LWP 19634 "maintenance_sch" 0x00007f544da3cfb9 in ?? ()
Thread 122 (LWP 19634):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5402ad56e0 in ?? ()
#2 0x00000000000002c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f5402ad5730 in ?? ()
#6 0x0000000000000592 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19626):
#0 0x00007f54513041e1 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 120 (LWP 19625):
#0 0x00007f54467710c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f5404323d90 in ?? ()
#3 0x00007f5404323dc0 in ?? ()
#4 0x00007f5404323ec0 in ?? ()
#5 0x00007f5404323db0 in ?? ()
#6 0x00007f5404323e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f544a5f7980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19624):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5404b3df40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007fff340afc50 in ?? ()
#5 0x00007f5404b3df30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19623):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5405356100 in ?? ()
#2 0x00000000000000b2 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f54053560f0 in ?? ()
#6 0x0000000000000164 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19622):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19621):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000024 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f5406385e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5406385eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 19620):
#0 0x00007f544da3cad3 in ?? ()
#1 0x000000000000001c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f5406b9de90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5406b9deb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 19619):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19618):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19617):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19616):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19615):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19614):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19613):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19612):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19611):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19610):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19609):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19608):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19607):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19606):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19605):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19604):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19603):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19602):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19601):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19600):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19599):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19598):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19597):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19596):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19595):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19594):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19593):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19592):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19591):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19590):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19589):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19588):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19587):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19586):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19585):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19584):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19583):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19582):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19581):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f5419f3ee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5419f3eeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19580):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19579):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19578):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19577):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19576):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19575):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19574):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19573):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19572):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19571):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19570):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19569):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19568):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19567):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19566):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19565):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19564):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19563):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19562):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19561):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f542411fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f542411feb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19560):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19559):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19558):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19557):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19556):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19555):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19554):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19553):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19552):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19551):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19550):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19549):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19548):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19547):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19546):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19545):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19544):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19543):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19542):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19541):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f542e300e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f542e300eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 19540):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19539):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19538):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19537):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19536):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19535):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19534):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19533):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19532):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19531):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19530):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19529):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19528):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19527):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19526):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19525):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19524):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19523):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19522):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19521):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f54383c2cc0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f54383c2cb0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19520):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19519):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f54393c4240 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f54393c4230 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19518):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19517):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543a3c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f543a3c7310 in ?? ()
#4 0x00007f543a3c7520 in ?? ()
#5 0x00007f543a3c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f543a3c73e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3f50624dd2f1a9fc in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x000000007c900000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000feb07470e7c in ?? ()
#17 0x00007f543a3c73c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543a3c7390 in ?? ()
#20 0x3f50624dd2f1a9fc in ?? ()
#21 0x000000003a3c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3f50624dd2f1a9fc in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19515):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543abc8320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f543abc8310 in ?? ()
#4 0x00007f543abc8520 in ?? ()
#5 0x00007f543abc8360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f543abc83e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb95cd615b3e000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000000c000000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000feb0757107c in ?? ()
#17 0x00007f543abc83c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543abc8390 in ?? ()
#20 0x3fb95cd615b3e000 in ?? ()
#21 0x000000003abc83e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb95cd615b3e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19514):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543b3c9320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f543b3c9310 in ?? ()
#4 0x00007f543b3c9520 in ?? ()
#5 0x00007f543b3c9360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f543b3c93e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb97a117a63e000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000d5000000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000feb0767127c in ?? ()
#17 0x00007f543b3c93c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543b3c9390 in ?? ()
#20 0x3fb97a117a63e000 in ?? ()
#21 0x000000003b3c93e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb97a117a63e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19513):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543cfb3320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f543cfb3310 in ?? ()
#4 0x00007f543cfb3520 in ?? ()
#5 0x00007f543cfb3360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f543cfb33e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb9821e2c56a000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000010800000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a5 in ?? ()
#16 0x00000feb079ee67c in ?? ()
#17 0x00007f543cfb33c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19510):
#0 0x00007f5446762cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f543efb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19509):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f543e7b60c0 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f543e7b60b0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19508):
#0 0x00007f544da409e2 in ?? ()
#1 0x00007f543d7b2ba0 in ?? ()
#2 0x00007f543d7b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f543d7b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f543d7b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f543d7b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f5453353edc in ?? ()
#12 0x00007f54533438d9 in ?? ()
#13 0x00007f5453347ec6 in ?? ()
#14 0x00007f545334c900 in ?? ()
#15 0x00007f545334c729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f544a6184fe in ?? ()
#18 0x00007f544da366db in ?? ()
#19 0x00000feb07aee484 in ?? ()
#20 0x00007f543d7b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19501):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f543ffb8c80 in ?? ()
#2 0x000000000000037e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f543ffb8c70 in ?? ()
#6 0x00000000000006fc in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19500):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19499):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19498):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f54417bc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19496):
#0 0x00007f544da40d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:27.932780 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID e1bf694ea1724daaa49f0f0746a03eeb and pid 19354
************************ BEGIN STACKS **************************
[New LWP 19359]
[New LWP 19360]
[New LWP 19361]
[New LWP 19362]
[New LWP 19368]
[New LWP 19369]
[New LWP 19370]
[New LWP 19373]
[New LWP 19374]
[New LWP 19375]
[New LWP 19376]
[New LWP 19377]
[New LWP 19378]
[New LWP 19379]
[New LWP 19380]
[New LWP 19381]
[New LWP 19382]
[New LWP 19383]
[New LWP 19384]
[New LWP 19385]
[New LWP 19386]
[New LWP 19388]
[New LWP 19389]
[New LWP 19390]
[New LWP 19391]
[New LWP 19392]
[New LWP 19393]
[New LWP 19394]
[New LWP 19395]
[New LWP 19396]
[New LWP 19397]
[New LWP 19398]
[New LWP 19399]
[New LWP 19400]
[New LWP 19401]
[New LWP 19402]
[New LWP 19403]
[New LWP 19404]
[New LWP 19405]
[New LWP 19406]
[New LWP 19407]
[New LWP 19408]
[New LWP 19409]
[New LWP 19410]
[New LWP 19411]
[New LWP 19412]
[New LWP 19413]
[New LWP 19414]
[New LWP 19415]
[New LWP 19416]
[New LWP 19417]
[New LWP 19418]
[New LWP 19419]
[New LWP 19420]
[New LWP 19421]
[New LWP 19422]
[New LWP 19423]
[New LWP 19424]
[New LWP 19425]
[New LWP 19426]
[New LWP 19427]
[New LWP 19428]
[New LWP 19429]
[New LWP 19430]
[New LWP 19431]
[New LWP 19432]
[New LWP 19433]
[New LWP 19434]
[New LWP 19435]
[New LWP 19436]
[New LWP 19437]
[New LWP 19438]
[New LWP 19439]
[New LWP 19440]
[New LWP 19441]
[New LWP 19442]
[New LWP 19443]
[New LWP 19444]
[New LWP 19445]
[New LWP 19446]
[New LWP 19447]
[New LWP 19448]
[New LWP 19449]
[New LWP 19450]
[New LWP 19451]
[New LWP 19452]
[New LWP 19453]
[New LWP 19454]
[New LWP 19455]
[New LWP 19456]
[New LWP 19457]
[New LWP 19458]
[New LWP 19459]
[New LWP 19460]
[New LWP 19461]
[New LWP 19462]
[New LWP 19463]
[New LWP 19464]
[New LWP 19465]
[New LWP 19466]
[New LWP 19467]
[New LWP 19468]
[New LWP 19469]
[New LWP 19470]
[New LWP 19471]
[New LWP 19472]
[New LWP 19473]
[New LWP 19474]
[New LWP 19475]
[New LWP 19476]
[New LWP 19477]
[New LWP 19478]
[New LWP 19479]
[New LWP 19480]
[New LWP 19481]
[New LWP 19482]
[New LWP 19483]
[New LWP 19484]
[New LWP 19485]
[New LWP 19486]
[New LWP 19487]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f120a610d50 in ?? ()
Id Target Id Frame
* 1 LWP 19354 "kudu" 0x00007f120a610d50 in ?? ()
2 LWP 19359 "kudu" 0x00007f120a60cfb9 in ?? ()
3 LWP 19360 "kudu" 0x00007f120a60cfb9 in ?? ()
4 LWP 19361 "kudu" 0x00007f120a60cfb9 in ?? ()
5 LWP 19362 "kernel-watcher-" 0x00007f120a60cfb9 in ?? ()
6 LWP 19368 "ntp client-1936" 0x00007f120a6109e2 in ?? ()
7 LWP 19369 "file cache-evic" 0x00007f120a60cfb9 in ?? ()
8 LWP 19370 "sq_acceptor" 0x00007f1203332cb9 in ?? ()
9 LWP 19373 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
10 LWP 19374 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
11 LWP 19375 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
12 LWP 19376 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
13 LWP 19377 "MaintenanceMgr " 0x00007f120a60cad3 in ?? ()
14 LWP 19378 "txn-status-mana" 0x00007f120a60cfb9 in ?? ()
15 LWP 19379 "collect_and_rem" 0x00007f120a60cfb9 in ?? ()
16 LWP 19380 "tc-session-exp-" 0x00007f120a60cfb9 in ?? ()
17 LWP 19381 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
18 LWP 19382 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
19 LWP 19383 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
20 LWP 19384 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
21 LWP 19385 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
22 LWP 19386 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
23 LWP 19388 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
24 LWP 19389 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
25 LWP 19390 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
26 LWP 19391 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
27 LWP 19392 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
28 LWP 19393 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
29 LWP 19394 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
30 LWP 19395 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
31 LWP 19396 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
32 LWP 19397 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
33 LWP 19398 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
34 LWP 19399 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
35 LWP 19400 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
36 LWP 19401 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
37 LWP 19402 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
38 LWP 19403 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
39 LWP 19404 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
40 LWP 19405 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
41 LWP 19406 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
42 LWP 19407 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
43 LWP 19408 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
44 LWP 19409 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
45 LWP 19410 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
46 LWP 19411 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
47 LWP 19412 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
48 LWP 19413 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
49 LWP 19414 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
50 LWP 19415 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
51 LWP 19416 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
52 LWP 19417 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
53 LWP 19418 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
54 LWP 19419 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
55 LWP 19420 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
56 LWP 19421 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
57 LWP 19422 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
58 LWP 19423 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
59 LWP 19424 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
60 LWP 19425 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
61 LWP 19426 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
62 LWP 19427 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
63 LWP 19428 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
64 LWP 19429 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
65 LWP 19430 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
66 LWP 19431 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
67 LWP 19432 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
68 LWP 19433 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
69 LWP 19434 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
70 LWP 19435 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
71 LWP 19436 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
72 LWP 19437 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
73 LWP 19438 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
74 LWP 19439 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
75 LWP 19440 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
76 LWP 19441 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
77 LWP 19442 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
78 LWP 19443 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
79 LWP 19444 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
80 LWP 19445 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
81 LWP 19446 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
82 LWP 19447 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
83 LWP 19448 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
84 LWP 19449 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
85 LWP 19450 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
86 LWP 19451 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
87 LWP 19452 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
88 LWP 19453 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
89 LWP 19454 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
90 LWP 19455 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
91 LWP 19456 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
92 LWP 19457 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
93 LWP 19458 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
94 LWP 19459 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
95 LWP 19460 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
96 LWP 19461 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
97 LWP 19462 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
98 LWP 19463 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
99 LWP 19464 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
100 LWP 19465 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
101 LWP 19466 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
102 LWP 19467 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
103 LWP 19468 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
104 LWP 19469 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
105 LWP 19470 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
106 LWP 19471 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
107 LWP 19472 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
108 LWP 19473 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
109 LWP 19474 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
110 LWP 19475 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
111 LWP 19476 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
112 LWP 19477 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
113 LWP 19478 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
114 LWP 19479 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
115 LWP 19480 "rpc worker-1948" 0x00007f120a60cad3 in ?? ()
116 LWP 19481 "rpc worker-1948" 0x00007f120a60cad3 in ?? ()
117 LWP 19482 "diag-logger-194" 0x00007f120a60cfb9 in ?? ()
118 LWP 19483 "result-tracker-" 0x00007f120a60cfb9 in ?? ()
119 LWP 19484 "excess-log-dele" 0x00007f120a60cfb9 in ?? ()
120 LWP 19485 "acceptor-19485" 0x00007f12033410c7 in ?? ()
121 LWP 19486 "heartbeat-19486" 0x00007f120a60cfb9 in ?? ()
122 LWP 19487 "maintenance_sch" 0x00007f120a60cfb9 in ?? ()
Thread 122 (LWP 19487):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11bfeb46e0 in ?? ()
#2 0x00000000000002d7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f11bfeb4730 in ?? ()
#6 0x00000000000005ae in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19486):
#0 0x00007f120a60cfb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000b5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f11c06cc5f0 in ?? ()
#6 0x000000000000016a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19485):
#0 0x00007f12033410c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f11c0ef3d90 in ?? ()
#3 0x00007f11c0ef3dc0 in ?? ()
#4 0x00007f11c0ef3ec0 in ?? ()
#5 0x00007f11c0ef3db0 in ?? ()
#6 0x00007f11c0ef3e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f12071c7980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19484):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11c170df40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffd064c3530 in ?? ()
#5 0x00007f11c170df30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19483):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11c1f26100 in ?? ()
#2 0x00000000000000b6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f11c1f260f0 in ?? ()
#6 0x000000000000016c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19482):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19481):
#0 0x00007f120a60cad3 in ?? ()
#1 0x000000000000002b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f11c2f55e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11c2f55eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f120a60c770 in ?? ()
#10 0x00007f11c2f55eb0 in ?? ()
#11 0x00000fe2385eabd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19480):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000015 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f11c376de90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11c376deb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f120a60c770 in ?? ()
#10 0x00007f11c376deb0 in ?? ()
#11 0x00000fe2386edbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19479):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19478):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19477):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19476):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19475):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19474):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19473):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19472):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19471):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19470):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19469):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19468):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19467):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19466):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19465):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19464):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19463):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19462):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19461):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19460):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19459):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19458):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19457):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19456):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19455):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19454):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19453):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19452):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19451):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19450):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19449):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19448):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19447):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19446):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19445):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19444):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19443):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19442):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19441):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f11d6b0ee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11d6b0eeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19440):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19439):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19438):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19437):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19436):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19435):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19434):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19433):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19432):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19431):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19430):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19429):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19428):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19427):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19426):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19425):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19424):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19423):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19422):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19421):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f11e0cefe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11e0cefeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19420):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19419):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19418):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19417):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19416):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19415):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19414):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19413):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19412):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19411):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19410):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19409):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19408):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19407):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19406):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19405):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19404):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19403):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19402):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19401):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f11eaed0e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11eaed0eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 19400):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19399):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19398):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19397):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19396):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19395):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19394):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19393):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19392):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19391):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19390):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19389):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19388):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19386):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19385):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19384):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19383):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19382):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19381):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19380):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11f50d3cc0 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f11f50d3cb0 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19379):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19378):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11f6111240 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f11f6111230 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19377):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19376):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f7160320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f11f7160310 in ?? ()
#4 0x00007f11f7160520 in ?? ()
#5 0x00007f11f7160360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f11f71603e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb963499566c000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000c4400000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe2bee2407c in ?? ()
#17 0x00007f11f71603c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f7160390 in ?? ()
#20 0x3fb963499566c000 in ?? ()
#21 0x00000000f71603e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb963499566c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19375):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f7977320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f11f7977310 in ?? ()
#4 0x00007f11f7977520 in ?? ()
#5 0x00007f11f7977360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f11f79773e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb94bc99c076000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000003c000000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fe2bef26e7c in ?? ()
#17 0x00007f11f79773c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f7977390 in ?? ()
#20 0x3fb94bc99c076000 in ?? ()
#21 0x00000000f79773e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb94bc99c076000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19374):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f818e320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f11f818e310 in ?? ()
#4 0x00007f11f818e520 in ?? ()
#5 0x00007f11f818e360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f11f818e3e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb94961f3e8a000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x0000000093c00000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fe2bf029c7c in ?? ()
#17 0x00007f11f818e3c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f818e390 in ?? ()
#20 0x3fb94961f3e8a000 in ?? ()
#21 0x00000000f818e3e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb94961f3e8a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19373):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11fa1ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f11fa1ad310 in ?? ()
#4 0x00007f11fa1ad520 in ?? ()
#5 0x00007f11fa1ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f11fa1ad3e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb98aab10228000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000009a800000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe2bf42da7c in ?? ()
#17 0x00007f11fa1ad3c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19370):
#0 0x00007f1203332cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f11fbbb77c8 in ?? ()
#3 0x000060200001ca10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19369):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11fb3b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f11fb3b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19368):
#0 0x00007f120a6109e2 in ?? ()
#1 0x00007f11fabb3ba0 in ?? ()
#2 0x00007f11fabb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f11fabb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f11fabb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f11fabb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f120ff23edc in ?? ()
#12 0x00007f120ff138d9 in ?? ()
#13 0x00007f120ff17ec6 in ?? ()
#14 0x00007f120ff1c900 in ?? ()
#15 0x00007f120ff1c729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f12071e84fe in ?? ()
#18 0x00007f120a6066db in ?? ()
#19 0x00000fe2bf56e684 in ?? ()
#20 0x00007f11fabb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19362):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11fcbb8c80 in ?? ()
#2 0x000000000000038f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f11fcbb8c70 in ?? ()
#6 0x000000000000071e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19361):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19360):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19359):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f11fe3bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19354):
#0 0x00007f120a610d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:29.398350 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID 439cf2177bdb4c19aa8449b6cbd440a7 and pid 19638
************************ BEGIN STACKS **************************
[New LWP 19641]
[New LWP 19642]
[New LWP 19643]
[New LWP 19644]
[New LWP 19650]
[New LWP 19651]
[New LWP 19652]
[New LWP 19655]
[New LWP 19656]
[New LWP 19657]
[New LWP 19658]
[New LWP 19659]
[New LWP 19660]
[New LWP 19662]
[New LWP 19663]
[New LWP 19664]
[New LWP 19665]
[New LWP 19666]
[New LWP 19667]
[New LWP 19668]
[New LWP 19669]
[New LWP 19670]
[New LWP 19671]
[New LWP 19672]
[New LWP 19673]
[New LWP 19674]
[New LWP 19675]
[New LWP 19676]
[New LWP 19677]
[New LWP 19678]
[New LWP 19679]
[New LWP 19680]
[New LWP 19681]
[New LWP 19682]
[New LWP 19683]
[New LWP 19684]
[New LWP 19685]
[New LWP 19686]
[New LWP 19687]
[New LWP 19688]
[New LWP 19689]
[New LWP 19690]
[New LWP 19691]
[New LWP 19692]
[New LWP 19693]
[New LWP 19694]
[New LWP 19695]
[New LWP 19696]
[New LWP 19697]
[New LWP 19698]
[New LWP 19699]
[New LWP 19700]
[New LWP 19701]
[New LWP 19702]
[New LWP 19703]
[New LWP 19704]
[New LWP 19705]
[New LWP 19706]
[New LWP 19707]
[New LWP 19708]
[New LWP 19709]
[New LWP 19712]
[New LWP 19713]
[New LWP 19714]
[New LWP 19715]
[New LWP 19716]
[New LWP 19717]
[New LWP 19718]
[New LWP 19719]
[New LWP 19720]
[New LWP 19721]
[New LWP 19722]
[New LWP 19723]
[New LWP 19724]
[New LWP 19725]
[New LWP 19726]
[New LWP 19727]
[New LWP 19728]
[New LWP 19729]
[New LWP 19730]
[New LWP 19731]
[New LWP 19732]
[New LWP 19733]
[New LWP 19734]
[New LWP 19735]
[New LWP 19736]
[New LWP 19737]
[New LWP 19738]
[New LWP 19739]
[New LWP 19740]
[New LWP 19741]
[New LWP 19742]
[New LWP 19743]
[New LWP 19744]
[New LWP 19745]
[New LWP 19746]
[New LWP 19747]
[New LWP 19748]
[New LWP 19749]
[New LWP 19750]
[New LWP 19751]
[New LWP 19752]
[New LWP 19753]
[New LWP 19754]
[New LWP 19755]
[New LWP 19756]
[New LWP 19757]
[New LWP 19758]
[New LWP 19759]
[New LWP 19760]
[New LWP 19761]
[New LWP 19762]
[New LWP 19764]
[New LWP 19765]
[New LWP 19766]
[New LWP 19767]
[New LWP 19768]
[New LWP 19769]
[New LWP 19770]
[New LWP 19771]
[New LWP 19772]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f5e91e4ad50 in ?? ()
Id Target Id Frame
* 1 LWP 19638 "kudu" 0x00007f5e91e4ad50 in ?? ()
2 LWP 19641 "kudu" 0x00007f5e91e46fb9 in ?? ()
3 LWP 19642 "kudu" 0x00007f5e91e46fb9 in ?? ()
4 LWP 19643 "kudu" 0x00007f5e91e46fb9 in ?? ()
5 LWP 19644 "kernel-watcher-" 0x00007f5e91e46fb9 in ?? ()
6 LWP 19650 "ntp client-1965" 0x00007f5e91e4a9e2 in ?? ()
7 LWP 19651 "file cache-evic" 0x00007f5e91e46fb9 in ?? ()
8 LWP 19652 "sq_acceptor" 0x00007f5e8ab6ccb9 in ?? ()
9 LWP 19655 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
10 LWP 19656 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
11 LWP 19657 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
12 LWP 19658 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
13 LWP 19659 "MaintenanceMgr " 0x00007f5e91e46ad3 in ?? ()
14 LWP 19660 "txn-status-mana" 0x00007f5e91e46fb9 in ?? ()
15 LWP 19662 "collect_and_rem" 0x00007f5e91e46fb9 in ?? ()
16 LWP 19663 "tc-session-exp-" 0x00007f5e91e46fb9 in ?? ()
17 LWP 19664 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
18 LWP 19665 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
19 LWP 19666 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
20 LWP 19667 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
21 LWP 19668 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
22 LWP 19669 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
23 LWP 19670 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
24 LWP 19671 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
25 LWP 19672 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
26 LWP 19673 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
27 LWP 19674 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
28 LWP 19675 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
29 LWP 19676 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
30 LWP 19677 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
31 LWP 19678 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
32 LWP 19679 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
33 LWP 19680 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
34 LWP 19681 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
35 LWP 19682 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
36 LWP 19683 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
37 LWP 19684 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
38 LWP 19685 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
39 LWP 19686 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
40 LWP 19687 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
41 LWP 19688 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
42 LWP 19689 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
43 LWP 19690 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
44 LWP 19691 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
45 LWP 19692 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
46 LWP 19693 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
47 LWP 19694 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
48 LWP 19695 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
49 LWP 19696 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
50 LWP 19697 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
51 LWP 19698 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
52 LWP 19699 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
53 LWP 19700 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
54 LWP 19701 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
55 LWP 19702 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
56 LWP 19703 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
57 LWP 19704 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
58 LWP 19705 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
59 LWP 19706 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
60 LWP 19707 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
61 LWP 19708 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
62 LWP 19709 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
63 LWP 19712 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
64 LWP 19713 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
65 LWP 19714 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
66 LWP 19715 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
67 LWP 19716 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
68 LWP 19717 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
69 LWP 19718 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
70 LWP 19719 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
71 LWP 19720 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
72 LWP 19721 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
73 LWP 19722 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
74 LWP 19723 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
75 LWP 19724 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
76 LWP 19725 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
77 LWP 19726 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
78 LWP 19727 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
79 LWP 19728 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
80 LWP 19729 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
81 LWP 19730 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
82 LWP 19731 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
83 LWP 19732 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
84 LWP 19733 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
85 LWP 19734 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
86 LWP 19735 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
87 LWP 19736 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
88 LWP 19737 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
89 LWP 19738 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
90 LWP 19739 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
91 LWP 19740 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
92 LWP 19741 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
93 LWP 19742 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
94 LWP 19743 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
95 LWP 19744 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
96 LWP 19745 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
97 LWP 19746 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
98 LWP 19747 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
99 LWP 19748 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
100 LWP 19749 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
101 LWP 19750 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
102 LWP 19751 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
103 LWP 19752 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
104 LWP 19753 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
105 LWP 19754 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
106 LWP 19755 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
107 LWP 19756 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
108 LWP 19757 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
109 LWP 19758 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
110 LWP 19759 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
111 LWP 19760 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
112 LWP 19761 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
113 LWP 19762 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
114 LWP 19764 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
115 LWP 19765 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
116 LWP 19766 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
117 LWP 19767 "diag-logger-197" 0x00007f5e91e46fb9 in ?? ()
118 LWP 19768 "result-tracker-" 0x00007f5e91e46fb9 in ?? ()
119 LWP 19769 "excess-log-dele" 0x00007f5e91e46fb9 in ?? ()
120 LWP 19770 "acceptor-19770" 0x00007f5e8ab7b0c7 in ?? ()
121 LWP 19771 "heartbeat-19771" 0x00007f5e91e46fb9 in ?? ()
122 LWP 19772 "maintenance_sch" 0x00007f5e91e46fb9 in ?? ()
Thread 122 (LWP 19772):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e46e966e0 in ?? ()
#2 0x00000000000002cc in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00006160000157a4 in ?? ()
#5 0x00007f5e46e96730 in ?? ()
#6 0x0000000000000599 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f5e46e96750 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f5e8e8d6380 in ?? ()
#12 0x00007f5e46e970d0 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 121 (LWP 19771):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000b3 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f5e476ae5f0 in ?? ()
#6 0x0000000000000167 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f5e476ae610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f5e476ae690 in ?? ()
#12 0x00000febc8ed5cd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 19770):
#0 0x00007f5e8ab7b0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f5e47ed5d90 in ?? ()
#3 0x00007f5e47ed5dc0 in ?? ()
#4 0x00007f5e47ed5ec0 in ?? ()
#5 0x00007f5e47ed5db0 in ?? ()
#6 0x00007f5e47ed5e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f5e8ea01980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19769):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e486eef40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe3466e6d0 in ?? ()
#5 0x00007f5e486eef30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19768):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e48f08100 in ?? ()
#2 0x00000000000000b3 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f5e48f080f0 in ?? ()
#6 0x0000000000000166 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19767):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19766):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000001b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b684c in ?? ()
#4 0x00007f5e49f37e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e49f37eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b6800 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e49f37eb0 in ?? ()
#11 0x00000febc93e6fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19765):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f5e4a74fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e4a74feb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e4a74feb0 in ?? ()
#11 0x00000febc94e9fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19764):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19762):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19761):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19760):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19759):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19758):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19757):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19756):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19755):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19754):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19753):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19752):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19751):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19750):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19749):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19748):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19747):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19746):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19745):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19744):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19743):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19742):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19741):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19740):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19739):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19738):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19737):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19736):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19735):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19734):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19733):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19732):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19731):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19730):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19729):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19728):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19727):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19726):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19725):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000196d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f5e5e308e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5e308eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5e308eb0 in ?? ()
#11 0x00000febcbc611d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 19724):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000203f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f5e5eb20e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5eb20eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5eb20eb0 in ?? ()
#11 0x00000febcbd641d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 19723):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x00000000000005e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a008c in ?? ()
#4 0x00007f5e5f338e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5f338eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a0040 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5f338eb0 in ?? ()
#11 0x00000febcbe671d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 19722):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19721):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19720):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19719):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19718):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19717):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19716):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19715):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19714):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19713):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19712):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19709):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19708):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19707):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19706):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19705):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19704):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19703):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f5e67cd1e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e67cd1eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19702):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19701):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19700):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19699):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19698):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19697):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19696):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19695):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19694):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19693):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19692):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19691):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19690):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19689):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19688):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19687):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19686):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19685):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19684):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19683):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000001c5d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f5e71eb2e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e71eb2eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e71eb2eb0 in ?? ()
#11 0x00000febce3d65d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 19682):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000294 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f5e726cae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e726caeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 19681):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000002297 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f5e72ee2e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e72ee2eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e72ee2eb0 in ?? ()
#11 0x00000febce5dc5d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 19680):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000001846 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008fff8 in ?? ()
#4 0x00007f5e736fae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e736faeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 19679):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000196e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008d008 in ?? ()
#4 0x00007f5e73f12e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e73f12eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 31 (LWP 19678):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x00000000000020a6 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000086808 in ?? ()
#4 0x00007f5e7472ae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e7472aeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 30 (LWP 19677):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000002341 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008000c in ?? ()
#4 0x00007f5e74f42e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e74f42eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00007ffc0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e74f42eb0 in ?? ()
#11 0x00000febce9e85d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 29 (LWP 19676):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19675):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19674):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19673):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19672):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19671):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19670):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19669):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19668):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19667):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19666):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19665):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19664):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19663):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e7c0b7cc0 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007f5e7c0b7cb0 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19662):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19660):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e7d94d240 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f5e7d94d230 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19659):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000003 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x00007f5e7e174ba4 in ?? ()
#4 0x00007f5e7e1748b0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e7e1748d0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00006160000159c0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e7e1748d0 in ?? ()
#11 0x00007f5e7e1748d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 12 (LWP 19658):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7e99c320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f5e7e99c310 in ?? ()
#4 0x00007f5e7e99c520 in ?? ()
#5 0x00007f5e7e99c360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f5e7e99c3e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb949ab08c8a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000055000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000fec4fd2b87c in ?? ()
#17 0x00007f5e7e99c3c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7e99c390 in ?? ()
#20 0x3fb949ab08c8a000 in ?? ()
#21 0x000000007e99c3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb949ab08c8a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19657):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7f1b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f5e7f1b3310 in ?? ()
#4 0x00007f5e7f1b3520 in ?? ()
#5 0x00007f5e7f1b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f5e7f1b33e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb960a45abfe000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000026c00000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fec4fe2e67c in ?? ()
#17 0x00007f5e7f1b33c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7f1b3390 in ?? ()
#20 0x3fb960a45abfe000 in ?? ()
#21 0x000000007f1b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb960a45abfe000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19656):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7f9b4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f5e7f9b4310 in ?? ()
#4 0x00007f5e7f9b4520 in ?? ()
#5 0x00007f5e7f9b4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f5e7f9b43e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb50896d6e28000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000001a000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fec4ff2e87c in ?? ()
#17 0x00007f5e7f9b43c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7f9b4390 in ?? ()
#20 0x3fb50896d6e28000 in ?? ()
#21 0x000000007f9b43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb50896d6e28000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19655):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e819ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f5e819ad310 in ?? ()
#4 0x00007f5e819ad520 in ?? ()
#5 0x00007f5e819ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f5e819ad3e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb9387ddc372000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000003000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fec5032da7c in ?? ()
#17 0x00007f5e819ad3c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19652):
#0 0x00007f5e8ab6ccb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f5e833b77c8 in ?? ()
#3 0x000060200001ccd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19651):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e82bb60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f5e82bb60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19650):
#0 0x00007f5e91e4a9e2 in ?? ()
#1 0x00007f5e823b3ba0 in ?? ()
#2 0x00007f5e823b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f5e823b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f5e823b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f5e823b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f5e9775dedc in ?? ()
#12 0x00007f5e9774d8d9 in ?? ()
#13 0x00007f5e97751ec6 in ?? ()
#14 0x00007f5e97756900 in ?? ()
#15 0x00007f5e97756729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f5e8ea224fe in ?? ()
#18 0x00007f5e91e406db in ?? ()
#19 0x00000fec5046e684 in ?? ()
#20 0x00007f5e823b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19644):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e843b8c80 in ?? ()
#2 0x0000000000000383 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f5e843b8c70 in ?? ()
#6 0x0000000000000706 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19643):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19642):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19641):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f5e85bbc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19638):
#0 0x00007f5e91e4ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:30.931772 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 3 with UUID d33fd96ef7d44ad8805718466ca9447f and pid 18856
************************ BEGIN STACKS **************************
[New LWP 18859]
[New LWP 18860]
[New LWP 18862]
[New LWP 18863]
[New LWP 18870]
[New LWP 18871]
[New LWP 18872]
[New LWP 18875]
[New LWP 18876]
[New LWP 18877]
[New LWP 18878]
[New LWP 18879]
[New LWP 18880]
[New LWP 18881]
[New LWP 18884]
[New LWP 18885]
[New LWP 18886]
[New LWP 18887]
[New LWP 18888]
[New LWP 18889]
[New LWP 18890]
[New LWP 18891]
[New LWP 18892]
[New LWP 18893]
[New LWP 18894]
[New LWP 18895]
[New LWP 18896]
[New LWP 18897]
[New LWP 18898]
[New LWP 18899]
[New LWP 18900]
[New LWP 18901]
[New LWP 18902]
[New LWP 18903]
[New LWP 18904]
[New LWP 18905]
[New LWP 18906]
[New LWP 18907]
[New LWP 18908]
[New LWP 18909]
[New LWP 18910]
[New LWP 18911]
[New LWP 18912]
[New LWP 18913]
[New LWP 18914]
[New LWP 18915]
[New LWP 18916]
[New LWP 18917]
[New LWP 18918]
[New LWP 18919]
[New LWP 18920]
[New LWP 18921]
[New LWP 18922]
[New LWP 18923]
[New LWP 18924]
[New LWP 18925]
[New LWP 18926]
[New LWP 18927]
[New LWP 18928]
[New LWP 18929]
[New LWP 18930]
[New LWP 18931]
[New LWP 18932]
[New LWP 18933]
[New LWP 18934]
[New LWP 18935]
[New LWP 18936]
[New LWP 18937]
[New LWP 18938]
[New LWP 18939]
[New LWP 18940]
[New LWP 18941]
[New LWP 18942]
[New LWP 18943]
[New LWP 18944]
[New LWP 18945]
[New LWP 18946]
[New LWP 18947]
[New LWP 18948]
[New LWP 18949]
[New LWP 18950]
[New LWP 18951]
[New LWP 18952]
[New LWP 18953]
[New LWP 18954]
[New LWP 18955]
[New LWP 18956]
[New LWP 18957]
[New LWP 18959]
[New LWP 18960]
[New LWP 18961]
[New LWP 18962]
[New LWP 18963]
[New LWP 18964]
[New LWP 18967]
[New LWP 18968]
[New LWP 18969]
[New LWP 18970]
[New LWP 18971]
[New LWP 18972]
[New LWP 18973]
[New LWP 18974]
[New LWP 18975]
[New LWP 18976]
[New LWP 18977]
[New LWP 18978]
[New LWP 18979]
[New LWP 18980]
[New LWP 18981]
[New LWP 18982]
[New LWP 18983]
[New LWP 18984]
[New LWP 18985]
[New LWP 18986]
[New LWP 18987]
[New LWP 18988]
[New LWP 18989]
[New LWP 18991]
[New LWP 18992]
[New LWP 18993]
[New LWP 18997]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f148abc7d50 in ?? ()
Id Target Id Frame
* 1 LWP 18856 "kudu" 0x00007f148abc7d50 in ?? ()
2 LWP 18859 "kudu" 0x00007f148abc3fb9 in ?? ()
3 LWP 18860 "kudu" 0x00007f148abc3fb9 in ?? ()
4 LWP 18862 "kudu" 0x00007f148abc3fb9 in ?? ()
5 LWP 18863 "kernel-watcher-" 0x00007f148abc3fb9 in ?? ()
6 LWP 18870 "ntp client-1887" 0x00007f148abc79e2 in ?? ()
7 LWP 18871 "file cache-evic" 0x00007f148abc3fb9 in ?? ()
8 LWP 18872 "sq_acceptor" 0x00007f14838e9cb9 in ?? ()
9 LWP 18875 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
10 LWP 18876 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
11 LWP 18877 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
12 LWP 18878 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
13 LWP 18879 "MaintenanceMgr " 0x00007f148abc3ad3 in ?? ()
14 LWP 18880 "txn-status-mana" 0x00007f148abc3fb9 in ?? ()
15 LWP 18881 "collect_and_rem" 0x00007f148abc3fb9 in ?? ()
16 LWP 18884 "tc-session-exp-" 0x00007f148abc3fb9 in ?? ()
17 LWP 18885 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
18 LWP 18886 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
19 LWP 18887 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
20 LWP 18888 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
21 LWP 18889 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
22 LWP 18890 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
23 LWP 18891 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
24 LWP 18892 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
25 LWP 18893 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
26 LWP 18894 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
27 LWP 18895 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
28 LWP 18896 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
29 LWP 18897 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
30 LWP 18898 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
31 LWP 18899 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
32 LWP 18900 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
33 LWP 18901 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
34 LWP 18902 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
35 LWP 18903 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
36 LWP 18904 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
37 LWP 18905 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
38 LWP 18906 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
39 LWP 18907 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
40 LWP 18908 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
41 LWP 18909 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
42 LWP 18910 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
43 LWP 18911 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
44 LWP 18912 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
45 LWP 18913 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
46 LWP 18914 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
47 LWP 18915 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
48 LWP 18916 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
49 LWP 18917 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
50 LWP 18918 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
51 LWP 18919 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
52 LWP 18920 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
53 LWP 18921 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
54 LWP 18922 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
55 LWP 18923 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
56 LWP 18924 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
57 LWP 18925 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
58 LWP 18926 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
59 LWP 18927 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
60 LWP 18928 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
61 LWP 18929 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
62 LWP 18930 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
63 LWP 18931 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
64 LWP 18932 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
65 LWP 18933 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
66 LWP 18934 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
67 LWP 18935 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
68 LWP 18936 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
69 LWP 18937 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
70 LWP 18938 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
71 LWP 18939 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
72 LWP 18940 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
73 LWP 18941 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
74 LWP 18942 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
75 LWP 18943 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
76 LWP 18944 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
77 LWP 18945 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
78 LWP 18946 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
79 LWP 18947 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
80 LWP 18948 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
81 LWP 18949 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
82 LWP 18950 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
83 LWP 18951 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
84 LWP 18952 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
85 LWP 18953 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
86 LWP 18954 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
87 LWP 18955 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
88 LWP 18956 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
89 LWP 18957 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
90 LWP 18959 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
91 LWP 18960 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
92 LWP 18961 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
93 LWP 18962 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
94 LWP 18963 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
95 LWP 18964 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
96 LWP 18967 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
97 LWP 18968 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
98 LWP 18969 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
99 LWP 18970 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
100 LWP 18971 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
101 LWP 18972 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
102 LWP 18973 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
103 LWP 18974 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
104 LWP 18975 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
105 LWP 18976 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
106 LWP 18977 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
107 LWP 18978 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
108 LWP 18979 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
109 LWP 18980 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
110 LWP 18981 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
111 LWP 18982 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
112 LWP 18983 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
113 LWP 18984 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
114 LWP 18985 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
115 LWP 18986 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
116 LWP 18987 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
117 LWP 18988 "diag-logger-189" 0x00007f148abc3fb9 in ?? ()
118 LWP 18989 "result-tracker-" 0x00007f148abc3fb9 in ?? ()
119 LWP 18991 "excess-log-dele" 0x00007f148abc3fb9 in ?? ()
120 LWP 18992 "acceptor-18992" 0x00007f14838f80c7 in ?? ()
121 LWP 18993 "heartbeat-18993" 0x00007f148abc3fb9 in ?? ()
122 LWP 18997 "maintenance_sch" 0x00007f148abc3fb9 in ?? ()
Thread 122 (LWP 18997):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f144046b6e0 in ?? ()
#2 0x0000000000000326 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f144046b730 in ?? ()
#6 0x000000000000064c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18993):
#0 0x00007f148abc3fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f1440c835f0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18992):
#0 0x00007f14838f80c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f14414aad90 in ?? ()
#3 0x00007f14414aadc0 in ?? ()
#4 0x00007f14414aaec0 in ?? ()
#5 0x00007f14414aadb0 in ?? ()
#6 0x00007f14414aae20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f148777e980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18991):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f1441cc4f40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffff38c78f0 in ?? ()
#5 0x00007f1441cc4f30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18989):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14424dd100 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f14424dd0f0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18988):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18987):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x000000000000001f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f144350ce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f144350ceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f144350ceb0 in ?? ()
#11 0x00000fe2886a19d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 18986):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f1443d24e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1443d24eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f1443d24eb0 in ?? ()
#11 0x00000fe2887a49d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 18985):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18984):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18983):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18982):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18981):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18980):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18979):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18978):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18977):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18976):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18975):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18974):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18973):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18972):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18971):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18970):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18969):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18968):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18967):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18964):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18963):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18962):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18961):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18960):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18959):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18957):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18956):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18955):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18954):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18953):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18952):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18951):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18950):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18949):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18948):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18947):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18946):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18945):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18944):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f14578dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f14578ddeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f14578ddeb0 in ?? ()
#11 0x00000fe28af1bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18943):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 18942):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18941):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18940):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18939):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18938):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18937):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18936):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18935):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18934):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18933):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18932):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18931):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18930):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18929):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18928):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18927):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18926):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18925):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18924):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f1461abee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1461abeeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f1461abeeb0 in ?? ()
#11 0x00000fe28c357dd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18923):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18922):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18921):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18920):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18919):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18918):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18917):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18916):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18915):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18914):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18913):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18912):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18911):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18910):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18909):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18908):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18907):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18906):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18905):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18904):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f146bc99e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f146bc99eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f146bc99eb0 in ?? ()
#11 0x00000fe28d7933d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18903):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18902):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18901):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18900):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18899):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18898):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18897):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18896):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18895):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18894):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18893):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18892):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18891):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18890):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18889):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18888):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18887):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18886):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18885):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18884):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14754c2cc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f14754c2cb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18881):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18880):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14764c4240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f14764c4230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18879):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18878):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f14774c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f14774c7310 in ?? ()
#4 0x00007f14774c7520 in ?? ()
#5 0x00007f14774c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f14774c73e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb95fd068734000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000024000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f839e in ?? ()
#16 0x00000fe30ee90e7c in ?? ()
#17 0x00007f14774c73c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f14774c7390 in ?? ()
#20 0x3fb95fd068734000 in ?? ()
#21 0x00000000774c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb95fd068734000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18877):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f147a0b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f147a0b3310 in ?? ()
#4 0x00007f147a0b3520 in ?? ()
#5 0x00007f147a0b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f147a0b33e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb96bdd088c2000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000e6000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe30f40e67c in ?? ()
#17 0x00007f147a0b33c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f147a0b3390 in ?? ()
#20 0x3fb96bdd088c2000 in ?? ()
#21 0x000000007a0b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb96bdd088c2000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18876):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f1477cc8320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f1477cc8310 in ?? ()
#4 0x00007f1477cc8520 in ?? ()
#5 0x00007f1477cc8360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f1477cc83e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb9799b07f1c000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000c7800000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe30ef9107c in ?? ()
#17 0x00007f1477cc83c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f1477cc8390 in ?? ()
#20 0x3fb9799b07f1c000 in ?? ()
#21 0x0000000077cc83e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb9799b07f1c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18875):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f14784c9320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f14784c9310 in ?? ()
#4 0x00007f14784c9520 in ?? ()
#5 0x00007f14784c9360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f14784c93e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb952c959604000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000b1000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000fe30f09127c in ?? ()
#17 0x00007f14784c93c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18872):
#0 0x00007f14838e9cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f147c0b77c8 in ?? ()
#3 0x000060200001dc90 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18871):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f147b8b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f147b8b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18870):
#0 0x00007f148abc79e2 in ?? ()
#1 0x00007f147a8b2ba0 in ?? ()
#2 0x00007f147a8b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f147a8b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f147a8b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f147a8b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f14904daedc in ?? ()
#12 0x00007f14904ca8d9 in ?? ()
#13 0x00007f14904ceec6 in ?? ()
#14 0x00007f14904d3900 in ?? ()
#15 0x00007f14904d3729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f148779f4fe in ?? ()
#18 0x00007f148abbd6db in ?? ()
#19 0x00000fe30f50e484 in ?? ()
#20 0x00007f147a8b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18863):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f147d0b8c80 in ?? ()
#2 0x00000000000003f5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f147d0b8c70 in ?? ()
#6 0x00000000000007ea in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18862):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18860):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18859):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f147e8bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18856):
#0 0x00007f148abc7d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:31.823567 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 4 with UUID 4f219d2e8b954461973a8ca8680a6383 and pid 20075
************************ BEGIN STACKS **************************
[New LWP 20078]
[New LWP 20079]
[New LWP 20080]
[New LWP 20081]
[New LWP 20087]
[New LWP 20088]
[New LWP 20089]
[New LWP 20092]
[New LWP 20093]
[New LWP 20094]
[New LWP 20095]
[New LWP 20096]
[New LWP 20097]
[New LWP 20099]
[New LWP 20100]
[New LWP 20101]
[New LWP 20102]
[New LWP 20103]
[New LWP 20104]
[New LWP 20105]
[New LWP 20106]
[New LWP 20107]
[New LWP 20108]
[New LWP 20109]
[New LWP 20110]
[New LWP 20111]
[New LWP 20112]
[New LWP 20113]
[New LWP 20114]
[New LWP 20115]
[New LWP 20116]
[New LWP 20117]
[New LWP 20118]
[New LWP 20119]
[New LWP 20120]
[New LWP 20121]
[New LWP 20122]
[New LWP 20123]
[New LWP 20124]
[New LWP 20125]
[New LWP 20126]
[New LWP 20127]
[New LWP 20128]
[New LWP 20129]
[New LWP 20130]
[New LWP 20131]
[New LWP 20132]
[New LWP 20133]
[New LWP 20134]
[New LWP 20135]
[New LWP 20136]
[New LWP 20137]
[New LWP 20139]
[New LWP 20141]
[New LWP 20142]
[New LWP 20143]
[New LWP 20144]
[New LWP 20146]
[New LWP 20147]
[New LWP 20148]
[New LWP 20149]
[New LWP 20150]
[New LWP 20151]
[New LWP 20152]
[New LWP 20153]
[New LWP 20154]
[New LWP 20155]
[New LWP 20156]
[New LWP 20157]
[New LWP 20158]
[New LWP 20159]
[New LWP 20160]
[New LWP 20161]
[New LWP 20162]
[New LWP 20163]
[New LWP 20164]
[New LWP 20165]
[New LWP 20166]
[New LWP 20167]
[New LWP 20168]
[New LWP 20169]
[New LWP 20170]
[New LWP 20171]
[New LWP 20172]
[New LWP 20173]
[New LWP 20174]
[New LWP 20175]
[New LWP 20176]
[New LWP 20178]
[New LWP 20179]
[New LWP 20180]
[New LWP 20181]
[New LWP 20182]
[New LWP 20183]
[New LWP 20184]
[New LWP 20185]
[New LWP 20186]
[New LWP 20187]
[New LWP 20188]
[New LWP 20189]
[New LWP 20190]
[New LWP 20191]
[New LWP 20192]
[New LWP 20193]
[New LWP 20194]
[New LWP 20195]
[New LWP 20196]
[New LWP 20197]
[New LWP 20198]
[New LWP 20199]
[New LWP 20200]
[New LWP 20201]
[New LWP 20202]
[New LWP 20203]
[New LWP 20204]
[New LWP 20205]
[New LWP 20206]
[New LWP 20207]
[New LWP 20209]
[New LWP 20210]
[New LWP 20211]
[New LWP 21677]
[New LWP 21695]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007ff5b495dd50 in ?? ()
Id Target Id Frame
* 1 LWP 20075 "kudu" 0x00007ff5b495dd50 in ?? ()
2 LWP 20078 "kudu" 0x00007ff5b4959fb9 in ?? ()
3 LWP 20079 "kudu" 0x00007ff5b4959fb9 in ?? ()
4 LWP 20080 "kudu" 0x00007ff5b4959fb9 in ?? ()
5 LWP 20081 "kernel-watcher-" 0x00007ff5b4959fb9 in ?? ()
6 LWP 20087 "ntp client-2008" 0x00007ff5b495d9e2 in ?? ()
7 LWP 20088 "file cache-evic" 0x00007ff5b4959fb9 in ?? ()
8 LWP 20089 "sq_acceptor" 0x00007ff5ad67fcb9 in ?? ()
9 LWP 20092 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
10 LWP 20093 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
11 LWP 20094 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
12 LWP 20095 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
13 LWP 20096 "MaintenanceMgr " 0x000000000096b06f in __sanitizer::theDepot ()
14 LWP 20097 "txn-status-mana" 0x00007ff5b4959fb9 in ?? ()
15 LWP 20099 "collect_and_rem" 0x00007ff5b4959fb9 in ?? ()
16 LWP 20100 "tc-session-exp-" 0x00007ff5b4959fb9 in ?? ()
17 LWP 20101 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
18 LWP 20102 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
19 LWP 20103 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
20 LWP 20104 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
21 LWP 20105 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
22 LWP 20106 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
23 LWP 20107 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
24 LWP 20108 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
25 LWP 20109 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
26 LWP 20110 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
27 LWP 20111 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
28 LWP 20112 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
29 LWP 20113 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
30 LWP 20114 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
31 LWP 20115 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
32 LWP 20116 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
33 LWP 20117 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
34 LWP 20118 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
35 LWP 20119 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
36 LWP 20120 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
37 LWP 20121 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
38 LWP 20122 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
39 LWP 20123 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
40 LWP 20124 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
41 LWP 20125 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
42 LWP 20126 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
43 LWP 20127 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
44 LWP 20128 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
45 LWP 20129 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
46 LWP 20130 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
47 LWP 20131 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
48 LWP 20132 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
49 LWP 20133 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
50 LWP 20134 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
51 LWP 20135 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
52 LWP 20136 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
53 LWP 20137 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
54 LWP 20139 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
55 LWP 20141 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
56 LWP 20142 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
57 LWP 20143 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
58 LWP 20144 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
59 LWP 20146 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
60 LWP 20147 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
61 LWP 20148 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
62 LWP 20149 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
63 LWP 20150 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
64 LWP 20151 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
65 LWP 20152 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
66 LWP 20153 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
67 LWP 20154 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
68 LWP 20155 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
69 LWP 20156 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
70 LWP 20157 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
71 LWP 20158 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
72 LWP 20159 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
73 LWP 20160 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
74 LWP 20161 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
75 LWP 20162 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
76 LWP 20163 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
77 LWP 20164 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
78 LWP 20165 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
79 LWP 20166 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
80 LWP 20167 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
81 LWP 20168 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
82 LWP 20169 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
83 LWP 20170 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
84 LWP 20171 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
85 LWP 20172 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
86 LWP 20173 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
87 LWP 20174 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
88 LWP 20175 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
89 LWP 20176 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
90 LWP 20178 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
91 LWP 20179 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
92 LWP 20180 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
93 LWP 20181 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
94 LWP 20182 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
95 LWP 20183 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
96 LWP 20184 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
97 LWP 20185 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
98 LWP 20186 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
99 LWP 20187 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
100 LWP 20188 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
101 LWP 20189 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
102 LWP 20190 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
103 LWP 20191 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
104 LWP 20192 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
105 LWP 20193 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
106 LWP 20194 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
107 LWP 20195 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
108 LWP 20196 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
109 LWP 20197 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
110 LWP 20198 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
111 LWP 20199 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
112 LWP 20200 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
113 LWP 20201 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
114 LWP 20202 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
115 LWP 20203 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
116 LWP 20204 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
117 LWP 20205 "diag-logger-202" 0x00007ff5b4959fb9 in ?? ()
118 LWP 20206 "result-tracker-" 0x00007ff5b4959fb9 in ?? ()
119 LWP 20207 "excess-log-dele" 0x00007ff5b4959fb9 in ?? ()
120 LWP 20209 "acceptor-20209" 0x00007ff5ad68e0c7 in ?? ()
121 LWP 20210 "heartbeat-20210" 0x00007ff5b4959fb9 in ?? ()
122 LWP 20211 "maintenance_sch" 0x00007ff5b4959fb9 in ?? ()
123 LWP 21677 "raft [worker]-2" 0x00007ff5b4959fb9 in ?? ()
124 LWP 21695 "raft [worker]-2" 0x00007ff5b4959fb9 in ?? ()
Thread 124 (LWP 21695):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007ff55518fba4 in ?? ()
#5 0x00007ff55518f830 in ?? ()
#6 0x000000000000000f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ff55518f850 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 123 (LWP 21677):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ff555990ba0 in ?? ()
#5 0x00007ff555990830 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 20211):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56902e6e0 in ?? ()
#2 0x0000000000000257 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007ff56902e730 in ?? ()
#6 0x00000000000004ae in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 20210):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000099 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007ff5698465f0 in ?? ()
#6 0x0000000000000133 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ff569846610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007ff569846690 in ?? ()
#12 0x00000ffead308cd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 20209):
#0 0x00007ff5ad68e0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007ff56a05dd90 in ?? ()
#3 0x00007ff56a05ddc0 in ?? ()
#4 0x00007ff56a05dec0 in ?? ()
#5 0x00007ff56a05ddb0 in ?? ()
#6 0x00007ff56a05de20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007ff5b1514980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 20207):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56a876f40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffff4670c90 in ?? ()
#5 0x00007ff56a876f30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 20206):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56b08f100 in ?? ()
#2 0x0000000000000096 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007ff56b08f0f0 in ?? ()
#6 0x000000000000012c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 20205):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 20204):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000000f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00034688c in ?? ()
#4 0x00007ff56c0cfe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff56c0cfeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000346840 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff56c0cfeb0 in ?? ()
#11 0x00000ffead819fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 20203):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0003367cc in ?? ()
#4 0x00007ff56c8e7e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff56c8e7eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000336780 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff56c8e7eb0 in ?? ()
#11 0x00000ffead91cfd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 20202):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 20201):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 20200):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 20199):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 20198):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 20197):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 20196):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 20195):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 20194):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 20193):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 20192):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 20191):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 20190):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 20189):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 20188):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 20187):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 20186):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 20185):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 20184):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 20183):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 20182):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 20181):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 20180):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 20179):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 20178):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 20176):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 20175):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 20174):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 20173):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 20172):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 20171):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 20170):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 20169):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 20168):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 20167):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 20166):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 20165):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 20164):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 20163):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000003c2 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007ff580604e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff580604eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 20162):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000002e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007ff580e1be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff580e1beb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff580e1beb0 in ?? ()
#11 0x00000ffeb01c37d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 20161):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 20160):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 20159):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 20158):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 20157):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 20156):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 20155):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 20154):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 20153):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 20152):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 20151):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 20150):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 20149):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 20148):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 20147):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 20146):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 20144):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 20143):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 20142):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000000d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007ff58a7e4e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff58a7e4eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff58a7e4eb0 in ?? ()
#11 0x00000ffeb14fc9d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 20141):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 20139):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 20137):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 20136):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 20135):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 20134):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 20133):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 20132):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 20131):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 20130):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 20129):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 20128):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 20127):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 20126):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 20125):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 20124):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 20123):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 20122):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 20121):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 20120):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000006c5 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007ff5949c5e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5949c5eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff5949c5eb0 in ?? ()
#11 0x00000ffeb2938bd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 20119):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000097e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007ff5951dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5951ddeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 20118):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000a29 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007ff5959f5e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5959f5eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff5959f5eb0 in ?? ()
#11 0x00000ffeb2b3ebd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 20117):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20116):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20115):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20114):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20113):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20112):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20111):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20110):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20109):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20108):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20107):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20106):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20105):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20104):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20103):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20102):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20101):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20100):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff59ebc6cc0 in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007ff59ebc6cb0 in ?? ()
#6 0x000000000000001e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20099):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20097):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a045b240 in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007ff5a045b230 in ?? ()
#6 0x000000000000001e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20096):
#0 0x000000000096b06f in __sanitizer::theDepot ()
#1 0x00007ff5a0c7ea20 in ?? ()
#2 0x0000000000a5d66c in __sanitizer::theDepot ()
#3 0x0000000041b58ab3 in ?? ()
I20250624 07:58:32.829517 21699 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.829730 21699 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.830974 21699 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:58:32.852766 21043 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:58:32.891990 21703 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.892247 21703 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.893267 21703 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:32.894335 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:32.894752 19723 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 2.
I20250624 07:58:32.895587 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:58:32.896028 21703 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250624 07:58:32.896157 21703 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.896262 21703 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:32.901054 21703 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.901824 21703 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 election: Requested vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:32.902967 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
I20250624 07:58:32.903340 19723 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:32.906291 19723 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 3.
I20250624 07:58:32.907157 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:58:32.907734 21703 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 FOLLOWER]: Leader election won for term 3
I20250624 07:58:32.908365 21703 raft_consensus.cc:695] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 LEADER]: Becoming Leader. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Running, Role: LEADER
I20250624 07:58:32.909085 21703 consensus_queue.cc:237] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15871, Committed index: 15871, Last appended: 2.15871, Last appended by leader: 15871, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.914839 17300 catalog_manager.cc:5582] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 reported cstate change: term changed from 2 to 3, leader changed from 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69) to 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73). New cstate: current_term: 3 leader_uuid: "6cbd668c726a4b1298e789270a0031e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } health_report { overall_health: HEALTHY } } }
I20250624 07:58:33.006711 19723 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Refusing update from remote peer 6cbd668c726a4b1298e789270a0031e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 15871. Preceding OpId from leader: term: 3 index: 15872. (index mismatch)
I20250624 07:58:33.007638 21703 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15872, Last known committed idx: 15871, Time since last communication: 0.000s
#4 0x00000000004518ac in kudu::itest::MaintenanceModeRF3ITest_TestMaintenanceModeDoesntObstructMove_Test::TestBody()::$_2::operator()() const (this=<optimized out>) at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:353
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Thread 12 (LWP 20095):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a14aa320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007ff5a14aa310 in ?? ()
#4 0x00007ff5a14aa520 in ?? ()
#5 0x00007ff5a14aa360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007ff5a14aa3e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fab99f70c918000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000d6000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fff3428d47c in ?? ()
#17 0x00007ff5a14aa3c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a14aa390 in ?? ()
#20 0x3fab99f70c918000 in ?? ()
#21 0x00000000a14aa3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fab99f70c918000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20094):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a1cb3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007ff5a1cb3310 in ?? ()
#4 0x00007ff5a1cb3520 in ?? ()
#5 0x00007ff5a1cb3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007ff5a1cb33e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb982c1b35b6000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000018000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fff3438e67c in ?? ()
#17 0x00007ff5a1cb33c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a1cb3390 in ?? ()
#20 0x3fb982c1b35b6000 in ?? ()
#21 0x00000000a1cb33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb982c1b35b6000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20093):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a24b4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007ff5a24b4310 in ?? ()
#4 0x00007ff5a24b4520 in ?? ()
#5 0x00007ff5a24b4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007ff5a24b43e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb0bc0deb7e8000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000f0000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a3 in ?? ()
#16 0x00000fff3448e87c in ?? ()
#17 0x00007ff5a24b43c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a24b4390 in ?? ()
#20 0x3fb0bc0deb7e8000 in ?? ()
#21 0x00000000a24b43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb0bc0deb7e8000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20092):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a44ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007ff5a44ad310 in ?? ()
#4 0x00007ff5a44ad520 in ?? ()
#5 0x00007ff5a44ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007ff5a44ad3e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb968fc768c2000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000001d800000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fff3488da7c in ?? ()
#17 0x00007ff5a44ad3c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20089):
#0 0x00007ff5ad67fcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007ff5a5eb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20088):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a56b60c0 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007ff5a56b60b0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20087):
#0 0x00007ff5b495d9e2 in ?? ()
#1 0x00007ff5a4eb3ba0 in ?? ()
#2 0x00007ff5a4eb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007ff5a4eb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007ff5a4eb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007ff5a4eb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007ff5ba270edc in ?? ()
#12 0x00007ff5ba2608d9 in ?? ()
#13 0x00007ff5ba264ec6 in ?? ()
#14 0x00007ff5ba269900 in ?? ()
#15 0x00007ff5ba269729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007ff5b15354fe in ?? ()
#18 0x00007ff5b49536db in ?? ()
#19 0x00000fff349ce684 in ?? ()
#20 0x00007ff5a4eb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20081):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a6eb8c80 in ?? ()
#2 0x00000000000002f2 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007ff5a6eb8c70 in ?? ()
#6 0x00000000000005e4 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20080):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20079):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20078):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007ff5a86bc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20075):
#0 0x00007ff5b495dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
I20250624 07:58:33.041600 19723 raft_consensus.cc:1238] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Rejecting Update request from peer 4f219d2e8b954461973a8ca8680a6383 for earlier term 2. Current term is 3. Ops: []
I20250624 07:58:33.042953 21695 consensus_queue.cc:1046] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: INVALID_TERM, Last received: 2.15871, Next index: 15872, Last known committed idx: 15872, Time since last communication: 0.000s
************************* END STACKS ***************************
I20250624 07:58:33.060803 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 5 with UUID bede5abcb6174794bd2283843157bcef and pid 18718
I20250624 07:58:33.072947 21714 raft_consensus.cc:3053] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 LEADER]: Stepping down as leader of term 2
I20250624 07:58:33.073102 21714 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f219d2e8b954461973a8ca8680a6383, State: Running, Role: LEADER
I20250624 07:58:33.073462 21714 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15871, Committed index: 15871, Last appended: 2.15871, Last appended by leader: 15871, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:33.073765 21714 logging.cc:424] LogThrottler TrackedPeer Lag: suppressed but not reported on 4 messages since previous log ~11 seconds ago
I20250624 07:58:33.074223 21714 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:33.075183 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:33.075238 20163 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383"
I20250624 07:58:33.077229 21043 raft_consensus.cc:1238] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 LEADER]: Rejecting Update request from peer 4f219d2e8b954461973a8ca8680a6383 for earlier term 2. Current term is 3. Ops: []
I20250624 07:58:33.085366 20162 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 3.
I20250624 07:58:33.086609 20163 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 3 FOLLOWER]: Refusing update from remote peer 6cbd668c726a4b1298e789270a0031e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 15871. Preceding OpId from leader: term: 3 index: 15872. (index mismatch)
I20250624 07:58:33.087695 21704 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15872, Last known committed idx: 15871, Time since last communication: 0.000s
I20250624 07:58:33.085345 20161 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:33.103713 19658 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7; no voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:58:33.104398 21699 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
************************ BEGIN STACKS **************************
[New LWP 18721]
[New LWP 18722]
[New LWP 18723]
[New LWP 18724]
[New LWP 18730]
[New LWP 18731]
[New LWP 18732]
[New LWP 18735]
[New LWP 18736]
[New LWP 18737]
[New LWP 18738]
[New LWP 18739]
[New LWP 18740]
[New LWP 18741]
[New LWP 18742]
[New LWP 18743]
[New LWP 18744]
[New LWP 18745]
[New LWP 18746]
[New LWP 18747]
[New LWP 18748]
[New LWP 18749]
[New LWP 18750]
[New LWP 18751]
[New LWP 18752]
[New LWP 18753]
[New LWP 18754]
[New LWP 18755]
[New LWP 18756]
[New LWP 18757]
[New LWP 18758]
[New LWP 18759]
[New LWP 18760]
[New LWP 18761]
[New LWP 18762]
[New LWP 18763]
[New LWP 18764]
[New LWP 18765]
[New LWP 18766]
[New LWP 18767]
[New LWP 18768]
[New LWP 18769]
[New LWP 18770]
[New LWP 18771]
[New LWP 18772]
[New LWP 18773]
[New LWP 18774]
[New LWP 18775]
[New LWP 18776]
[New LWP 18777]
[New LWP 18778]
[New LWP 18779]
[New LWP 18780]
[New LWP 18781]
[New LWP 18782]
[New LWP 18783]
[New LWP 18784]
[New LWP 18785]
[New LWP 18786]
[New LWP 18787]
[New LWP 18788]
[New LWP 18789]
[New LWP 18790]
[New LWP 18791]
[New LWP 18792]
[New LWP 18793]
[New LWP 18794]
[New LWP 18795]
[New LWP 18796]
[New LWP 18797]
[New LWP 18798]
[New LWP 18799]
[New LWP 18800]
[New LWP 18801]
[New LWP 18802]
[New LWP 18803]
[New LWP 18804]
[New LWP 18805]
[New LWP 18806]
[New LWP 18807]
[New LWP 18808]
[New LWP 18809]
[New LWP 18810]
[New LWP 18812]
[New LWP 18813]
[New LWP 18815]
[New LWP 18816]
[New LWP 18817]
[New LWP 18818]
[New LWP 18819]
[New LWP 18820]
[New LWP 18821]
[New LWP 18822]
[New LWP 18823]
[New LWP 18824]
[New LWP 18825]
[New LWP 18826]
[New LWP 18827]
[New LWP 18828]
[New LWP 18829]
[New LWP 18830]
[New LWP 18831]
[New LWP 18832]
[New LWP 18833]
[New LWP 18834]
[New LWP 18835]
[New LWP 18836]
[New LWP 18837]
[New LWP 18838]
[New LWP 18839]
[New LWP 18840]
[New LWP 18841]
[New LWP 18842]
[New LWP 18843]
[New LWP 18844]
[New LWP 18845]
[New LWP 18846]
[New LWP 18847]
[New LWP 18848]
[New LWP 18849]
[New LWP 18850]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f7575afed50 in ?? ()
Id Target Id Frame
* 1 LWP 18718 "kudu" 0x00007f7575afed50 in ?? ()
2 LWP 18721 "kudu" 0x00007f7575afafb9 in ?? ()
3 LWP 18722 "kudu" 0x00007f7575afafb9 in ?? ()
4 LWP 18723 "kudu" 0x00007f7575afafb9 in ?? ()
5 LWP 18724 "kernel-watcher-" 0x00007f7575afafb9 in ?? ()
6 LWP 18730 "ntp client-1873" 0x00007f7575afe9e2 in ?? ()
7 LWP 18731 "file cache-evic" 0x00007f7575afafb9 in ?? ()
8 LWP 18732 "sq_acceptor" 0x00007f756e820cb9 in ?? ()
9 LWP 18735 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
10 LWP 18736 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
11 LWP 18737 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
12 LWP 18738 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
13 LWP 18739 "MaintenanceMgr " 0x00007f7575afaad3 in ?? ()
14 LWP 18740 "txn-status-mana" 0x00007f7575afafb9 in ?? ()
15 LWP 18741 "collect_and_rem" 0x00007f7575afafb9 in ?? ()
16 LWP 18742 "tc-session-exp-" 0x00007f7575afafb9 in ?? ()
17 LWP 18743 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
18 LWP 18744 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
19 LWP 18745 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
20 LWP 18746 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
21 LWP 18747 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
22 LWP 18748 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
23 LWP 18749 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
24 LWP 18750 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
25 LWP 18751 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
26 LWP 18752 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
27 LWP 18753 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
28 LWP 18754 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
29 LWP 18755 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
30 LWP 18756 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
31 LWP 18757 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
32 LWP 18758 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
33 LWP 18759 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
34 LWP 18760 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
35 LWP 18761 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
36 LWP 18762 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
37 LWP 18763 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
38 LWP 18764 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
39 LWP 18765 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
40 LWP 18766 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
41 LWP 18767 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
42 LWP 18768 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
43 LWP 18769 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
44 LWP 18770 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
45 LWP 18771 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
46 LWP 18772 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
47 LWP 18773 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
48 LWP 18774 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
49 LWP 18775 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
50 LWP 18776 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
51 LWP 18777 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
52 LWP 18778 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
53 LWP 18779 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
54 LWP 18780 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
55 LWP 18781 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
56 LWP 18782 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
57 LWP 18783 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
58 LWP 18784 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
59 LWP 18785 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
60 LWP 18786 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
61 LWP 18787 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
62 LWP 18788 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
63 LWP 18789 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
64 LWP 18790 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
65 LWP 18791 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
66 LWP 18792 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
67 LWP 18793 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
68 LWP 18794 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
69 LWP 18795 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
70 LWP 18796 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
71 LWP 18797 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
72 LWP 18798 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
73 LWP 18799 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
74 LWP 18800 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
75 LWP 18801 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
76 LWP 18802 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
77 LWP 18803 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
78 LWP 18804 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
79 LWP 18805 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
80 LWP 18806 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
81 LWP 18807 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
82 LWP 18808 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
83 LWP 18809 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
84 LWP 18810 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
85 LWP 18812 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
86 LWP 18813 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
87 LWP 18815 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
88 LWP 18816 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
89 LWP 18817 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
90 LWP 18818 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
91 LWP 18819 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
92 LWP 18820 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
93 LWP 18821 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
94 LWP 18822 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
95 LWP 18823 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
96 LWP 18824 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
97 LWP 18825 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
98 LWP 18826 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
99 LWP 18827 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
100 LWP 18828 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
101 LWP 18829 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
102 LWP 18830 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
103 LWP 18831 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
104 LWP 18832 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
105 LWP 18833 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
106 LWP 18834 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
107 LWP 18835 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
108 LWP 18836 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
109 LWP 18837 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
110 LWP 18838 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
111 LWP 18839 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
112 LWP 18840 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
113 LWP 18841 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
114 LWP 18842 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
115 LWP 18843 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
116 LWP 18844 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
117 LWP 18845 "diag-logger-188" 0x00007f7575afafb9 in ?? ()
118 LWP 18846 "result-tracker-" 0x00007f7575afafb9 in ?? ()
119 LWP 18847 "excess-log-dele" 0x00007f7575afafb9 in ?? ()
120 LWP 18848 "acceptor-18848" 0x00007f756e82f0c7 in ?? ()
121 LWP 18849 "heartbeat-18849" 0x00007f7575afafb9 in ?? ()
122 LWP 18850 "maintenance_sch" 0x00007f7575afafb9 in ?? ()
Thread 122 (LWP 18850):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752b3a26e0 in ?? ()
#2 0x0000000000000338 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f752b3a2730 in ?? ()
#6 0x0000000000000670 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18849):
#0 0x00007f7575afafb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000cd in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f752bbba5f0 in ?? ()
#6 0x000000000000019a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18848):
#0 0x00007f756e82f0c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f752c3e1d90 in ?? ()
#3 0x00007f752c3e1dc0 in ?? ()
#4 0x00007f752c3e1ec0 in ?? ()
#5 0x00007f752c3e1db0 in ?? ()
#6 0x00007f752c3e1e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f75726b5980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18847):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752cbfbf40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffee7883d50 in ?? ()
#5 0x00007f752cbfbf30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18846):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752d414100 in ?? ()
#2 0x00000000000000ce in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f752d4140f0 in ?? ()
#6 0x000000000000019c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18845):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18844):
#0 0x00007f7575afaad3 in ?? ()
#1 0x000000000000002c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f752e443e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f752e443eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 18843):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000018 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f752ec5be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f752ec5beb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 18842):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18841):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18840):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18839):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18838):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18837):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18836):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18835):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18834):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18833):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18832):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18831):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18830):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18829):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18828):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18827):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18826):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18825):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18824):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18823):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18822):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18821):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18820):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18819):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18818):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18817):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18816):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18815):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18813):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18812):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18810):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18809):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18808):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18807):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18806):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18805):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18804):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18803):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18802):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f7542814e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f7542814eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f7542814eb0 in ?? ()
#11 0x00000feea85029d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18801):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 18800):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18799):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18798):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18797):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18796):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18795):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18794):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18793):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18792):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18791):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18790):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18789):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18788):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18787):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18786):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18785):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18784):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18783):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18782):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f754c1dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f754c1ddeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f754c1ddeb0 in ?? ()
#11 0x00000feea983bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18781):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18780):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18779):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18778):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18777):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18776):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18775):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18774):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18773):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18772):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18771):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18770):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18769):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18768):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18767):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18766):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18765):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18764):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18763):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18762):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f75563aee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f75563aeeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f75563aeeb0 in ?? ()
#11 0x00000feeaac75dd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18761):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18760):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18759):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18758):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18757):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18756):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18755):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18754):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18753):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18752):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18751):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18750):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18749):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18748):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18747):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18746):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18745):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18744):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18743):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18742):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75603c2cc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f75603c2cb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18741):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18740):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75613c4240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f75613c4230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18739):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18738):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f75623c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f75623c7310 in ?? ()
#4 0x00007f75623c7520 in ?? ()
#5 0x00007f75623c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f75623c73e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb98202d391a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x000000002b000000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fef2c470e7c in ?? ()
#17 0x00007f75623c73c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x00007f75623c7390 in ?? ()
#20 0x3fb98202d391a000 in ?? ()
#21 0x00000000623c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb98202d391a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18737):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f7562bc8320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f7562bc8310 in ?? ()
#4 0x00007f7562bc8520 in ?? ()
#5 0x00007f7562bc8360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f7562bc83e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb973330202a000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 10 (LWP 18736):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f75633c9320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f75633c9310 in ?? ()
#4 0x00007f75633c9520 in ?? ()
#5 0x00007f75633c9360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f75633c93e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb97161122de000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000cb000000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fef2c67127c in ?? ()
#17 0x00007f75633c93c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x00007f75633c9390 in ?? ()
#20 0x3fb97161122de000 in ?? ()
#21 0x00000000633c93e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb97161122de000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18735):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f7564fb3320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f7564fb3310 in ?? ()
#4 0x00007f7564fb3520 in ?? ()
#5 0x00007f7564fb3360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f7564fb33e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb987139ac58000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000029c00000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a9 in ?? ()
#16 0x00000fef2c9ee67c in ?? ()
#17 0x00007f7564fb33c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18732):
#0 0x00007f756e820cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f7566fb77c8 in ?? ()
#3 0x000060200001ccd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18731):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75667b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f75667b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18730):
#0 0x00007f7575afe9e2 in ?? ()
#1 0x00007f75657b2ba0 in ?? ()
#2 0x00007f75657b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f75657b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f75657b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f75657b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f757b411edc in ?? ()
#12 0x00007f757b4018d9 in ?? ()
#13 0x00007f757b405ec6 in ?? ()
#14 0x00007f757b40a900 in ?? ()
#15 0x00007f757b40a729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f75726d64fe in ?? ()
#18 0x00007f7575af46db in ?? ()
#19 0x00000fef2caee484 in ?? ()
#20 0x00007f75657b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18724):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f7567fb8c80 in ?? ()
#2 0x000000000000040a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f7567fb8c70 in ?? ()
#6 0x0000000000000814 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18723):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18722):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18721):
#0 0x00007f7575afafb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f75697bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18718):
#0 0x00007f7575afed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:33.994892 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 6 with UUID 1fe60cc8e38948b694506041d7734713 and pid 19000
************************ BEGIN STACKS **************************
[New LWP 19002]
[New LWP 19003]
[New LWP 19004]
[New LWP 19005]
[New LWP 19011]
[New LWP 19012]
[New LWP 19013]
[New LWP 19016]
[New LWP 19017]
[New LWP 19018]
[New LWP 19019]
[New LWP 19020]
[New LWP 19021]
[New LWP 19022]
[New LWP 19023]
[New LWP 19024]
[New LWP 19025]
[New LWP 19026]
[New LWP 19027]
[New LWP 19028]
[New LWP 19029]
[New LWP 19030]
[New LWP 19031]
[New LWP 19032]
[New LWP 19033]
[New LWP 19034]
[New LWP 19035]
[New LWP 19036]
[New LWP 19037]
[New LWP 19038]
[New LWP 19039]
[New LWP 19040]
[New LWP 19041]
[New LWP 19042]
[New LWP 19043]
[New LWP 19044]
[New LWP 19045]
[New LWP 19046]
[New LWP 19047]
[New LWP 19048]
[New LWP 19049]
[New LWP 19050]
[New LWP 19051]
[New LWP 19052]
[New LWP 19053]
[New LWP 19054]
[New LWP 19055]
[New LWP 19056]
[New LWP 19057]
[New LWP 19058]
[New LWP 19059]
[New LWP 19060]
[New LWP 19061]
[New LWP 19062]
[New LWP 19063]
[New LWP 19064]
[New LWP 19065]
[New LWP 19066]
[New LWP 19067]
[New LWP 19068]
[New LWP 19069]
[New LWP 19070]
[New LWP 19071]
[New LWP 19072]
[New LWP 19073]
[New LWP 19074]
[New LWP 19075]
[New LWP 19076]
[New LWP 19077]
[New LWP 19078]
[New LWP 19079]
[New LWP 19080]
[New LWP 19081]
[New LWP 19082]
[New LWP 19083]
[New LWP 19084]
[New LWP 19085]
[New LWP 19086]
[New LWP 19087]
[New LWP 19088]
[New LWP 19089]
[New LWP 19090]
[New LWP 19091]
[New LWP 19092]
[New LWP 19093]
[New LWP 19094]
[New LWP 19095]
[New LWP 19096]
[New LWP 19097]
[New LWP 19098]
[New LWP 19099]
[New LWP 19100]
[New LWP 19101]
[New LWP 19102]
[New LWP 19103]
[New LWP 19104]
[New LWP 19106]
[New LWP 19107]
[New LWP 19108]
[New LWP 19109]
[New LWP 19110]
[New LWP 19111]
[New LWP 19112]
[New LWP 19113]
[New LWP 19114]
[New LWP 19115]
[New LWP 19116]
[New LWP 19117]
[New LWP 19118]
[New LWP 19119]
[New LWP 19120]
[New LWP 19121]
[New LWP 19122]
[New LWP 19123]
[New LWP 19124]
[New LWP 19125]
[New LWP 19126]
[New LWP 19127]
[New LWP 19128]
[New LWP 19129]
[New LWP 19132]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f564721dd50 in ?? ()
Id Target Id Frame
* 1 LWP 19000 "kudu" 0x00007f564721dd50 in ?? ()
2 LWP 19002 "kudu" 0x00007f5647219fb9 in ?? ()
3 LWP 19003 "kudu" 0x00007f5647219fb9 in ?? ()
4 LWP 19004 "kudu" 0x00007f5647219fb9 in ?? ()
5 LWP 19005 "kernel-watcher-" 0x00007f5647219fb9 in ?? ()
6 LWP 19011 "ntp client-1901" 0x00007f564721d9e2 in ?? ()
7 LWP 19012 "file cache-evic" 0x00007f5647219fb9 in ?? ()
8 LWP 19013 "sq_acceptor" 0x00007f563ff3fcb9 in ?? ()
9 LWP 19016 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
10 LWP 19017 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
11 LWP 19018 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
12 LWP 19019 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
13 LWP 19020 "MaintenanceMgr " 0x00007f5647219ad3 in ?? ()
14 LWP 19021 "txn-status-mana" 0x00007f5647219fb9 in ?? ()
15 LWP 19022 "collect_and_rem" 0x00007f5647219fb9 in ?? ()
16 LWP 19023 "tc-session-exp-" 0x00007f5647219fb9 in ?? ()
17 LWP 19024 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
18 LWP 19025 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
19 LWP 19026 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
20 LWP 19027 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
21 LWP 19028 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
22 LWP 19029 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
23 LWP 19030 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
24 LWP 19031 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
25 LWP 19032 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
26 LWP 19033 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
27 LWP 19034 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
28 LWP 19035 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
29 LWP 19036 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
30 LWP 19037 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
31 LWP 19038 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
32 LWP 19039 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
33 LWP 19040 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
34 LWP 19041 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
35 LWP 19042 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
36 LWP 19043 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
37 LWP 19044 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
38 LWP 19045 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
39 LWP 19046 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
40 LWP 19047 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
41 LWP 19048 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
42 LWP 19049 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
43 LWP 19050 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
44 LWP 19051 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
45 LWP 19052 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
46 LWP 19053 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
47 LWP 19054 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
48 LWP 19055 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
49 LWP 19056 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
50 LWP 19057 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
51 LWP 19058 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
52 LWP 19059 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
53 LWP 19060 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
54 LWP 19061 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
55 LWP 19062 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
56 LWP 19063 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
57 LWP 19064 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
58 LWP 19065 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
59 LWP 19066 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
60 LWP 19067 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
61 LWP 19068 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
62 LWP 19069 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
63 LWP 19070 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
64 LWP 19071 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
65 LWP 19072 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
66 LWP 19073 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
67 LWP 19074 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
68 LWP 19075 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
69 LWP 19076 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
70 LWP 19077 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
71 LWP 19078 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
72 LWP 19079 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
73 LWP 19080 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
74 LWP 19081 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
75 LWP 19082 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
76 LWP 19083 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
77 LWP 19084 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
78 LWP 19085 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
79 LWP 19086 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
80 LWP 19087 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
81 LWP 19088 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
82 LWP 19089 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
83 LWP 19090 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
84 LWP 19091 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
85 LWP 19092 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
86 LWP 19093 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
87 LWP 19094 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
88 LWP 19095 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
89 LWP 19096 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
90 LWP 19097 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
91 LWP 19098 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
92 LWP 19099 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
93 LWP 19100 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
94 LWP 19101 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
95 LWP 19102 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
96 LWP 19103 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
97 LWP 19104 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
98 LWP 19106 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
99 LWP 19107 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
100 LWP 19108 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
101 LWP 19109 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
102 LWP 19110 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
103 LWP 19111 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
104 LWP 19112 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
105 LWP 19113 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
106 LWP 19114 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
107 LWP 19115 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
108 LWP 19116 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
109 LWP 19117 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
110 LWP 19118 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
111 LWP 19119 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
112 LWP 19120 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
113 LWP 19121 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
114 LWP 19122 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
115 LWP 19123 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
116 LWP 19124 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
117 LWP 19125 "diag-logger-191" 0x00007f5647219fb9 in ?? ()
118 LWP 19126 "result-tracker-" 0x00007f5647219fb9 in ?? ()
119 LWP 19127 "excess-log-dele" 0x00007f5647219fb9 in ?? ()
120 LWP 19128 "acceptor-19128" 0x00007f563ff4e0c7 in ?? ()
121 LWP 19129 "heartbeat-19129" 0x00007f5647219fb9 in ?? ()
122 LWP 19132 "maintenance_sch" 0x00007f5647219fb9 in ?? ()
Thread 122 (LWP 19132):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55fc2b26e0 in ?? ()
#2 0x000000000000032c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f55fc2b2730 in ?? ()
#6 0x0000000000000658 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19129):
#0 0x00007f5647219fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000ca in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f55fd2d95f0 in ?? ()
#6 0x0000000000000194 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19128):
#0 0x00007f563ff4e0c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007f55fdb00d90 in ?? ()
#3 0x00007f55fdb00dc0 in ?? ()
#4 0x00007f55fdb00ec0 in ?? ()
#5 0x00007f55fdb00db0 in ?? ()
#6 0x00007f55fdb00e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f5643dd4980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19127):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55fe31af40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc5442a1d0 in ?? ()
#5 0x00007f55fe31af30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19126):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55feb33100 in ?? ()
#2 0x00000000000000cb in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f55feb330f0 in ?? ()
#6 0x0000000000000196 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19125):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19124):
#0 0x00007f5647219ad3 in ?? ()
#1 0x000000000000001b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f5638fb7e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5638fb7eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f5638fb7eb0 in ?? ()
#11 0x00000feac71f6fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19123):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000029 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f55ffb62e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f55ffb62eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f55ffb62eb0 in ?? ()
#11 0x00000feabff6c5d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19122):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19121):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19120):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19119):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19118):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19117):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19116):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19115):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19114):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19113):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19112):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19111):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19110):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19109):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19108):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19107):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19106):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19104):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19103):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19102):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19101):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19100):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19099):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19098):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19097):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19096):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19095):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19094):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19093):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19092):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19091):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19090):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19089):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19088):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19087):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19086):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19085):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19084):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19083):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f561371be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f561371beb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f561371beb0 in ?? ()
#11 0x00000feac26e37d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 19082):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19081):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19080):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19079):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19078):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19077):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19076):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19075):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19074):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19073):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19072):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19071):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19070):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19069):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19068):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19067):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19066):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19065):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19064):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19063):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f561d8fce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f561d8fceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f561d8fceb0 in ?? ()
#11 0x00000feac3b1f9d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 19062):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19061):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19060):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19059):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19058):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19057):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19056):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19055):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19054):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19053):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19052):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19051):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19050):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19049):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19048):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19047):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19046):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19045):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19044):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19043):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f5627adde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5627addeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f5627addeb0 in ?? ()
#11 0x00000feac4f5bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 19042):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19041):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19040):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19039):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19038):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19037):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19036):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19035):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19034):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19033):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19032):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19031):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19030):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19029):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19028):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19027):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19026):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19025):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19024):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19023):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5631cdccc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f5631cdccb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19022):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19021):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5632d1a240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f5632d1a230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19020):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19019):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5633d69320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f5633d69310 in ?? ()
#4 0x00007f5633d69520 in ?? ()
#5 0x00007f5633d69360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f5633d693e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb96cb25277a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000de000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000feb467a527c in ?? ()
#17 0x00007f5633d693c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5633d69390 in ?? ()
#20 0x3fb96cb25277a000 in ?? ()
#21 0x0000000033d693e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb96cb25277a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19018):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5634580320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f5634580310 in ?? ()
#4 0x00007f5634580520 in ?? ()
#5 0x00007f5634580360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f56345803e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb96f7416db4000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000f8000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000feb468a807c in ?? ()
#17 0x00007f56345803c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5634580390 in ?? ()
#20 0x3fb96f7416db4000 in ?? ()
#21 0x00000000345803e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb96f7416db4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19017):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5634d97320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f5634d97310 in ?? ()
#4 0x00007f5634d97520 in ?? ()
#5 0x00007f5634d97360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f5634d973e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb965dc1b2f0000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000f2000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000feb469aae7c in ?? ()
#17 0x00007f5634d973c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5634d97390 in ?? ()
#20 0x3fb965dc1b2f0000 in ?? ()
#21 0x0000000034d973e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb965dc1b2f0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19016):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5636dad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f5636dad310 in ?? ()
#4 0x00007f5636dad520 in ?? ()
#5 0x00007f5636dad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f5636dad3e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb98c28d86e6000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000001f000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000feb46dada7c in ?? ()
#17 0x00007f5636dad3c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19013):
#0 0x00007f563ff3fcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f56387b77c8 in ?? ()
#3 0x000060200001e510 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19012):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5637fb60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f5637fb60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19011):
#0 0x00007f564721d9e2 in ?? ()
#1 0x00007f56377b3ba0 in ?? ()
#2 0x00007f56377b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f56377b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f56377b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f56377b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f564cb30edc in ?? ()
#12 0x00007f564cb208d9 in ?? ()
#13 0x00007f564cb24ec6 in ?? ()
#14 0x00007f564cb29900 in ?? ()
#15 0x00007f564cb29729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f5643df54fe in ?? ()
#18 0x00007f56472136db in ?? ()
#19 0x00000feb46eee684 in ?? ()
#20 0x00007f56377b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19005):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f56397b8c80 in ?? ()
#2 0x00000000000003f9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f56397b8c70 in ?? ()
#6 0x00000000000007f2 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19004):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19003):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19002):
#0 0x00007f5647219fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f563afbc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19000):
#0 0x00007f564721dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:35.045310 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 7 with UUID 8150d3eeb1c84234a13484098d17ce8f and pid 20806
************************ BEGIN STACKS **************************
[New LWP 20807]
[New LWP 20808]
[New LWP 20809]
[New LWP 20811]
[New LWP 20818]
[New LWP 20819]
[New LWP 20820]
[New LWP 20825]
[New LWP 20827]
[New LWP 20828]
[New LWP 20829]
[New LWP 20830]
[New LWP 20831]
[New LWP 20832]
[New LWP 20833]
[New LWP 20834]
[New LWP 20835]
[New LWP 20836]
[New LWP 20837]
[New LWP 20838]
[New LWP 20839]
[New LWP 20840]
[New LWP 20841]
[New LWP 20842]
[New LWP 20843]
[New LWP 20844]
[New LWP 20845]
[New LWP 20846]
[New LWP 20847]
[New LWP 20848]
[New LWP 20849]
[New LWP 20850]
[New LWP 20851]
[New LWP 20852]
[New LWP 20853]
[New LWP 20854]
[New LWP 20855]
[New LWP 20856]
[New LWP 20857]
[New LWP 20858]
[New LWP 20859]
[New LWP 20860]
[New LWP 20861]
[New LWP 20863]
[New LWP 20864]
[New LWP 20865]
[New LWP 20866]
[New LWP 20867]
[New LWP 20868]
[New LWP 20869]
[New LWP 20870]
[New LWP 20871]
[New LWP 20872]
[New LWP 20873]
[New LWP 20874]
[New LWP 20875]
[New LWP 20876]
[New LWP 20877]
[New LWP 20878]
[New LWP 20879]
[New LWP 20880]
[New LWP 20881]
[New LWP 20882]
[New LWP 20883]
[New LWP 20884]
[New LWP 20885]
[New LWP 20886]
[New LWP 20887]
[New LWP 20888]
[New LWP 20889]
[New LWP 20890]
[New LWP 20891]
[New LWP 20892]
[New LWP 20893]
[New LWP 20894]
[New LWP 20895]
[New LWP 20896]
[New LWP 20897]
[New LWP 20898]
[New LWP 20899]
[New LWP 20900]
[New LWP 20901]
[New LWP 20902]
[New LWP 20903]
[New LWP 20904]
[New LWP 20905]
[New LWP 20906]
[New LWP 20907]
[New LWP 20908]
[New LWP 20909]
[New LWP 20910]
[New LWP 20911]
[New LWP 20912]
[New LWP 20913]
[New LWP 20914]
[New LWP 20915]
[New LWP 20916]
[New LWP 20917]
[New LWP 20918]
[New LWP 20919]
[New LWP 20920]
[New LWP 20921]
[New LWP 20922]
[New LWP 20923]
[New LWP 20924]
[New LWP 20925]
[New LWP 20926]
[New LWP 20927]
[New LWP 20928]
[New LWP 20929]
[New LWP 20930]
[New LWP 20931]
[New LWP 20932]
[New LWP 20933]
[New LWP 20934]
[New LWP 20935]
[New LWP 20936]
[New LWP 20937]
[New LWP 20938]
[New LWP 20939]
[New LWP 20944]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f0f8125bd50 in ?? ()
Id Target Id Frame
* 1 LWP 20806 "kudu" 0x00007f0f8125bd50 in ?? ()
2 LWP 20807 "kudu" 0x00007f0f81257fb9 in ?? ()
3 LWP 20808 "kudu" 0x00007f0f81257fb9 in ?? ()
4 LWP 20809 "kudu" 0x00007f0f81257fb9 in ?? ()
5 LWP 20811 "kernel-watcher-" 0x00007f0f81257fb9 in ?? ()
6 LWP 20818 "ntp client-2081" 0x00007f0f8125b9e2 in ?? ()
7 LWP 20819 "file cache-evic" 0x00007f0f81257fb9 in ?? ()
8 LWP 20820 "sq_acceptor" 0x00007f0f79f7dcb9 in ?? ()
9 LWP 20825 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
10 LWP 20827 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
11 LWP 20828 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
12 LWP 20829 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
13 LWP 20830 "MaintenanceMgr " 0x00007f0f81257ad3 in ?? ()
14 LWP 20831 "txn-status-mana" 0x00007f0f81257fb9 in ?? ()
15 LWP 20832 "collect_and_rem" 0x00007f0f81257fb9 in ?? ()
16 LWP 20833 "tc-session-exp-" 0x00007f0f81257fb9 in ?? ()
17 LWP 20834 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
18 LWP 20835 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
19 LWP 20836 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
20 LWP 20837 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
21 LWP 20838 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
22 LWP 20839 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
23 LWP 20840 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
24 LWP 20841 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
25 LWP 20842 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
26 LWP 20843 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
27 LWP 20844 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
28 LWP 20845 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
29 LWP 20846 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
30 LWP 20847 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
31 LWP 20848 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
32 LWP 20849 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
33 LWP 20850 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
34 LWP 20851 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
35 LWP 20852 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
36 LWP 20853 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
37 LWP 20854 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
38 LWP 20855 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
39 LWP 20856 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
40 LWP 20857 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
41 LWP 20858 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
42 LWP 20859 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
43 LWP 20860 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
44 LWP 20861 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
45 LWP 20863 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
46 LWP 20864 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
47 LWP 20865 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
48 LWP 20866 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
49 LWP 20867 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
50 LWP 20868 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
51 LWP 20869 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
52 LWP 20870 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
53 LWP 20871 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
54 LWP 20872 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
55 LWP 20873 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
56 LWP 20874 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
57 LWP 20875 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
58 LWP 20876 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
59 LWP 20877 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
60 LWP 20878 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
61 LWP 20879 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
62 LWP 20880 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
63 LWP 20881 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
64 LWP 20882 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
65 LWP 20883 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
66 LWP 20884 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
67 LWP 20885 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
68 LWP 20886 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
69 LWP 20887 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
70 LWP 20888 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
71 LWP 20889 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
72 LWP 20890 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
73 LWP 20891 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
74 LWP 20892 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
75 LWP 20893 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
76 LWP 20894 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
77 LWP 20895 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
78 LWP 20896 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
79 LWP 20897 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
80 LWP 20898 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
81 LWP 20899 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
82 LWP 20900 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
83 LWP 20901 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
84 LWP 20902 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
85 LWP 20903 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
86 LWP 20904 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
87 LWP 20905 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
88 LWP 20906 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
89 LWP 20907 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
90 LWP 20908 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
91 LWP 20909 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
92 LWP 20910 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
93 LWP 20911 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
94 LWP 20912 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
95 LWP 20913 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
96 LWP 20914 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
97 LWP 20915 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
98 LWP 20916 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
99 LWP 20917 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
100 LWP 20918 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
101 LWP 20919 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
102 LWP 20920 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
103 LWP 20921 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
104 LWP 20922 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
105 LWP 20923 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
106 LWP 20924 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
107 LWP 20925 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
108 LWP 20926 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
109 LWP 20927 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
110 LWP 20928 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
111 LWP 20929 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
112 LWP 20930 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
113 LWP 20931 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
114 LWP 20932 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
115 LWP 20933 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
116 LWP 20934 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
117 LWP 20935 "diag-logger-209" 0x00007f0f81257fb9 in ?? ()
118 LWP 20936 "result-tracker-" 0x00007f0f81257fb9 in ?? ()
119 LWP 20937 "excess-log-dele" 0x00007f0f81257fb9 in ?? ()
120 LWP 20938 "acceptor-20938" 0x00007f0f79f8c0c7 in ?? ()
121 LWP 20939 "heartbeat-20939" 0x00007f0f81257fb9 in ?? ()
122 LWP 20944 "maintenance_sch" 0x00007f0f81257fb9 in ?? ()
Thread 122 (LWP 20944):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f362f06e0 in ?? ()
#2 0x000000000000015a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f0f362f0730 in ?? ()
#6 0x00000000000002b4 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 20939):
#0 0x00007f0f81257fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f0f373175f0 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 20938):
#0 0x00007f0f79f8c0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f0f37b3ed90 in ?? ()
#3 0x00007f0f37b3edc0 in ?? ()
#4 0x00007f0f37b3eec0 in ?? ()
#5 0x00007f0f37b3edb0 in ?? ()
#6 0x00007f0f37b3ee20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 20937):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f38358f40 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffde53678d0 in ?? ()
#5 0x00007f0f38358f30 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 20936):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f38b71100 in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f0f38b710f0 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 20935):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 20934):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x000000000000000f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f0f39ba0e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f39ba0eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f0f81257770 in ?? ()
#10 0x00007f0f39ba0eb0 in ?? ()
#11 0x00000fe1e73741d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 20933):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f0f3a3b8e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f3a3b8eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f0f81257770 in ?? ()
#10 0x00007f0f3a3b8eb0 in ?? ()
#11 0x00000fe1e74771d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 20932):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 20931):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 20930):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 20929):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 20928):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 20927):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 20926):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 20925):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 20924):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 20923):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 20922):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 20921):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 20920):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 20919):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 20918):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 20917):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 20916):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 20915):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 20914):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 20913):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 20912):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 20911):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 20910):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 20909):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 20908):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 20907):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 20906):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 20905):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 20904):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 20903):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 20902):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 20901):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 20900):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 20899):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 20898):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 20897):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 20896):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 20895):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 20894):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f0f4df71e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f4df71eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 20893):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 20892):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 20891):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 20890):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 20889):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 20888):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 20887):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 20886):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 20885):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 20884):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 20883):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 20882):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 20881):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 20880):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 20879):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 20878):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 20877):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 20876):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 20875):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 20874):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f0f58152e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f58152eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 20873):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 20872):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 20871):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 20870):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 20869):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 20868):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 20867):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 20866):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 20865):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 20864):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 20863):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 20861):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 20860):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 20859):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 20858):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 20857):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 20856):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 20855):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 20854):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 20853):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f0f62333e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f62333eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 20852):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 20851):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 20850):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20849):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20848):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20847):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20846):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20845):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20844):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20843):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20842):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20841):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20840):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20839):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20838):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20837):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20836):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20835):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20834):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20833):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f6bd1dcc0 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f0f6bd1dcb0 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20832):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20831):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f6cd5b240 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f0f6cd5b230 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20830):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 20829):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6ddaa320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f0f6ddaa310 in ?? ()
#4 0x00007f0f6ddaa520 in ?? ()
#5 0x00007f0f6ddaa360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f0f6ddaa3e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb97ec9bd418000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000c1000000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fe26dbad47c in ?? ()
#17 0x00007f0f6ddaa3c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6ddaa390 in ?? ()
#20 0x3fb97ec9bd418000 in ?? ()
#21 0x000000006ddaa3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb97ec9bd418000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20828):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6e5b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f0f6e5b3310 in ?? ()
#4 0x00007f0f6e5b3520 in ?? ()
#5 0x00007f0f6e5b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f0f6e5b33e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb97e76d7d2a000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000080c00000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe26dcae67c in ?? ()
#17 0x00007f0f6e5b33c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6e5b3390 in ?? ()
#20 0x3fb97e76d7d2a000 in ?? ()
#21 0x000000006e5b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb97e76d7d2a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20827):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6edb4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f0f6edb4310 in ?? ()
#4 0x00007f0f6edb4520 in ?? ()
#5 0x00007f0f6edb4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f0f6edb43e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb96e659b47e000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000fa400000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000fe26ddae87c in ?? ()
#17 0x00007f0f6edb43c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6edb4390 in ?? ()
#20 0x3fb96e659b47e000 in ?? ()
#21 0x000000006edb43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb96e659b47e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20825):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f70dad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f0f70dad310 in ?? ()
#4 0x00007f0f70dad520 in ?? ()
#5 0x00007f0f70dad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f0f70dad3e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb9827036a3c000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000d9000000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a3 in ?? ()
#16 0x00000fe26e1ada7c in ?? ()
#17 0x00007f0f70dad3c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20820):
#0 0x00007f0f79f7dcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f0f727b77c8 in ?? ()
#3 0x000060200001e9d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20819):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f71fb60c0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f0f71fb60b0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20818):
#0 0x00007f0f8125b9e2 in ?? ()
#1 0x00007f0f717b3ba0 in ?? ()
#2 0x00007f0f717b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f0f717b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f0f717b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f0f717b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f0f86b6eedc in ?? ()
#12 0x00007f0f86b5e8d9 in ?? ()
#13 0x00007f0f86b62ec6 in ?? ()
#14 0x00007f0f86b67900 in ?? ()
#15 0x00007f0f86b67729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f0f7de334fe in ?? ()
#18 0x00007f0f812516db in ?? ()
#19 0x00000fe26e2ee684 in ?? ()
#20 0x00007f0f717b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20811):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f737b8c80 in ?? ()
#2 0x00000000000001bb in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f0f737b8c70 in ?? ()
#6 0x0000000000000376 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20809):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20808):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20807):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f0f74fbc450 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20806):
#0 0x00007f0f8125bd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:35.951174 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 8 with UUID 6cbd668c726a4b1298e789270a0031e2 and pid 20953
************************ BEGIN STACKS **************************
[New LWP 20957]
[New LWP 20958]
[New LWP 20959]
[New LWP 20961]
[New LWP 20967]
[New LWP 20968]
[New LWP 20969]
[New LWP 20974]
[New LWP 20975]
[New LWP 20976]
[New LWP 20977]
[New LWP 20978]
[New LWP 20979]
[New LWP 20981]
[New LWP 20982]
[New LWP 20983]
[New LWP 20984]
[New LWP 20985]
[New LWP 20986]
[New LWP 20987]
[New LWP 20988]
[New LWP 20989]
[New LWP 20990]
[New LWP 20991]
[New LWP 20992]
[New LWP 20993]
[New LWP 20994]
[New LWP 20995]
[New LWP 20996]
[New LWP 20997]
[New LWP 20998]
[New LWP 20999]
[New LWP 21000]
[New LWP 21001]
[New LWP 21002]
[New LWP 21003]
[New LWP 21004]
[New LWP 21005]
[New LWP 21006]
[New LWP 21007]
[New LWP 21008]
[New LWP 21009]
[New LWP 21010]
[New LWP 21011]
[New LWP 21012]
[New LWP 21013]
[New LWP 21014]
[New LWP 21015]
[New LWP 21016]
[New LWP 21017]
[New LWP 21018]
[New LWP 21019]
[New LWP 21020]
[New LWP 21021]
[New LWP 21022]
[New LWP 21023]
[New LWP 21024]
[New LWP 21025]
[New LWP 21026]
[New LWP 21028]
[New LWP 21029]
[New LWP 21030]
[New LWP 21031]
[New LWP 21032]
[New LWP 21033]
[New LWP 21034]
[New LWP 21035]
[New LWP 21036]
[New LWP 21037]
[New LWP 21038]
[New LWP 21039]
[New LWP 21040]
[New LWP 21041]
[New LWP 21042]
[New LWP 21043]
[New LWP 21044]
[New LWP 21045]
[New LWP 21046]
[New LWP 21047]
[New LWP 21048]
[New LWP 21049]
[New LWP 21050]
[New LWP 21051]
[New LWP 21052]
[New LWP 21053]
[New LWP 21054]
[New LWP 21059]
[New LWP 21061]
[New LWP 21062]
[New LWP 21063]
[New LWP 21064]
[New LWP 21065]
[New LWP 21066]
[New LWP 21067]
[New LWP 21068]
[New LWP 21069]
[New LWP 21070]
[New LWP 21071]
[New LWP 21072]
[New LWP 21073]
[New LWP 21074]
[New LWP 21075]
[New LWP 21076]
[New LWP 21077]
[New LWP 21078]
[New LWP 21079]
[New LWP 21080]
[New LWP 21081]
[New LWP 21082]
[New LWP 21083]
[New LWP 21084]
[New LWP 21085]
[New LWP 21086]
[New LWP 21087]
[New LWP 21088]
[New LWP 21089]
[New LWP 21090]
[New LWP 21091]
[New LWP 21092]
[New LWP 21094]
[New LWP 21095]
[New LWP 21740]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007ffa6e71ed50 in ?? ()
Id Target Id Frame
* 1 LWP 20953 "kudu" 0x00007ffa6e71ed50 in ?? ()
2 LWP 20957 "kudu" 0x00007ffa6e71afb9 in ?? ()
3 LWP 20958 "kudu" 0x00007ffa6e71afb9 in ?? ()
4 LWP 20959 "kudu" 0x00007ffa6e71afb9 in ?? ()
5 LWP 20961 "kernel-watcher-" 0x00007ffa6e71afb9 in ?? ()
6 LWP 20967 "ntp client-2096" 0x00007ffa6e71e9e2 in ?? ()
7 LWP 20968 "file cache-evic" 0x00007ffa6e71afb9 in ?? ()
8 LWP 20969 "sq_acceptor" 0x00007ffa67440cb9 in ?? ()
9 LWP 20974 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
10 LWP 20975 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
11 LWP 20976 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
12 LWP 20977 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
13 LWP 20978 "MaintenanceMgr " 0x00007ffa6e71aad3 in ?? ()
14 LWP 20979 "txn-status-mana" 0x00007ffa6e71afb9 in ?? ()
15 LWP 20981 "collect_and_rem" 0x00007ffa6e71afb9 in ?? ()
16 LWP 20982 "tc-session-exp-" 0x00007ffa6e71afb9 in ?? ()
17 LWP 20983 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
18 LWP 20984 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
19 LWP 20985 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
20 LWP 20986 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
21 LWP 20987 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
22 LWP 20988 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
23 LWP 20989 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
24 LWP 20990 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
25 LWP 20991 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
26 LWP 20992 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
27 LWP 20993 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
28 LWP 20994 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
29 LWP 20995 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
30 LWP 20996 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
31 LWP 20997 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
32 LWP 20998 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
33 LWP 20999 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
34 LWP 21000 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
35 LWP 21001 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
36 LWP 21002 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
37 LWP 21003 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
38 LWP 21004 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
39 LWP 21005 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
40 LWP 21006 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
41 LWP 21007 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
42 LWP 21008 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
43 LWP 21009 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
44 LWP 21010 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
45 LWP 21011 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
46 LWP 21012 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
47 LWP 21013 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
48 LWP 21014 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
49 LWP 21015 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
50 LWP 21016 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
51 LWP 21017 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
52 LWP 21018 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
53 LWP 21019 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
54 LWP 21020 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
55 LWP 21021 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
56 LWP 21022 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
57 LWP 21023 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
58 LWP 21024 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
59 LWP 21025 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
60 LWP 21026 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
61 LWP 21028 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
62 LWP 21029 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
63 LWP 21030 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
64 LWP 21031 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
65 LWP 21032 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
66 LWP 21033 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
67 LWP 21034 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
68 LWP 21035 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
69 LWP 21036 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
70 LWP 21037 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
71 LWP 21038 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
72 LWP 21039 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
73 LWP 21040 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
74 LWP 21041 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
75 LWP 21042 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
76 LWP 21043 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
77 LWP 21044 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
78 LWP 21045 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
79 LWP 21046 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
80 LWP 21047 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
81 LWP 21048 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
82 LWP 21049 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
83 LWP 21050 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
84 LWP 21051 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
85 LWP 21052 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
86 LWP 21053 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
87 LWP 21054 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
88 LWP 21059 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
89 LWP 21061 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
90 LWP 21062 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
91 LWP 21063 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
92 LWP 21064 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
93 LWP 21065 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
94 LWP 21066 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
95 LWP 21067 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
96 LWP 21068 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
97 LWP 21069 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
98 LWP 21070 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
99 LWP 21071 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
100 LWP 21072 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
101 LWP 21073 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
102 LWP 21074 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
103 LWP 21075 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
104 LWP 21076 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
105 LWP 21077 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
106 LWP 21078 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
107 LWP 21079 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
108 LWP 21080 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
109 LWP 21081 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
110 LWP 21082 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
111 LWP 21083 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
112 LWP 21084 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
113 LWP 21085 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
114 LWP 21086 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
115 LWP 21087 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
116 LWP 21088 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
117 LWP 21089 "diag-logger-210" 0x00007ffa6e71afb9 in ?? ()
118 LWP 21090 "result-tracker-" 0x00007ffa6e71afb9 in ?? ()
119 LWP 21091 "excess-log-dele" 0x00007ffa6e71afb9 in ?? ()
120 LWP 21092 "acceptor-21092" 0x00007ffa6744f0c7 in ?? ()
121 LWP 21094 "heartbeat-21094" 0x00007ffa6e71afb9 in ?? ()
122 LWP 21095 "maintenance_sch" 0x00007ffa6e71afb9 in ?? ()
123 LWP 21740 "raft [worker]-2" 0x00007ffa6e71afb9 in ?? ()
Thread 123 (LWP 21740):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000002f in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007ffa198beba4 in ?? ()
#5 0x00007ffa198be830 in ?? ()
#6 0x000000000000005f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ffa198be850 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 21095):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa225d06e0 in ?? ()
#2 0x000000000000014e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007ffa225d0730 in ?? ()
#6 0x000000000000029c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 21094):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000057 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007ffa22df65f0 in ?? ()
#6 0x00000000000000af in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ffa22df6610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007ffa22df6690 in ?? ()
#12 0x00000fff445becd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 21092):
#0 0x00007ffa6744f0c7 in ?? ()
#1 0x0000614000040ea8 in ?? ()
#2 0x00007ffa23e1ed90 in ?? ()
#3 0x00007ffa23e1edc0 in ?? ()
#4 0x00007ffa23e1eec0 in ?? ()
#5 0x00007ffa23e1edb0 in ?? ()
#6 0x00007ffa23e1ee20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 21091):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa24637f40 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffef798a370 in ?? ()
#5 0x00007ffa24637f30 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 21090):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa24e50100 in ?? ()
#2 0x0000000000000054 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007ffa24e500f0 in ?? ()
#6 0x00000000000000a8 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 21089):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 21088):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000018 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000350078 in ?? ()
#4 0x00007ffa25e7fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa25e7feb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 21087):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00034d088 in ?? ()
#4 0x00007ffa266d9e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa266d9eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 21086):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 21085):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 21084):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 21083):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 21082):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 21081):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 21080):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 21079):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 21078):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 21077):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 21076):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 21075):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 21074):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 21073):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 21072):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 21071):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 21070):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 21069):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 21068):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 21067):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 21066):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 21065):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 21064):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 21063):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 21062):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 21061):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 21059):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 21054):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 21053):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 21052):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 21051):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 21050):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 21049):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 21048):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 21047):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 21046):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 21045):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 21044):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 21043):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x00000000000000ad in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007ffa3abcce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa3abcceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007ffa6e71a770 in ?? ()
#10 0x00007ffa3abcceb0 in ?? ()
#11 0x00000fff475799d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 21042):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000206 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007ffa3b3e4e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa3b3e4eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 21041):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 21040):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 21039):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 21038):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 21037):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 21036):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 21035):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 21034):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 21033):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 21032):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 21031):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 21030):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 21029):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 21028):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 21026):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 21025):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 21024):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 21023):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 21022):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007ffa44dade90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa44dadeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 21021):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 21020):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 21019):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 21018):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 21017):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 21016):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 21015):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 21014):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 21013):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 21012):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 21011):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 21010):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 21009):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 21008):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 21007):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 21006):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 21005):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 21004):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 21003):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 21002):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x000000000000000e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007ffa4d746e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa4d746eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 21001):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 21000):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 20999):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20998):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20997):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20996):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20995):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20994):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20993):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20992):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20991):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20990):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20989):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20988):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20987):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20986):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20985):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20984):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20983):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20982):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa5794bcc0 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007ffa5794bcb0 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20981):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20979):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa591e3240 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007ffa591e3230 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20978):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 20977):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5a232320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007ffa5a232310 in ?? ()
#4 0x00007ffa5a232520 in ?? ()
#5 0x00007ffa5a232360 in ?? ()
#6 0x0000614000041098 in ?? ()
#7 0x00007ffa5a2323e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3facce345eb50000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000076c00000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fffcb43e47c in ?? ()
#17 0x00007ffa5a2323c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5a232390 in ?? ()
#20 0x3facce345eb50000 in ?? ()
#21 0x000000005a2323e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000041098 in ?? ()
#24 0x3facce345eb50000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20976):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5aa49320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007ffa5aa49310 in ?? ()
#4 0x00007ffa5aa49520 in ?? ()
#5 0x00007ffa5aa49360 in ?? ()
#6 0x0000614000040e98 in ?? ()
#7 0x00007ffa5aa493e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fb952ce8cb38000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000008d000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fffcb54127c in ?? ()
#17 0x00007ffa5aa493c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5aa49390 in ?? ()
#20 0x3fb952ce8cb38000 in ?? ()
#21 0x000000005aa493e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000040e98 in ?? ()
#24 0x3fb952ce8cb38000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20975):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5b260320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007ffa5b260310 in ?? ()
#4 0x00007ffa5b260520 in ?? ()
#5 0x00007ffa5b260360 in ?? ()
#6 0x0000614000040c98 in ?? ()
#7 0x00007ffa5b2603e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fadbf5754b6c000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000007e000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a5 in ?? ()
#16 0x00000fffcb64407c in ?? ()
#17 0x00007ffa5b2603c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5b260390 in ?? ()
#20 0x3fadbf5754b6c000 in ?? ()
#21 0x000000005b2603e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000040c98 in ?? ()
#24 0x3fadbf5754b6c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20974):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5d27f320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007ffa5d27f310 in ?? ()
#4 0x00007ffa5d27f520 in ?? ()
#5 0x00007ffa5d27f360 in ?? ()
#6 0x0000614000040a98 in ?? ()
#7 0x00007ffa5d27f3e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fa7800ebf46c000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000042000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f839e in ?? ()
#16 0x00000fffcba47e7c in ?? ()
#17 0x00007ffa5d27f3c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20969):
#0 0x00007ffa67440cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007ffa5fcb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20968):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa5f4b60c0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007ffa5f4b60b0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20967):
#0 0x00007ffa6e71e9e2 in ?? ()
#1 0x00007ffa5ecb3ba0 in ?? ()
#2 0x00007ffa5ecb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007ffa5ecb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007ffa5ecb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007ffa5ecb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007ffa74031edc in ?? ()
#12 0x00007ffa740218d9 in ?? ()
#13 0x00007ffa74025ec6 in ?? ()
#14 0x00007ffa7402a900 in ?? ()
#15 0x00007ffa7402a729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007ffa6b2f64fe in ?? ()
#18 0x00007ffa6e7146db in ?? ()
#19 0x00000fffcbd8e684 in ?? ()
#20 0x00007ffa5ecb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20961):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa60cb8c80 in ?? ()
#2 0x00000000000001ad in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007ffa60cb8c70 in ?? ()
#6 0x000000000000035a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20959):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20958):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20957):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007ffa624bc450 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20953):
#0 0x00007ffa6e71ed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:36.885171 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19496
I20250624 07:58:36.895292 19621 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.049970 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19496
W20250624 07:58:37.079545 17294 connection.cc:537] server connection from 127.16.221.65:41515 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250624 07:58:37.079855 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19354
I20250624 07:58:37.087955 19480 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.241357 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19354
I20250624 07:58:37.270782 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19638
I20250624 07:58:37.279215 19765 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.743018 21760 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cbd668c726a4b1298e789270a0031e2)
I20250624 07:58:37.743366 21760 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:37.743672 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19638
W20250624 07:58:37.823227 20974 connection.cc:537] client connection to 127.16.221.67:41189 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250624 07:58:37.823588 20974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250624 07:58:37.824283 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18856
W20250624 07:58:37.827579 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250624 07:58:37.835234 18986 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.983995 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18856
I20250624 07:58:38.026615 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20075
I20250624 07:58:38.036019 20203 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.328924 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250624 07:58:38.545120 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20075
I20250624 07:58:38.617695 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18718
I20250624 07:58:38.626900 18844 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.627946 20977 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250624 07:58:38.774062 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18718
I20250624 07:58:38.804785 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19000
I20250624 07:58:38.813060 19123 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.858184 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250624 07:58:38.972692 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19000
I20250624 07:58:39.002192 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20806
I20250624 07:58:39.012413 20933 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:39.117089 20977 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250624 07:58:39.126052 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20806
I20250624 07:58:39.149242 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20953
I20250624 07:58:39.158293 21088 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:39.600015 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20953
I20250624 07:58:39.671206 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17277
I20250624 07:58:39.679673 17337 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:39.920516 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17277
2025-06-24T07:58:39Z chronyd exiting
I20250624 07:58:39.987267 17269 test_util.cc:183] -----------------------------------------------
I20250624 07:58:39.987383 17269 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0
Note: This is test shard 8 of 8.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RollingRestartArgs/RollingRestartITest
[ RUN ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3
2025-06-24T07:54:52Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-06-24T07:54:52Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250624 07:54:52.584996 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.221.126:44487
--webserver_interface=127.16.221.126
--webserver_port=0
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.221.126:44487
--location_mapping_cmd=/tmp/dist-test-taskaI4uEx/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/location-assignment.state --map /L0:3 --map /L1:4 --map /L2:4
--master_client_location_assignment_enabled=false with env {}
W20250624 07:54:52.872473 17277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:52.872745 17277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:52.872819 17277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:52.879987 17277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250624 07:54:52.880069 17277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:52.880121 17277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250624 07:54:52.880168 17277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250624 07:54:52.888782 17277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal
--location_mapping_cmd=/tmp/dist-test-taskaI4uEx/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/location-assignment.state --map /L0:3 --map /L1:4 --map /L2:4
--ipki_ca_key_size=768
--master_addresses=127.16.221.126:44487
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.221.126:44487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.221.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:52.890318 17277 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:52.892093 17277 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:52.903129 17282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:52.903950 17283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:52.904160 17285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:52.904647 17277 server_base.cc:1048] running on GCE node
I20250624 07:54:52.905716 17277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:52.907652 17277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:52.908882 17277 hybrid_clock.cc:648] HybridClock initialized: now 1750751692908832 us; error 39 us; skew 500 ppm
I20250624 07:54:52.909217 17277 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:52.911353 17277 webserver.cc:469] Webserver started at http://127.16.221.126:43027/ using document root <none> and password file <none>
I20250624 07:54:52.912065 17277 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:52.912170 17277 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:52.912518 17277 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:52.914925 17277 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/instance:
uuid: "b182c77196c04b549d02ed8a4be16054"
format_stamp: "Formatted at 2025-06-24 07:54:52 on dist-test-slave-sl8c"
I20250624 07:54:52.915614 17277 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal/instance:
uuid: "b182c77196c04b549d02ed8a4be16054"
format_stamp: "Formatted at 2025-06-24 07:54:52 on dist-test-slave-sl8c"
I20250624 07:54:52.920637 17277 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250624 07:54:52.924211 17291 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:52.925418 17277 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250624 07:54:52.925614 17277 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal
uuid: "b182c77196c04b549d02ed8a4be16054"
format_stamp: "Formatted at 2025-06-24 07:54:52 on dist-test-slave-sl8c"
I20250624 07:54:52.925791 17277 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:52.942721 17277 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:52.943538 17277 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:52.943827 17277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:52.962101 17277 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.126:44487
I20250624 07:54:52.962124 17342 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.126:44487 every 8 connection(s)
I20250624 07:54:52.963515 17277 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/info.pb
I20250624 07:54:52.967206 17343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250624 07:54:52.969452 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17277
I20250624 07:54:52.969730 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/wal/instance
I20250624 07:54:52.978461 17343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054: Bootstrap starting.
I20250624 07:54:52.981263 17343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054: Neither blocks nor log segments found. Creating new log.
I20250624 07:54:52.982343 17343 log.cc:826] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054: Log is configured to *not* fsync() on all Append() calls
I20250624 07:54:52.985515 17343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054: No bootstrap required, opened a new log
I20250624 07:54:52.991561 17343 raft_consensus.cc:357] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } }
I20250624 07:54:52.991865 17343 raft_consensus.cc:383] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250624 07:54:52.991978 17343 raft_consensus.cc:738] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b182c77196c04b549d02ed8a4be16054, State: Initialized, Role: FOLLOWER
I20250624 07:54:52.992501 17343 consensus_queue.cc:260] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } }
I20250624 07:54:52.992720 17343 raft_consensus.cc:397] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250624 07:54:52.992846 17343 raft_consensus.cc:491] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250624 07:54:52.993047 17343 raft_consensus.cc:3058] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 0 FOLLOWER]: Advancing to term 1
I20250624 07:54:52.995471 17343 raft_consensus.cc:513] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } }
I20250624 07:54:52.995971 17343 leader_election.cc:304] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b182c77196c04b549d02ed8a4be16054; no voters:
I20250624 07:54:52.996605 17343 leader_election.cc:290] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250624 07:54:52.996711 17348 raft_consensus.cc:2802] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 1 FOLLOWER]: Leader election won for term 1
I20250624 07:54:52.998631 17348 raft_consensus.cc:695] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [term 1 LEADER]: Becoming Leader. State: Replica: b182c77196c04b549d02ed8a4be16054, State: Running, Role: LEADER
I20250624 07:54:52.998920 17343 sys_catalog.cc:564] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [sys.catalog]: configured and running, proceeding with master startup.
I20250624 07:54:52.999140 17348 consensus_queue.cc:237] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } }
I20250624 07:54:53.002210 17350 sys_catalog.cc:455] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b182c77196c04b549d02ed8a4be16054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } } }
I20250624 07:54:53.002533 17350 sys_catalog.cc:458] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [sys.catalog]: This master's current role is: LEADER
I20250624 07:54:53.003007 17349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b182c77196c04b549d02ed8a4be16054. Latest consensus state: current_term: 1 leader_uuid: "b182c77196c04b549d02ed8a4be16054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b182c77196c04b549d02ed8a4be16054" member_type: VOTER last_known_addr { host: "127.16.221.126" port: 44487 } } }
I20250624 07:54:53.003314 17349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054 [sys.catalog]: This master's current role is: LEADER
I20250624 07:54:53.014993 17357 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250624 07:54:53.019315 17357 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250624 07:54:53.027420 17357 catalog_manager.cc:1349] Generated new cluster ID: 936208d2f9034f8983b67e3bc6440736
I20250624 07:54:53.027565 17357 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250624 07:54:53.046209 17357 catalog_manager.cc:1372] Generated new certificate authority record
I20250624 07:54:53.047739 17357 catalog_manager.cc:1506] Loading token signing keys...
I20250624 07:54:53.064383 17357 catalog_manager.cc:5955] T 00000000000000000000000000000000 P b182c77196c04b549d02ed8a4be16054: Generated new TSK 0
I20250624 07:54:53.065488 17357 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250624 07:54:53.082446 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.65:0
--local_ip_for_outbound_sockets=127.16.221.65
--webserver_interface=127.16.221.65
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:54:53.387923 17367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:53.388202 17367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:53.388293 17367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:53.395680 17367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:53.395845 17367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.65
I20250624 07:54:53.404572 17367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.221.65
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.65
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:53.406086 17367 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:53.407778 17367 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:53.418725 17375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:53.418825 17372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:53.418766 17373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:53.419555 17367 server_base.cc:1048] running on GCE node
I20250624 07:54:53.420044 17367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:53.420855 17367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:53.422053 17367 hybrid_clock.cc:648] HybridClock initialized: now 1750751693422003 us; error 44 us; skew 500 ppm
I20250624 07:54:53.422286 17367 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:53.424512 17367 webserver.cc:469] Webserver started at http://127.16.221.65:45465/ using document root <none> and password file <none>
I20250624 07:54:53.425195 17367 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:53.425276 17367 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:53.425580 17367 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:53.427708 17367 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/instance:
uuid: "1c0cb4e9f1d94dfbbfbaf9138aa4e655"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.428375 17367 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal/instance:
uuid: "1c0cb4e9f1d94dfbbfbaf9138aa4e655"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.432994 17367 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20250624 07:54:53.436085 17381 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.437312 17367 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250624 07:54:53.437446 17367 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
uuid: "1c0cb4e9f1d94dfbbfbaf9138aa4e655"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.437628 17367 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:53.458642 17367 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:53.459462 17367 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:53.459713 17367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:53.460634 17367 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:53.462160 17367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:53.462257 17367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.462365 17367 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:53.462466 17367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.493441 17367 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.65:45529
I20250624 07:54:53.493510 17493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.65:45529 every 8 connection(s)
I20250624 07:54:53.494920 17367 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
I20250624 07:54:53.501927 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17367
I20250624 07:54:53.502111 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal/instance
I20250624 07:54:53.505388 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.66:0
--local_ip_for_outbound_sockets=127.16.221.66
--webserver_interface=127.16.221.66
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:53.507604 17494 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:53.507987 17494 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:53.508921 17494 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:53.554132 17308 ts_manager.cc:194] Registered new tserver with Master: 1c0cb4e9f1d94dfbbfbaf9138aa4e655 (127.16.221.65:45529)
I20250624 07:54:53.556651 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.65:54277
W20250624 07:54:53.822378 17498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:53.822708 17498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:53.822811 17498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:53.830032 17498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:53.830205 17498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.66
I20250624 07:54:53.838860 17498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.221.66
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.66
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:53.840322 17498 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:53.842217 17498 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:53.853539 17507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:53.853564 17505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:53.853642 17504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:53.854509 17498 server_base.cc:1048] running on GCE node
I20250624 07:54:53.855058 17498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:53.855967 17498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:53.857164 17498 hybrid_clock.cc:648] HybridClock initialized: now 1750751693857121 us; error 42 us; skew 500 ppm
I20250624 07:54:53.857436 17498 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:53.859575 17498 webserver.cc:469] Webserver started at http://127.16.221.66:34317/ using document root <none> and password file <none>
I20250624 07:54:53.860286 17498 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:53.860391 17498 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:53.860745 17498 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:53.863119 17498 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/instance:
uuid: "e1bf694ea1724daaa49f0f0746a03eeb"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.863857 17498 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal/instance:
uuid: "e1bf694ea1724daaa49f0f0746a03eeb"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.868752 17498 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250624 07:54:53.871678 17513 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.872788 17498 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20250624 07:54:53.872925 17498 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
uuid: "e1bf694ea1724daaa49f0f0746a03eeb"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:54:53.873083 17498 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:53.891021 17498 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:53.891808 17498 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:53.892062 17498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:53.892920 17498 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:53.894385 17498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:53.894501 17498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.894609 17498 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:53.894681 17498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:53.926489 17498 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.66:42593
I20250624 07:54:53.926570 17625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.66:42593 every 8 connection(s)
I20250624 07:54:53.927896 17498 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
I20250624 07:54:53.937095 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17498
I20250624 07:54:53.937248 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal/instance
I20250624 07:54:53.940990 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.67:0
--local_ip_for_outbound_sockets=127.16.221.67
--webserver_interface=127.16.221.67
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:53.941195 17626 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:53.941579 17626 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:53.942502 17626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:53.985586 17308 ts_manager.cc:194] Registered new tserver with Master: e1bf694ea1724daaa49f0f0746a03eeb (127.16.221.66:42593)
I20250624 07:54:53.986671 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.66:57859
W20250624 07:54:54.252781 17629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:54.253082 17629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:54.253232 17629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:54.260589 17629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:54.260927 17629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.67
I20250624 07:54:54.269734 17629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.221.67
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.67
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:54.271210 17629 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:54.272846 17629 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:54.283776 17635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:54.283872 17638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:54.283870 17636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:54.285218 17629 server_base.cc:1048] running on GCE node
I20250624 07:54:54.285848 17629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:54.286859 17629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:54.288103 17629 hybrid_clock.cc:648] HybridClock initialized: now 1750751694288039 us; error 68 us; skew 500 ppm
I20250624 07:54:54.288398 17629 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:54.293419 17629 webserver.cc:469] Webserver started at http://127.16.221.67:33763/ using document root <none> and password file <none>
I20250624 07:54:54.294147 17629 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:54.294251 17629 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:54.294637 17629 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:54.296978 17629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/instance:
uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.297756 17629 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/instance:
uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.303273 17629 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250624 07:54:54.307116 17644 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.308326 17629 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250624 07:54:54.308470 17629 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.308658 17629 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:54.362723 17629 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:54.363607 17629 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:54.363878 17629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:54.364828 17629 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:54.366328 17629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:54.366406 17629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.366523 17629 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:54.366598 17629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.398058 17629 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.67:41189
I20250624 07:54:54.398115 17756 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.67:41189 every 8 connection(s)
I20250624 07:54:54.399451 17629 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
I20250624 07:54:54.408529 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17629
I20250624 07:54:54.408695 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/instance
I20250624 07:54:54.411881 17757 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:54.412083 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.68:0
--local_ip_for_outbound_sockets=127.16.221.68
--webserver_interface=127.16.221.68
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:54.412218 17757 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:54.412981 17757 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:54.454303 17308 ts_manager.cc:194] Registered new tserver with Master: 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189)
I20250624 07:54:54.455756 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.67:41935
I20250624 07:54:54.559922 17494 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:54.708979 17760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:54.709268 17760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:54.709393 17760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:54.716707 17760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:54.716909 17760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.68
I20250624 07:54:54.725595 17760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.16.221.68
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.68
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:54.727126 17760 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:54.728837 17760 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:54.739420 17769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:54.739643 17767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:54.739615 17766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:54.740446 17760 server_base.cc:1048] running on GCE node
I20250624 07:54:54.740906 17760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:54.741689 17760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:54.742910 17760 hybrid_clock.cc:648] HybridClock initialized: now 1750751694742863 us; error 45 us; skew 500 ppm
I20250624 07:54:54.743180 17760 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:54.745272 17760 webserver.cc:469] Webserver started at http://127.16.221.68:42787/ using document root <none> and password file <none>
I20250624 07:54:54.745955 17760 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:54.746057 17760 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:54.746431 17760 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:54.748762 17760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/instance:
uuid: "d33fd96ef7d44ad8805718466ca9447f"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.749485 17760 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal/instance:
uuid: "d33fd96ef7d44ad8805718466ca9447f"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.754011 17760 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20250624 07:54:54.756958 17775 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.758143 17760 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250624 07:54:54.758272 17760 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
uuid: "d33fd96ef7d44ad8805718466ca9447f"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:54:54.758448 17760 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:54.772707 17760 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:54.773412 17760 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:54.773638 17760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:54.774626 17760 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:54.776069 17760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:54.776156 17760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.776257 17760 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:54.776320 17760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:54.807112 17760 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.68:42243
I20250624 07:54:54.807188 17887 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.68:42243 every 8 connection(s)
I20250624 07:54:54.808378 17760 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
I20250624 07:54:54.808737 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17760
I20250624 07:54:54.808917 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal/instance
I20250624 07:54:54.812522 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.69:0
--local_ip_for_outbound_sockets=127.16.221.69
--webserver_interface=127.16.221.69
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:54.822384 17888 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:54.822777 17888 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:54.823617 17888 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:54.867465 17308 ts_manager.cc:194] Registered new tserver with Master: d33fd96ef7d44ad8805718466ca9447f (127.16.221.68:42243)
I20250624 07:54:54.868716 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.68:47557
I20250624 07:54:54.989899 17626 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:55.116737 17891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:55.117050 17891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:55.117187 17891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:55.124420 17891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:55.124622 17891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.69
I20250624 07:54:55.133102 17891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
--webserver_interface=127.16.221.69
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.69
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:55.134598 17891 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:55.136426 17891 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:55.147289 17897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.147218 17900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.147253 17898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:55.148634 17891 server_base.cc:1048] running on GCE node
I20250624 07:54:55.149153 17891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:55.150008 17891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:55.151234 17891 hybrid_clock.cc:648] HybridClock initialized: now 1750751695151172 us; error 55 us; skew 500 ppm
I20250624 07:54:55.151510 17891 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:55.153642 17891 webserver.cc:469] Webserver started at http://127.16.221.69:37317/ using document root <none> and password file <none>
I20250624 07:54:55.154340 17891 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:55.154455 17891 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:55.154783 17891 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:55.157159 17891 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/instance:
uuid: "4f219d2e8b954461973a8ca8680a6383"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.157963 17891 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal/instance:
uuid: "4f219d2e8b954461973a8ca8680a6383"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.162807 17891 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20250624 07:54:55.165830 17906 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.167127 17891 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250624 07:54:55.167273 17891 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
uuid: "4f219d2e8b954461973a8ca8680a6383"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.167449 17891 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:55.180060 17891 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:55.180759 17891 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:55.181007 17891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:55.181804 17891 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:55.183312 17891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:55.183406 17891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.183512 17891 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:55.183583 17891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.214347 17891 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.69:35369
I20250624 07:54:55.214402 18018 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.69:35369 every 8 connection(s)
I20250624 07:54:55.215651 17891 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
I20250624 07:54:55.218818 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 17891
I20250624 07:54:55.219007 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal/instance
I20250624 07:54:55.222455 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.70:0
--local_ip_for_outbound_sockets=127.16.221.70
--webserver_interface=127.16.221.70
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:55.229179 18019 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:55.229496 18019 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:55.230477 18019 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:55.272080 17308 ts_manager.cc:194] Registered new tserver with Master: 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:54:55.273319 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.69:56965
I20250624 07:54:55.459276 17757 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:55.522732 18022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:55.523044 18022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:55.523178 18022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:55.530296 18022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:55.530519 18022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.70
I20250624 07:54:55.539091 18022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
--webserver_interface=127.16.221.70
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.70
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:55.540565 18022 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:55.542305 18022 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:55.553284 18028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.553138 18029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.553287 18031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:55.554231 18022 server_base.cc:1048] running on GCE node
I20250624 07:54:55.554771 18022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:55.555634 18022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:55.556819 18022 hybrid_clock.cc:648] HybridClock initialized: now 1750751695556780 us; error 32 us; skew 500 ppm
I20250624 07:54:55.557118 18022 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:55.559305 18022 webserver.cc:469] Webserver started at http://127.16.221.70:46621/ using document root <none> and password file <none>
I20250624 07:54:55.559988 18022 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:55.560109 18022 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:55.560433 18022 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:55.562885 18022 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/instance:
uuid: "bede5abcb6174794bd2283843157bcef"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.563700 18022 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal/instance:
uuid: "bede5abcb6174794bd2283843157bcef"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.568603 18022 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250624 07:54:55.571879 18037 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.573076 18022 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250624 07:54:55.573223 18022 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
uuid: "bede5abcb6174794bd2283843157bcef"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.573412 18022 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:55.586283 18022 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:55.587047 18022 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:55.587280 18022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:55.588119 18022 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:55.589522 18022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:55.589612 18022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.589708 18022 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:55.589780 18022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.620422 18022 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.70:41185
I20250624 07:54:55.620472 18149 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.70:41185 every 8 connection(s)
I20250624 07:54:55.621711 18022 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
I20250624 07:54:55.629586 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18022
I20250624 07:54:55.629762 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal/instance
I20250624 07:54:55.633278 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.71:0
--local_ip_for_outbound_sockets=127.16.221.71
--webserver_interface=127.16.221.71
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:55.633936 18150 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:55.634300 18150 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:55.635316 18150 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:55.677292 17308 ts_manager.cc:194] Registered new tserver with Master: bede5abcb6174794bd2283843157bcef (127.16.221.70:41185)
I20250624 07:54:55.678808 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.70:36993
I20250624 07:54:55.871568 17888 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:55.933238 18153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:55.933537 18153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:55.933627 18153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:55.940723 18153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:55.940913 18153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.71
I20250624 07:54:55.949681 18153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
--webserver_interface=127.16.221.71
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.71
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:55.951244 18153 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:55.952992 18153 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:55.964185 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.964272 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:55.964570 18162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:55.965049 18153 server_base.cc:1048] running on GCE node
I20250624 07:54:55.965512 18153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:55.966248 18153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:55.967404 18153 hybrid_clock.cc:648] HybridClock initialized: now 1750751695967371 us; error 30 us; skew 500 ppm
I20250624 07:54:55.967653 18153 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:55.969703 18153 webserver.cc:469] Webserver started at http://127.16.221.71:43087/ using document root <none> and password file <none>
I20250624 07:54:55.970333 18153 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:55.970440 18153 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:55.970759 18153 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:55.972896 18153 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/instance:
uuid: "1fe60cc8e38948b694506041d7734713"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.973551 18153 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal/instance:
uuid: "1fe60cc8e38948b694506041d7734713"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.978188 18153 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20250624 07:54:55.981063 18168 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:55.982192 18153 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250624 07:54:55.982309 18153 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
uuid: "1fe60cc8e38948b694506041d7734713"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:54:55.982513 18153 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:56.007007 18153 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:56.007793 18153 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:56.008044 18153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:56.008934 18153 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:56.010366 18153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:56.010474 18153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.010569 18153 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:56.010640 18153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.042045 18153 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.71:44909
I20250624 07:54:56.042119 18280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.71:44909 every 8 connection(s)
I20250624 07:54:56.043345 18153 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
I20250624 07:54:56.050473 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18153
I20250624 07:54:56.050644 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal/instance
I20250624 07:54:56.053679 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.72:0
--local_ip_for_outbound_sockets=127.16.221.72
--webserver_interface=127.16.221.72
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:56.055893 18281 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:56.056252 18281 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:56.057155 18281 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:56.123307 17308 ts_manager.cc:194] Registered new tserver with Master: 1fe60cc8e38948b694506041d7734713 (127.16.221.71:44909)
I20250624 07:54:56.124647 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.71:50835
I20250624 07:54:56.276016 18019 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:56.362537 18284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:56.362864 18284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:56.363009 18284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:56.370105 18284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:56.370254 18284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.72
I20250624 07:54:56.379359 18284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
--webserver_interface=127.16.221.72
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.72
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:56.380841 18284 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:56.382584 18284 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:56.393584 18291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:56.393540 18293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:56.393777 18290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:56.394436 18284 server_base.cc:1048] running on GCE node
I20250624 07:54:56.395001 18284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:56.395897 18284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:56.397095 18284 hybrid_clock.cc:648] HybridClock initialized: now 1750751696397042 us; error 45 us; skew 500 ppm
I20250624 07:54:56.397369 18284 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:56.399948 18284 webserver.cc:469] Webserver started at http://127.16.221.72:34785/ using document root <none> and password file <none>
I20250624 07:54:56.400902 18284 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:56.401058 18284 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:56.401460 18284 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:56.403844 18284 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/instance:
uuid: "8150d3eeb1c84234a13484098d17ce8f"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.404587 18284 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal/instance:
uuid: "8150d3eeb1c84234a13484098d17ce8f"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.409466 18284 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250624 07:54:56.412567 18299 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.413782 18284 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250624 07:54:56.413905 18284 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
uuid: "8150d3eeb1c84234a13484098d17ce8f"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.414060 18284 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:56.437199 18284 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:56.437961 18284 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:56.438169 18284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:56.439074 18284 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:56.440557 18284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:56.440645 18284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.440712 18284 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:56.440758 18284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.472010 18284 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.72:40741
I20250624 07:54:56.472074 18411 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.72:40741 every 8 connection(s)
I20250624 07:54:56.473661 18284 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
I20250624 07:54:56.480011 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18284
I20250624 07:54:56.480155 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal/instance
I20250624 07:54:56.483239 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.73:0
--local_ip_for_outbound_sockets=127.16.221.73
--webserver_interface=127.16.221.73
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:54:56.486719 18412 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:56.487123 18412 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:56.487946 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:56.544308 17308 ts_manager.cc:194] Registered new tserver with Master: 8150d3eeb1c84234a13484098d17ce8f (127.16.221.72:40741)
I20250624 07:54:56.545689 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.72:47121
I20250624 07:54:56.681625 18150 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:54:56.801354 18415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:54:56.801638 18415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:54:56.801724 18415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:54:56.809010 18415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:54:56.809156 18415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.73
I20250624 07:54:56.817677 18415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.73:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
--webserver_interface=127.16.221.73
--webserver_port=0
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.73
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:54:56.819187 18415 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:54:56.820982 18415 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:54:56.831792 18422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:56.831746 18424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:54:56.831841 18421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:54:56.832607 18415 server_base.cc:1048] running on GCE node
I20250624 07:54:56.833170 18415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:54:56.834015 18415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:54:56.835238 18415 hybrid_clock.cc:648] HybridClock initialized: now 1750751696835191 us; error 34 us; skew 500 ppm
I20250624 07:54:56.835544 18415 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:54:56.837568 18415 webserver.cc:469] Webserver started at http://127.16.221.73:35519/ using document root <none> and password file <none>
I20250624 07:54:56.838240 18415 fs_manager.cc:362] Metadata directory not provided
I20250624 07:54:56.838322 18415 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:54:56.838651 18415 server_base.cc:896] This appears to be a new deployment of Kudu; creating new FS layout
I20250624 07:54:56.840829 18415 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/instance:
uuid: "6cbd668c726a4b1298e789270a0031e2"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.841555 18415 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal/instance:
uuid: "6cbd668c726a4b1298e789270a0031e2"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.846484 18415 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250624 07:54:56.849612 18430 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.850826 18415 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250624 07:54:56.850962 18415 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
uuid: "6cbd668c726a4b1298e789270a0031e2"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:54:56.851130 18415 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:54:56.879557 18415 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:54:56.880412 18415 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:54:56.880690 18415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:54:56.881569 18415 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:54:56.883165 18415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:54:56.883249 18415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.883359 18415 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:54:56.883436 18415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:54:56.914288 18415 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.73:38265
I20250624 07:54:56.914355 18542 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.73:38265 every 8 connection(s)
I20250624 07:54:56.915689 18415 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
I20250624 07:54:56.920742 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18415
I20250624 07:54:56.920929 17269 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal/instance
I20250624 07:54:56.928890 18543 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:54:56.929226 18543 heartbeater.cc:461] Registering TS with master...
I20250624 07:54:56.930044 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:54:56.973284 17308 ts_manager.cc:194] Registered new tserver with Master: 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:54:56.974210 17308 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.73:37481
I20250624 07:54:56.976580 17269 external_mini_cluster.cc:934] 9 TS(s) registered with all masters
I20250624 07:54:56.996798 17269 test_util.cc:276] Using random seed: -1526904419
I20250624 07:54:57.015763 17308 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56308:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250624 07:54:57.046458 17692 tablet_service.cc:1468] Processing CreateTablet for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa (DEFAULT_TABLE table=test-workload [id=f9df05cf71804129be5f76955c6bd5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250624 07:54:57.048143 17692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250624 07:54:57.057459 18478 tablet_service.cc:1468] Processing CreateTablet for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa (DEFAULT_TABLE table=test-workload [id=f9df05cf71804129be5f76955c6bd5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250624 07:54:57.058990 18478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250624 07:54:57.059413 18567 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap starting.
I20250624 07:54:57.059089 17954 tablet_service.cc:1468] Processing CreateTablet for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa (DEFAULT_TABLE table=test-workload [id=f9df05cf71804129be5f76955c6bd5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250624 07:54:57.060781 17954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ca5ea09d51f4eb390c4433fec8ed9fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250624 07:54:57.062754 18567 tablet_bootstrap.cc:654] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Neither blocks nor log segments found. Creating new log.
I20250624 07:54:57.063731 18567 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Log is configured to *not* fsync() on all Append() calls
I20250624 07:54:57.065970 18567 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: No bootstrap required, opened a new log
I20250624 07:54:57.066238 18567 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20250624 07:54:57.070780 18567 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.071043 18567 raft_consensus.cc:383] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250624 07:54:57.071175 18567 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439cf2177bdb4c19aa8449b6cbd440a7, State: Initialized, Role: FOLLOWER
I20250624 07:54:57.071754 18567 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.073462 18567 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250624 07:54:57.074597 18570 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap starting.
I20250624 07:54:57.078269 18570 tablet_bootstrap.cc:654] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Neither blocks nor log segments found. Creating new log.
I20250624 07:54:57.078646 18571 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap starting.
I20250624 07:54:57.079568 18570 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Log is configured to *not* fsync() on all Append() calls
I20250624 07:54:57.081979 18571 tablet_bootstrap.cc:654] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Neither blocks nor log segments found. Creating new log.
I20250624 07:54:57.082317 18570 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: No bootstrap required, opened a new log
I20250624 07:54:57.082607 18570 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20250624 07:54:57.083379 18571 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Log is configured to *not* fsync() on all Append() calls
I20250624 07:54:57.085999 18571 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: No bootstrap required, opened a new log
I20250624 07:54:57.086248 18571 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20250624 07:54:57.089185 18570 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.089520 18570 raft_consensus.cc:383] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250624 07:54:57.089636 18570 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Initialized, Role: FOLLOWER
I20250624 07:54:57.090278 18570 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.091251 18543 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:54:57.091877 18570 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20250624 07:54:57.092748 18571 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.092986 18571 raft_consensus.cc:383] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250624 07:54:57.093091 18571 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f219d2e8b954461973a8ca8680a6383, State: Initialized, Role: FOLLOWER
I20250624 07:54:57.093590 18571 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.095217 18571 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20250624 07:54:57.127400 18281 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:54:57.132345 18574 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250624 07:54:57.132591 18574 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.133699 18575 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250624 07:54:57.133929 18575 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.134604 18574 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:54:57.137856 18575 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:54:57.142251 17974 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:54:57.142839 17974 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 0.
I20250624 07:54:57.143303 17712 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:54:57.143646 18434 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:54:57.143812 17712 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 0.
I20250624 07:54:57.144068 18574 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250624 07:54:57.144214 18574 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250624 07:54:57.144315 18574 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 0 FOLLOWER]: Advancing to term 1
I20250624 07:54:57.146770 18574 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.147127 17712 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:54:57.147351 17712 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f219d2e8b954461973a8ca8680a6383 in term 0.
I20250624 07:54:57.147881 17712 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
I20250624 07:54:57.147979 17907 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383; no voters:
I20250624 07:54:57.148070 17712 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 0 FOLLOWER]: Advancing to term 1
I20250624 07:54:57.148443 18575 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250624 07:54:57.148582 18575 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250624 07:54:57.148695 18575 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 0 FOLLOWER]: Advancing to term 1
I20250624 07:54:57.150754 18574 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 1 election: Requested vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:54:57.151295 17974 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383"
I20250624 07:54:57.152139 17712 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 1.
I20250624 07:54:57.152758 18431 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:54:57.152756 18575 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.153254 17974 raft_consensus.cc:2391] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6cbd668c726a4b1298e789270a0031e2 in current term 1: Already voted for candidate 4f219d2e8b954461973a8ca8680a6383 in this term.
I20250624 07:54:57.153556 18575 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 1 election: Requested vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
W20250624 07:54:57.153672 17758 tablet.cc:2378] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250624 07:54:57.154726 18574 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 FOLLOWER]: Leader election won for term 1
I20250624 07:54:57.154711 17711 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
I20250624 07:54:57.155125 17711 raft_consensus.cc:2391] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4f219d2e8b954461973a8ca8680a6383 in current term 1: Already voted for candidate 6cbd668c726a4b1298e789270a0031e2 in this term.
I20250624 07:54:57.155335 18574 raft_consensus.cc:695] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 LEADER]: Becoming Leader. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Running, Role: LEADER
I20250624 07:54:57.155917 18574 consensus_queue.cc:237] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:54:57.156304 18498 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:54:57.156351 18497 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2"
I20250624 07:54:57.157272 17909 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f219d2e8b954461973a8ca8680a6383; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:54:57.157531 18575 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250624 07:54:57.161137 17308 catalog_manager.cc:5582] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73). New cstate: current_term: 1 leader_uuid: "6cbd668c726a4b1298e789270a0031e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } health_report { overall_health: HEALTHY } } }
W20250624 07:54:57.167205 18544 tablet.cc:2378] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250624 07:54:57.197364 17269 maintenance_mode-itest.cc:745] Restarting batch of 3 tservers: bede5abcb6174794bd2283843157bcef,d33fd96ef7d44ad8805718466ca9447f,1fe60cc8e38948b694506041d7734713
W20250624 07:54:57.218276 18020 tablet.cc:2378] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250624 07:54:57.239917 18574 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250624 07:54:57.250558 18574 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250624 07:54:57.548456 18412 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:54:57.830181 17308 ts_manager.cc:295] Set tserver state for 1fe60cc8e38948b694506041d7734713 to MAINTENANCE_MODE
I20250624 07:54:57.848816 17308 ts_manager.cc:295] Set tserver state for d33fd96ef7d44ad8805718466ca9447f to MAINTENANCE_MODE
I20250624 07:54:58.780716 17308 ts_manager.cc:295] Set tserver state for bede5abcb6174794bd2283843157bcef to MAINTENANCE_MODE
I20250624 07:55:00.890690 17823 tablet_service.cc:1423] Tablet server d33fd96ef7d44ad8805718466ca9447f set to quiescing
I20250624 07:55:00.890956 17823 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:01.891839 18216 tablet_service.cc:1423] Tablet server 1fe60cc8e38948b694506041d7734713 set to quiescing
I20250624 07:55:01.892112 18216 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:03.695204 18085 tablet_service.cc:1423] Tablet server bede5abcb6174794bd2283843157bcef set to quiescing
I20250624 07:55:03.695421 18085 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:05.273710 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18022
I20250624 07:55:05.327889 18145 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:05.511485 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18022
I20250624 07:55:05.540584 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.70:41185
--local_ip_for_outbound_sockets=127.16.221.70
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=46621
--webserver_interface=127.16.221.70
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:55:06.183051 18718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:06.183643 18718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:06.183918 18718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:06.202641 18718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:06.203083 18718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.70
I20250624 07:55:06.222656 18718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.70:41185
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
--webserver_interface=127.16.221.70
--webserver_port=46621
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.70
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:06.225152 18718 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:06.228219 18718 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:06.257635 18725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:06.254006 18726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:06.280064 18729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:06.281857 18718 server_base.cc:1048] running on GCE node
I20250624 07:55:06.282738 18718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:06.283969 18718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:06.285312 18718 hybrid_clock.cc:648] HybridClock initialized: now 1750751706285244 us; error 50 us; skew 500 ppm
I20250624 07:55:06.285791 18718 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:06.322548 18718 webserver.cc:469] Webserver started at http://127.16.221.70:46621/ using document root <none> and password file <none>
I20250624 07:55:06.323840 18718 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:06.324057 18718 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:06.334704 18718 fs_manager.cc:714] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20250624 07:55:06.338479 18734 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:06.341491 18718 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20250624 07:55:06.342192 18718 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
uuid: "bede5abcb6174794bd2283843157bcef"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:55:06.343423 18718 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:06.384093 18718 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:06.392123 18718 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:06.392431 18718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:06.402447 18718 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:06.509496 18718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:55:06.509725 18718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250624 07:55:06.509969 18718 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:55:06.510102 18718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:07.313117 18848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.70:41185 every 8 connection(s)
I20250624 07:55:07.313509 18718 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.70:41185
I20250624 07:55:07.315534 18718 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb
I20250624 07:55:07.320170 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18718
I20250624 07:55:07.320415 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17760
I20250624 07:55:07.367197 17883 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:07.372530 18849 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:07.372887 18849 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:07.374007 18849 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:07.380666 17304 ts_manager.cc:194] Re-registered known tserver with Master: bede5abcb6174794bd2283843157bcef (127.16.221.70:41185)
I20250624 07:55:07.381858 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.70:43221
I20250624 07:55:07.511034 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17760
I20250624 07:55:07.532989 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.68:42243
--local_ip_for_outbound_sockets=127.16.221.68
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=42787
--webserver_interface=127.16.221.68
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:55:08.094113 18856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:08.094647 18856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:08.094830 18856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:08.114174 18856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:08.115475 18856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.68
I20250624 07:55:08.136883 18856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.68:42243
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.16.221.68
--webserver_port=42787
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.68
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:08.140214 18856 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:08.142894 18856 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:08.196990 18866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:08.205864 18869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:08.230958 18856 server_base.cc:1048] running on GCE node
W20250624 07:55:08.231938 18864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:08.235606 18856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:08.244899 18856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:08.262287 18856 hybrid_clock.cc:648] HybridClock initialized: now 1750751708262237 us; error 40 us; skew 500 ppm
I20250624 07:55:08.262876 18856 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:08.266518 18856 webserver.cc:469] Webserver started at http://127.16.221.68:42787/ using document root <none> and password file <none>
I20250624 07:55:08.267781 18856 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:08.267959 18856 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:08.290647 18856 fs_manager.cc:714] Time spent opening directory manager: real 0.018s user 0.006s sys 0.004s
I20250624 07:55:08.312978 18874 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:08.314765 18856 fs_manager.cc:730] Time spent opening block manager: real 0.022s user 0.000s sys 0.010s
I20250624 07:55:08.314989 18856 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
uuid: "d33fd96ef7d44ad8805718466ca9447f"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:55:08.320834 18856 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:08.385849 18849 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:55:08.520977 18856 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:08.529799 18856 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:08.534204 18856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:08.541059 18856 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:08.543941 18856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:55:08.544028 18856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:08.544169 18856 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:55:08.544224 18856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:09.734840 18992 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.68:42243 every 8 connection(s)
I20250624 07:55:09.734799 18856 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.68:42243
I20250624 07:55:09.737363 18856 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb
I20250624 07:55:09.743328 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 18856
I20250624 07:55:09.743605 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18153
I20250624 07:55:09.792794 18276 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:09.914103 18993 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:09.914600 18993 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:09.915678 18993 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:09.917567 17304 ts_manager.cc:194] Re-registered known tserver with Master: d33fd96ef7d44ad8805718466ca9447f (127.16.221.68:42243)
I20250624 07:55:09.918597 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.68:37697
I20250624 07:55:09.998119 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18153
I20250624 07:55:10.030931 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.71:44909
--local_ip_for_outbound_sockets=127.16.221.71
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=43087
--webserver_interface=127.16.221.71
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:55:10.723683 19000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:10.724222 19000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:10.724406 19000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:10.738111 19000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:10.738303 19000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.71
I20250624 07:55:10.753079 19000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.71:44909
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
--webserver_interface=127.16.221.71
--webserver_port=43087
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.71
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:10.761324 19000 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:10.763830 19000 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:10.815718 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:10.817589 19007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:10.818104 19009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:10.819631 19000 server_base.cc:1048] running on GCE node
I20250624 07:55:10.820593 19000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:10.821768 19000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:10.822994 19000 hybrid_clock.cc:648] HybridClock initialized: now 1750751710822922 us; error 58 us; skew 500 ppm
I20250624 07:55:10.823382 19000 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:10.836923 19000 webserver.cc:469] Webserver started at http://127.16.221.71:43087/ using document root <none> and password file <none>
I20250624 07:55:10.847936 19000 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:10.848105 19000 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:10.872602 19000 fs_manager.cc:714] Time spent opening directory manager: real 0.017s user 0.005s sys 0.000s
I20250624 07:55:10.880986 19015 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:10.924499 18993 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:55:10.924801 19000 fs_manager.cc:730] Time spent opening block manager: real 0.049s user 0.005s sys 0.001s
I20250624 07:55:10.925048 19000 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
uuid: "1fe60cc8e38948b694506041d7734713"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:55:10.926077 19000 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:11.089150 19000 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:11.090267 19000 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:11.090641 19000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:11.096557 19000 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:11.100633 19000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:55:11.100775 19000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20250624 07:55:11.100941 19000 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:55:11.101061 19000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:11.430325 19128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.71:44909 every 8 connection(s)
I20250624 07:55:11.430699 19000 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.71:44909
I20250624 07:55:11.432740 19000 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb
I20250624 07:55:11.439661 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 19000
I20250624 07:55:11.511731 19129 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:11.512272 19129 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:11.513588 19129 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:11.515805 17304 ts_manager.cc:194] Re-registered known tserver with Master: 1fe60cc8e38948b694506041d7734713 (127.16.221.71:44909)
I20250624 07:55:11.516947 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.71:39443
I20250624 07:55:12.521061 19129 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:55:12.621255 17429 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.696424 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.703213 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.710814 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:55:12.719071 17692 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.734387 17954 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.829092 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.852851 17561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:12.871457 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:55:14.370464 18431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.221.67:41189, user_credentials={real_user=slave}} blocked reactor thread for 44171.3us
I20250624 07:55:14.444803 17308 ts_manager.cc:284] Unset tserver state for d33fd96ef7d44ad8805718466ca9447f from MAINTENANCE_MODE
I20250624 07:55:14.447389 18849 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.449172 17757 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.450894 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.450974 18019 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.463713 17302 ts_manager.cc:284] Unset tserver state for bede5abcb6174794bd2283843157bcef from MAINTENANCE_MODE
I20250624 07:55:14.527323 19129 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.598618 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.606719 17494 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:14.742861 17304 ts_manager.cc:284] Unset tserver state for 1fe60cc8e38948b694506041d7734713 from MAINTENANCE_MODE
I20250624 07:55:14.944308 18993 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.037314 17626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.468303 18849 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.470757 17757 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.474280 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.475525 18019 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.530716 19129 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.603053 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:15.610126 17494 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:17.692900 17269 maintenance_mode-itest.cc:745] Restarting batch of 3 tservers: e1bf694ea1724daaa49f0f0746a03eeb,1c0cb4e9f1d94dfbbfbaf9138aa4e655,439cf2177bdb4c19aa8449b6cbd440a7
I20250624 07:55:18.940596 17303 ts_manager.cc:295] Set tserver state for 439cf2177bdb4c19aa8449b6cbd440a7 to MAINTENANCE_MODE
I20250624 07:55:18.958820 17303 ts_manager.cc:295] Set tserver state for e1bf694ea1724daaa49f0f0746a03eeb to MAINTENANCE_MODE
I20250624 07:55:19.215140 17304 ts_manager.cc:295] Set tserver state for 1c0cb4e9f1d94dfbbfbaf9138aa4e655 to MAINTENANCE_MODE
W20250624 07:55:21.695528 18583 meta_cache.cc:1261] Time spent looking up entry by key: real 0.063s user 0.000s sys 0.000s
I20250624 07:55:22.426828 17429 tablet_service.cc:1423] Tablet server 1c0cb4e9f1d94dfbbfbaf9138aa4e655 set to quiescing
I20250624 07:55:22.427132 17429 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:22.954852 17692 tablet_service.cc:1423] Tablet server 439cf2177bdb4c19aa8449b6cbd440a7 set to quiescing
I20250624 07:55:22.955111 17692 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:23.212321 17561 tablet_service.cc:1423] Tablet server e1bf694ea1724daaa49f0f0746a03eeb set to quiescing
I20250624 07:55:23.212611 17561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:25.410761 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17498
I20250624 07:55:25.485615 17621 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:25.658900 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17498
I20250624 07:55:25.688190 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.66:42593
--local_ip_for_outbound_sockets=127.16.221.66
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=34317
--webserver_interface=127.16.221.66
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:55:26.314689 19354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:26.315208 19354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:26.315379 19354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:26.330641 19354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:26.330940 19354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.66
I20250624 07:55:26.343461 19354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.66:42593
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.221.66
--webserver_port=34317
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.66
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:26.345885 19354 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:26.356204 19354 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:26.375574 19364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:26.376209 19363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:26.377774 19366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:26.382783 19354 server_base.cc:1048] running on GCE node
I20250624 07:55:26.384511 19354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:26.387483 19354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:26.391479 19354 hybrid_clock.cc:648] HybridClock initialized: now 1750751726391410 us; error 62 us; skew 500 ppm
I20250624 07:55:26.392066 19354 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:26.446715 19354 webserver.cc:469] Webserver started at http://127.16.221.66:34317/ using document root <none> and password file <none>
I20250624 07:55:26.447868 19354 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:26.448030 19354 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:26.466599 19354 fs_manager.cc:714] Time spent opening directory manager: real 0.016s user 0.005s sys 0.000s
I20250624 07:55:26.477438 19372 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:26.484259 19354 fs_manager.cc:730] Time spent opening block manager: real 0.015s user 0.001s sys 0.002s
I20250624 07:55:26.484462 19354 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
uuid: "e1bf694ea1724daaa49f0f0746a03eeb"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:55:26.485443 19354 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:26.537289 19354 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:26.538708 19354 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:26.539326 19354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:26.576887 19354 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:26.585784 19354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:55:26.585980 19354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250624 07:55:26.586210 19354 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:55:26.586342 19354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:27.075330 19485 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.66:42593 every 8 connection(s)
I20250624 07:55:27.075634 19354 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.66:42593
I20250624 07:55:27.077562 19354 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb
I20250624 07:55:27.089027 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 19354
I20250624 07:55:27.089210 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17367
I20250624 07:55:27.143162 17489 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:27.162143 19486 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:27.162488 19486 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:27.163563 19486 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:27.165292 17304 ts_manager.cc:194] Re-registered known tserver with Master: e1bf694ea1724daaa49f0f0746a03eeb (127.16.221.66:42593)
I20250624 07:55:27.166288 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.66:53297
I20250624 07:55:27.383962 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17367
I20250624 07:55:27.443136 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.65:45529
--local_ip_for_outbound_sockets=127.16.221.65
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=45465
--webserver_interface=127.16.221.65
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250624 07:55:28.173012 19486 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:55:28.185772 19496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:28.186196 19496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:28.186350 19496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:28.198367 19496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:28.198727 19496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.65
I20250624 07:55:28.213191 19496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.65:45529
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.221.65
--webserver_port=45465
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.65
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:28.215451 19496 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:28.217749 19496 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:28.272745 19502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:28.299376 19503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:28.315865 19507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:28.317011 19496 server_base.cc:1048] running on GCE node
I20250624 07:55:28.327879 19496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:28.329229 19496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:28.330505 19496 hybrid_clock.cc:648] HybridClock initialized: now 1750751728330446 us; error 49 us; skew 500 ppm
I20250624 07:55:28.330868 19496 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:28.344964 19496 webserver.cc:469] Webserver started at http://127.16.221.65:45465/ using document root <none> and password file <none>
I20250624 07:55:28.346101 19496 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:28.346293 19496 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:28.365787 19496 fs_manager.cc:714] Time spent opening directory manager: real 0.008s user 0.005s sys 0.001s
I20250624 07:55:28.369871 19512 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:28.378350 19496 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.001s sys 0.003s
I20250624 07:55:28.378667 19496 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
uuid: "1c0cb4e9f1d94dfbbfbaf9138aa4e655"
format_stamp: "Formatted at 2025-06-24 07:54:53 on dist-test-slave-sl8c"
I20250624 07:55:28.379714 19496 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:28.499836 19496 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:28.501013 19496 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:28.501348 19496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:28.504091 19496 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:28.515251 19496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:55:28.515372 19496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:28.515542 19496 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:55:28.515615 19496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:29.050024 19496 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.65:45529
I20250624 07:55:29.050082 19625 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.65:45529 every 8 connection(s)
I20250624 07:55:29.052176 19496 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb
I20250624 07:55:29.061540 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 19496
I20250624 07:55:29.061750 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17629
I20250624 07:55:29.128436 17752 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:55:29.223430 19626 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:29.223978 19626 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:29.226020 19626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:29.233528 17304 ts_manager.cc:194] Re-registered known tserver with Master: 1c0cb4e9f1d94dfbbfbaf9138aa4e655 (127.16.221.65:45529)
I20250624 07:55:29.234668 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.65:41515
I20250624 07:55:29.624406 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17629
I20250624 07:55:29.723776 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.67:41189
--local_ip_for_outbound_sockets=127.16.221.67
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=33763
--webserver_interface=127.16.221.67
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:55:29.723752 18431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.221.67:41189 (error 108)
W20250624 07:55:29.727715 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250624 07:55:30.207845 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250624 07:55:30.238265 19626 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:55:30.351920 19638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:55:30.352416 19638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:55:30.352640 19638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:55:30.364475 19638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:55:30.364763 19638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.67
I20250624 07:55:30.379431 19638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.67:41189
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.221.67
--webserver_port=33763
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.67
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:55:30.381639 19638 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:55:30.384236 19638 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:55:30.422106 19645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:30.423887 19646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:55:30.428371 19648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:55:30.432500 19638 server_base.cc:1048] running on GCE node
I20250624 07:55:30.433351 19638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:55:30.435153 19638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:55:30.438935 19638 hybrid_clock.cc:648] HybridClock initialized: now 1750751730438872 us; error 71 us; skew 500 ppm
I20250624 07:55:30.439368 19638 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:55:30.448103 19638 webserver.cc:469] Webserver started at http://127.16.221.67:33763/ using document root <none> and password file <none>
I20250624 07:55:30.449216 19638 fs_manager.cc:362] Metadata directory not provided
I20250624 07:55:30.449368 19638 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:55:30.462446 19638 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.003s sys 0.001s
I20250624 07:55:30.475950 19654 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:55:30.478305 19638 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.004s sys 0.001s
I20250624 07:55:30.478597 19638 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
format_stamp: "Formatted at 2025-06-24 07:54:54 on dist-test-slave-sl8c"
I20250624 07:55:30.479923 19638 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:55:30.519520 19638 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:55:30.520668 19638 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:55:30.521028 19638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:55:30.525713 19638 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:55:30.547319 19661 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20250624 07:55:30.560644 19638 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250624 07:55:30.561646 19638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.028s user 0.003s sys 0.011s
I20250624 07:55:30.561848 19638 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250624 07:55:30.566011 19638 ts_tablet_manager.cc:610] Registered 1 tablets
I20250624 07:55:30.566161 19638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20250624 07:55:30.566599 19661 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap starting.
W20250624 07:55:30.703760 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250624 07:55:31.286819 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250624 07:55:31.634281 19770 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.67:41189 every 8 connection(s)
I20250624 07:55:31.634665 19638 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.67:41189
I20250624 07:55:31.636966 19638 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb
I20250624 07:55:31.644572 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 19638
I20250624 07:55:31.699698 19771 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:55:31.700110 19771 heartbeater.cc:461] Registering TS with master...
I20250624 07:55:31.701256 19771 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:31.710269 17304 ts_manager.cc:194] Re-registered known tserver with Master: 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189)
I20250624 07:55:31.712256 17304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.67:43469
W20250624 07:55:31.814841 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
W20250624 07:55:32.290479 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
I20250624 07:55:32.601511 19661 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Log is configured to *not* fsync() on all Append() calls
I20250624 07:55:32.727079 19771 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:55:32.835363 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20250624 07:55:33.131273 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.141839 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.206676 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.262006 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:55:33.331578 17954 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.334167 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.401576 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.406332 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:55:33.419329 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250624 07:55:33.467518 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:33.790303 19822 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 439cf2177bdb4c19aa8449b6cbd440a7 is lagging by at least 7 ops behind the committed index
W20250624 07:55:33.946022 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
W20250624 07:55:34.522395 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
W20250624 07:55:35.111406 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
W20250624 07:55:35.608088 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
W20250624 07:55:36.176486 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
W20250624 07:55:36.684504 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
I20250624 07:55:36.838315 19828 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 439cf2177bdb4c19aa8449b6cbd440a7 is lagging by at least 311 ops behind the committed index [suppressed 27 similar messages]
W20250624 07:55:37.252231 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
I20250624 07:55:37.636648 19661 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap replaying log segment 1/1 (1.64M/4.33M this segment, stats: ops{read=979 overwritten=0 applied=979 ignored=0} inserts{seen=48900 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:55:37.813925 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250624 07:55:38.474495 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250624 07:55:38.917922 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250624 07:55:38.937214 18554 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.73:38265, user_credentials={real_user=slave}} blocked reactor thread for 43857.1us
W20250624 07:55:39.481953 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250624 07:55:39.919907 19822 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 439cf2177bdb4c19aa8449b6cbd440a7 is lagging by at least 619 ops behind the committed index [suppressed 26 similar messages]
W20250624 07:55:40.082008 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
I20250624 07:55:40.446841 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.583725 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.638602 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:55:40.641153 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.641603 17954 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:55:40.643673 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
I20250624 07:55:40.670889 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.671375 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.679077 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:40.703804 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:55:41.138016 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
W20250624 07:55:41.676483 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20250624 07:55:42.274829 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
I20250624 07:55:42.639714 19661 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap replaying log segment 1/1 (3.18M/4.33M this segment, stats: ops{read=1904 overwritten=0 applied=1904 ignored=0} inserts{seen=95150 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:55:42.789043 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250624 07:55:42.978294 19843 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 439cf2177bdb4c19aa8449b6cbd440a7 is lagging by at least 890 ops behind the committed index [suppressed 27 similar messages]
W20250624 07:55:43.305109 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250624 07:55:43.873452 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
W20250624 07:55:44.392458 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 136: this message will repeat every 5th retry.
W20250624 07:55:44.827917 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 141: this message will repeat every 5th retry.
W20250624 07:55:45.355177 18431 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 146: this message will repeat every 5th retry.
I20250624 07:55:45.451843 19661 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2596 overwritten=0 applied=2595 ignored=0} inserts{seen=129700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250624 07:55:45.452756 19661 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Bootstrap complete.
I20250624 07:55:45.454825 19661 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Time spent bootstrapping tablet: real 14.889s user 10.594s sys 0.603s
I20250624 07:55:45.465210 19661 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:55:45.467677 19661 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439cf2177bdb4c19aa8449b6cbd440a7, State: Initialized, Role: FOLLOWER
I20250624 07:55:45.468375 19661 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2595, Last appended: 1.2596, Last appended by leader: 2596, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:55:45.469838 19661 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20250624 07:55:45.849467 19900 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cbd668c726a4b1298e789270a0031e2)
I20250624 07:55:45.849746 19900 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:55:45.851814 19900 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:55:45.868736 17974 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3021 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:55:45.926326 18497 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3021 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:55:45.927090 19657 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7; no voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:55:46.299391 19900 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250624 07:55:46.428051 19892 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 439cf2177bdb4c19aa8449b6cbd440a7 is lagging by at least 388 ops behind the committed index [suppressed 26 similar messages]
I20250624 07:55:47.052866 19916 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cbd668c726a4b1298e789270a0031e2)
I20250624 07:55:47.053131 19916 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:55:47.061025 17974 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4278 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:55:47.062664 18497 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4278 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:55:47.065354 19916 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:55:47.069283 19657 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7; no voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:55:47.070079 19916 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250624 07:55:48.149385 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.243829 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.284093 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:55:48.288842 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.295487 17954 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.373164 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.376597 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.409255 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:55:48.425477 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:55:49.647848 18554 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.73:38265, user_credentials={real_user=slave}} blocked reactor thread for 225147us
I20250624 07:55:50.702098 17304 ts_manager.cc:284] Unset tserver state for 439cf2177bdb4c19aa8449b6cbd440a7 from MAINTENANCE_MODE
I20250624 07:55:50.704365 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:50.894829 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.052973 19771 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.062016 18993 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.199519 17303 ts_manager.cc:284] Unset tserver state for e1bf694ea1724daaa49f0f0746a03eeb from MAINTENANCE_MODE
I20250624 07:55:51.255635 19486 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.311848 19626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.543895 17303 ts_manager.cc:284] Unset tserver state for 1c0cb4e9f1d94dfbbfbaf9138aa4e655 from MAINTENANCE_MODE
I20250624 07:55:51.569406 18849 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.577371 18019 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.622018 19129 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.708141 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:51.903108 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:52.066587 18993 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:52.069597 19771 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:52.261122 19486 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:52.320096 19626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:55:54.652083 17269 maintenance_mode-itest.cc:745] Restarting batch of 1 tservers: 4f219d2e8b954461973a8ca8680a6383
I20250624 07:55:56.082969 17304 ts_manager.cc:295] Set tserver state for 4f219d2e8b954461973a8ca8680a6383 to MAINTENANCE_MODE
I20250624 07:55:58.606194 17954 tablet_service.cc:1423] Tablet server 4f219d2e8b954461973a8ca8680a6383 set to quiescing
I20250624 07:55:58.606614 17954 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:00.014657 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17891
I20250624 07:56:00.050127 18014 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:56:00.487063 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17891
W20250624 07:56:00.581493 18434 connection.cc:537] client connection to 127.16.221.69:35369 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250624 07:56:00.581655 18434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 18 similar messages]
I20250624 07:56:00.582444 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.69:35369
--local_ip_for_outbound_sockets=127.16.221.69
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=37317
--webserver_interface=127.16.221.69
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:56:00.590724 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250624 07:56:01.068049 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250624 07:56:01.094535 20075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:56:01.095000 20075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:56:01.095181 20075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:56:01.109026 20075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:56:01.109411 20075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.69
I20250624 07:56:01.123553 20075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.69:35369
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
--webserver_interface=127.16.221.69
--webserver_port=37317
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.69
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:56:01.126019 20075 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:56:01.128713 20075 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:56:01.153167 20083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:56:01.157785 20082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:56:01.162192 20075 server_base.cc:1048] running on GCE node
W20250624 07:56:01.164827 20085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:56:01.165602 20075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:56:01.168128 20075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:56:01.169397 20075 hybrid_clock.cc:648] HybridClock initialized: now 1750751761169293 us; error 95 us; skew 500 ppm
I20250624 07:56:01.169853 20075 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:56:01.173508 20075 webserver.cc:469] Webserver started at http://127.16.221.69:37317/ using document root <none> and password file <none>
I20250624 07:56:01.174593 20075 fs_manager.cc:362] Metadata directory not provided
I20250624 07:56:01.174755 20075 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:56:01.187582 20075 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.005s sys 0.001s
I20250624 07:56:01.192958 20091 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:56:01.195200 20075 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.001s sys 0.001s
I20250624 07:56:01.198210 20075 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
uuid: "4f219d2e8b954461973a8ca8680a6383"
format_stamp: "Formatted at 2025-06-24 07:54:55 on dist-test-slave-sl8c"
I20250624 07:56:01.199297 20075 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:56:01.259109 20075 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:56:01.260419 20075 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:56:01.260874 20075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:56:01.284394 20075 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:56:01.286809 20098 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20250624 07:56:01.299944 20075 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250624 07:56:01.300446 20075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.000s sys 0.001s
I20250624 07:56:01.300637 20075 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250624 07:56:01.306527 20075 ts_tablet_manager.cc:610] Registered 1 tablets
I20250624 07:56:01.306682 20075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.006s user 0.003s sys 0.002s
I20250624 07:56:01.310894 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap starting.
W20250624 07:56:01.560685 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250624 07:56:01.732769 20098 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Log is configured to *not* fsync() on all Append() calls
W20250624 07:56:02.108923 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250624 07:56:02.176313 20075 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.69:35369
I20250624 07:56:02.176399 20209 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.69:35369 every 8 connection(s)
I20250624 07:56:02.178334 20075 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb
I20250624 07:56:02.186578 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 20075
I20250624 07:56:02.220206 20210 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:56:02.220633 20210 heartbeater.cc:461] Registering TS with master...
I20250624 07:56:02.221767 20210 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:02.224913 17303 ts_manager.cc:194] Re-registered known tserver with Master: 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:56:02.226656 17303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.69:45215
W20250624 07:56:02.612113 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
W20250624 07:56:03.158360 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
I20250624 07:56:03.232314 20210 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
I20250624 07:56:03.518568 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.541567 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.636802 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.667574 20142 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.721794 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:03.726491 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.800806 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:03.803442 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:03.836601 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20250624 07:56:03.859694 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:04.374244 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250624 07:56:04.458923 20138 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 9 ops behind the committed index
W20250624 07:56:04.928541 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
W20250624 07:56:05.454665 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
W20250624 07:56:05.975195 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
W20250624 07:56:06.459816 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
I20250624 07:56:06.756724 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replaying log segment 1/2 (1.57M/8.00M this segment, stats: ops{read=922 overwritten=0 applied=922 ignored=0} inserts{seen=46050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:56:06.984158 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
I20250624 07:56:07.463231 20225 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 301 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:56:07.499018 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
W20250624 07:56:08.046963 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
W20250624 07:56:08.588856 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250624 07:56:09.094739 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250624 07:56:09.629354 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250624 07:56:10.151405 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250624 07:56:10.535719 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.545156 20297 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 618 ops behind the committed index [suppressed 28 similar messages]
I20250624 07:56:10.586290 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.663203 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.696794 20142 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:10.703868 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
I20250624 07:56:10.744359 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:10.775524 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.777474 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.843243 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:10.843714 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:11.277175 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
I20250624 07:56:11.758555 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replaying log segment 1/2 (3.20M/8.00M this segment, stats: ops{read=1880 overwritten=0 applied=1880 ignored=0} inserts{seen=93950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:56:11.895030 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
W20250624 07:56:12.478628 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20250624 07:56:13.034606 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
W20250624 07:56:13.539381 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250624 07:56:13.633841 20307 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 913 ops behind the committed index [suppressed 26 similar messages]
W20250624 07:56:14.138576 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250624 07:56:14.586683 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
W20250624 07:56:15.135094 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 136: this message will repeat every 5th retry.
W20250624 07:56:15.632303 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 141: this message will repeat every 5th retry.
W20250624 07:56:16.163648 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 146: this message will repeat every 5th retry.
W20250624 07:56:16.667095 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 151: this message will repeat every 5th retry.
I20250624 07:56:16.756603 20265 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 1232 ops behind the committed index [suppressed 29 similar messages]
I20250624 07:56:16.758671 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replaying log segment 1/2 (5.36M/8.00M this segment, stats: ops{read=3150 overwritten=0 applied=3148 ignored=0} inserts{seen=157350 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:56:17.214969 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 156: this message will repeat every 5th retry.
W20250624 07:56:17.712374 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 161: this message will repeat every 5th retry.
I20250624 07:56:17.755467 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.801928 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.810949 20142 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.818197 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.850945 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:17.935961 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.955390 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:17.999747 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:18.015563 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:18.209517 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 166: this message will repeat every 5th retry.
W20250624 07:56:18.727416 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 171: this message will repeat every 5th retry.
W20250624 07:56:19.270641 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 176: this message will repeat every 5th retry.
I20250624 07:56:19.777098 20307 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 1518 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:56:19.801514 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 181: this message will repeat every 5th retry.
W20250624 07:56:20.338922 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 186: this message will repeat every 5th retry.
W20250624 07:56:20.914269 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 191: this message will repeat every 5th retry.
W20250624 07:56:21.477352 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 196: this message will repeat every 5th retry.
I20250624 07:56:21.759532 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replaying log segment 1/2 (7.15M/8.00M this segment, stats: ops{read=4201 overwritten=0 applied=4200 ignored=0} inserts{seen=209950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:56:21.990818 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 201: this message will repeat every 5th retry.
W20250624 07:56:22.495102 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 206: this message will repeat every 5th retry.
I20250624 07:56:22.864809 20330 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 1845 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:56:22.998618 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 211: this message will repeat every 5th retry.
W20250624 07:56:23.604751 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 216: this message will repeat every 5th retry.
I20250624 07:56:23.977205 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replayed 1/2 log segments. Stats: ops{read=4699 overwritten=0 applied=4698 ignored=0} inserts{seen=234850 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20250624 07:56:24.130692 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 221: this message will repeat every 5th retry.
W20250624 07:56:24.669521 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 226: this message will repeat every 5th retry.
I20250624 07:56:25.070664 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.100842 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.173832 20142 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.196748 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.234527 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.274099 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:25.296442 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:25.379715 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:25.386152 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 231: this message will repeat every 5th retry.
I20250624 07:56:25.441602 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:56:25.815974 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 236: this message will repeat every 5th retry.
I20250624 07:56:25.873351 20330 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Peer 4f219d2e8b954461973a8ca8680a6383 is lagging by at least 2117 ops behind the committed index [suppressed 26 similar messages]
W20250624 07:56:26.359750 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 241: this message will repeat every 5th retry.
I20250624 07:56:26.762048 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replaying log segment 2/2 (884.1K/968.1K this segment, stats: ops{read=5203 overwritten=0 applied=5202 ignored=0} inserts{seen=260050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:56:26.828636 18434 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 246: this message will repeat every 5th retry.
I20250624 07:56:26.936908 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap replayed 2/2 log segments. Stats: ops{read=5251 overwritten=0 applied=5250 ignored=0} inserts{seen=262450 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250624 07:56:26.937867 20098 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Bootstrap complete.
I20250624 07:56:26.941953 20098 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Time spent bootstrapping tablet: real 25.631s user 19.212s sys 0.748s
I20250624 07:56:26.947175 20098 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:26.977382 20098 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f219d2e8b954461973a8ca8680a6383, State: Initialized, Role: FOLLOWER
I20250624 07:56:26.978304 20098 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5250, Last appended: 1.5251, Last appended by leader: 5251, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:26.990669 20098 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383: Time spent starting tablet: real 0.049s user 0.028s sys 0.020s
I20250624 07:56:27.763129 20445 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6cbd668c726a4b1298e789270a0031e2)
I20250624 07:56:27.763463 20445 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:27.765825 20445 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:56:27.797627 18497 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 2 candidate_status { last_received { term: 1 index: 6101 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:56:27.841518 19724 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 2 candidate_status { last_received { term: 1 index: 6101 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:56:27.842555 20092 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f219d2e8b954461973a8ca8680a6383; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:56:28.069262 20445 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250624 07:56:32.777058 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.787744 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:32.812852 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.845989 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.848788 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.867671 20142 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.898128 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.915683 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:32.946918 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:34.798171 17304 ts_manager.cc:284] Unset tserver state for 4f219d2e8b954461973a8ca8680a6383 from MAINTENANCE_MODE
I20250624 07:56:34.824740 18412 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.226337 18993 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.226760 19771 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.352432 18543 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.356357 20210 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.381106 19486 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.424109 19626 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.684880 18849 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:35.733673 19129 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:56:36.637573 17269 maintenance_mode-itest.cc:745] Restarting batch of 2 tservers: 8150d3eeb1c84234a13484098d17ce8f,6cbd668c726a4b1298e789270a0031e2
I20250624 07:56:37.461892 17303 ts_manager.cc:295] Set tserver state for 8150d3eeb1c84234a13484098d17ce8f to MAINTENANCE_MODE
W20250624 07:56:38.427851 18431 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.221.67:41189, user_credentials={real_user=slave}} blocked reactor thread for 57665.5us
I20250624 07:56:38.864909 17300 ts_manager.cc:295] Set tserver state for 6cbd668c726a4b1298e789270a0031e2 to MAINTENANCE_MODE
W20250624 07:56:40.114274 18554 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.73:38265, user_credentials={real_user=slave}} blocked reactor thread for 137258us
I20250624 07:56:41.494486 18347 tablet_service.cc:1423] Tablet server 8150d3eeb1c84234a13484098d17ce8f set to quiescing
I20250624 07:56:41.494768 18347 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:56:41.864218 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:41.864485 18478 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250624 07:56:41.883952 20593 raft_consensus.cc:991] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: : Instructing follower 4f219d2e8b954461973a8ca8680a6383 to start an election
I20250624 07:56:41.884115 20593 raft_consensus.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 LEADER]: Signalling peer 4f219d2e8b954461973a8ca8680a6383 to start an election
I20250624 07:56:41.885480 20162 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa"
dest_uuid: "4f219d2e8b954461973a8ca8680a6383"
from {username='slave'} at 127.16.221.73:37211
I20250624 07:56:41.885782 20162 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250624 07:56:41.885932 20162 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 1 FOLLOWER]: Advancing to term 2
I20250624 07:56:41.891551 20162 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:41.892792 18497 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 2 candidate_status { last_received { term: 1 index: 9119 } } ignore_live_leader: true dest_uuid: "6cbd668c726a4b1298e789270a0031e2"
I20250624 07:56:41.892751 19724 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "4f219d2e8b954461973a8ca8680a6383" candidate_term: 2 candidate_status { last_received { term: 1 index: 9119 } } ignore_live_leader: true dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
I20250624 07:56:41.892999 18497 raft_consensus.cc:3053] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 LEADER]: Stepping down as leader of term 1
I20250624 07:56:41.893060 18497 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Running, Role: LEADER
I20250624 07:56:41.893056 19724 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 1 FOLLOWER]: Advancing to term 2
I20250624 07:56:41.893329 18497 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9119, Committed index: 9119, Last appended: 1.9119, Last appended by leader: 9119, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:41.893577 18497 logging.cc:424] LogThrottler TrackedPeer Lag: suppressed but not reported on 16 messages since previous log ~16 seconds ago
I20250624 07:56:41.893671 18497 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 1 FOLLOWER]: Advancing to term 2
I20250624 07:56:41.896847 18497 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f219d2e8b954461973a8ca8680a6383 in term 2.
I20250624 07:56:41.897576 20094 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:56:41.905594 20162 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [CANDIDATE]: Term 2 election: Requested vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
W20250624 07:56:41.909577 18596 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:39978: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.026s ago.
I20250624 07:56:41.917272 19724 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f219d2e8b954461973a8ca8680a6383 in term 2.
I20250624 07:56:41.918694 20626 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 FOLLOWER]: Leader election won for term 2
W20250624 07:56:41.955106 18452 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:39978: Illegal state: replica 6cbd668c726a4b1298e789270a0031e2 is not leader of this config: current role FOLLOWER
I20250624 07:56:41.993393 20626 raft_consensus.cc:695] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 LEADER]: Becoming Leader. State: Replica: 4f219d2e8b954461973a8ca8680a6383, State: Running, Role: LEADER
I20250624 07:56:41.994385 20626 consensus_queue.cc:237] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9119, Committed index: 9119, Last appended: 1.9119, Last appended by leader: 9119, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:56:41.998983 17300 catalog_manager.cc:5582] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 reported cstate change: term changed from 1 to 2, leader changed from 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73) to 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69). New cstate: current_term: 2 leader_uuid: "4f219d2e8b954461973a8ca8680a6383" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } health_report { overall_health: UNKNOWN } } }
W20250624 07:56:42.008821 19683 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:48668: Illegal state: replica 439cf2177bdb4c19aa8449b6cbd440a7 is not leader of this config: current role FOLLOWER
W20250624 07:56:42.010679 19683 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:48668: Illegal state: replica 439cf2177bdb4c19aa8449b6cbd440a7 is not leader of this config: current role FOLLOWER
W20250624 07:56:42.011911 19683 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:48668: Illegal state: replica 439cf2177bdb4c19aa8449b6cbd440a7 is not leader of this config: current role FOLLOWER
I20250624 07:56:42.090675 18497 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Refusing update from remote peer 4f219d2e8b954461973a8ca8680a6383: Log matching property violated. Preceding OpId in replica: term: 1 index: 9119. Preceding OpId from leader: term: 2 index: 9120. (index mismatch)
I20250624 07:56:42.091430 20626 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9120, Last known committed idx: 9119, Time since last communication: 0.000s
I20250624 07:56:42.097683 19724 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Refusing update from remote peer 4f219d2e8b954461973a8ca8680a6383: Log matching property violated. Preceding OpId in replica: term: 1 index: 9118. Preceding OpId from leader: term: 2 index: 9120. (index mismatch)
I20250624 07:56:42.099020 20627 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Connected to new peer: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9120, Last known committed idx: 9117, Time since last communication: 0.000s
I20250624 07:56:43.297257 20513 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46736 (ReqId={client: d9d79f4177644471b1eee5e36c5e8d49, seq_no=9118, attempt_no=2}) took 1257 ms. Trace:
I20250624 07:56:43.298245 20513 rpcz_store.cc:276] 0624 07:56:42.039890 (+ 0us) service_pool.cc:168] Inserting onto call queue
0624 07:56:42.040530 (+ 640us) service_pool.cc:225] Handling call
0624 07:56:43.297205 (+1256675us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0624 07:56:43.268848 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 5ca5ea09d51f4eb390c4433fec8ed9fa
0624 07:56:43.268932 (+ 84us) write_op.cc:432] Acquiring schema lock in shared mode
0624 07:56:43.268937 (+ 5us) write_op.cc:435] Acquired schema lock
0624 07:56:43.268939 (+ 2us) tablet.cc:660] Decoding operations
0624 07:56:43.269329 (+ 390us) write_op.cc:620] Acquiring the partition lock for write op
0624 07:56:43.269337 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0624 07:56:43.269342 (+ 5us) tablet.cc:683] Acquiring locks for 50 operations
0624 07:56:43.269649 (+ 307us) tablet.cc:699] Row locks acquired
0624 07:56:43.269651 (+ 2us) write_op.cc:260] PREPARE: finished
0624 07:56:43.269744 (+ 93us) write_op.cc:270] Start()
0624 07:56:43.269797 (+ 53us) write_op.cc:276] Timestamp: P: 1750751803269590 usec, L: 0
0624 07:56:43.269805 (+ 8us) op_driver.cc:348] REPLICATION: starting
0624 07:56:43.270075 (+ 270us) log.cc:844] Serialized 2467 byte log entry
0624 07:56:43.271938 (+ 1863us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x616000139fb8. stack: 00007ff5b1510336 00007ff5b15100f0 00007ff5b05063ad 0000000000971004 0000000000970fcd 000000000097099a 00007ff5bce95c19 00007ff5bce6ed89 00007ff5bce77b0f 00007ff5bce777fd 0000000000a7573c 00007ff5b15559df 00007ff5b1558cab 00007ff5b1558b28 0000000000a7573c 00007ff5b15354fd
0624 07:56:43.293835 (+ 21897us) op_driver.cc:464] REPLICATION: finished
0624 07:56:43.293977 (+ 142us) write_op.cc:301] APPLY: starting
0624 07:56:43.294001 (+ 24us) tablet.cc:1365] starting BulkCheckPresence
0624 07:56:43.294617 (+ 616us) tablet.cc:1368] finished BulkCheckPresence
0624 07:56:43.294620 (+ 3us) tablet.cc:1370] starting ApplyRowOperation cycle
0624 07:56:43.296316 (+ 1696us) tablet.cc:1381] finished ApplyRowOperation cycle
0624 07:56:43.296383 (+ 67us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0624 07:56:43.296393 (+ 10us) write_op.cc:312] APPLY: finished
0624 07:56:43.296575 (+ 182us) log.cc:844] Serialized 320 byte log entry
0624 07:56:43.296707 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0624 07:56:43.296763 (+ 56us) write_op.cc:454] Released schema lock
0624 07:56:43.296775 (+ 12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":14208,"child_traces":[["op",{"apply.queue_time_us":70,"num_ops":50,"prepare.queue_time_us":1227702,"prepare.run_cpu_time_us":1639,"prepare.run_wall_time_us":5533,"raft.queue_time_us":2461,"raft.run_cpu_time_us":993,"raft.run_wall_time_us":2014,"replication_time_us":23987,"spinlock_wait_cycles":2574720,"thread_start_us":4182,"threads_started":1}]]}
I20250624 07:56:43.309536 20596 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46736 (ReqId={client: d9d79f4177644471b1eee5e36c5e8d49, seq_no=9120, attempt_no=2}) took 1269 ms. Trace:
I20250624 07:56:43.309718 20596 rpcz_store.cc:276] 0624 07:56:42.039617 (+ 0us) service_pool.cc:168] Inserting onto call queue
0624 07:56:42.039940 (+ 323us) service_pool.cc:225] Handling call
0624 07:56:42.041354 (+ 1414us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x611000089858. stack: 00007ff5b1510336 00007ff5b15100f0 00007ff5b05063ad 0000000000971004 0000000000970fcd 000000000097099a 00007ff5b8247bc3 00007ff5b82b2cdc 00007ff5b82b8d95 00007ff5b82b96bf 00007ff5b82b94e8 0000000000a7573c 00007ff5b15354fd 00007ff5b49536da 00007ff5ad68c71e
0624 07:56:43.309508 (+1268154us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0624 07:56:43.275827 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 5ca5ea09d51f4eb390c4433fec8ed9fa
0624 07:56:43.275929 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0624 07:56:43.275934 (+ 5us) write_op.cc:435] Acquired schema lock
0624 07:56:43.275936 (+ 2us) tablet.cc:660] Decoding operations
0624 07:56:43.277911 (+ 1975us) write_op.cc:620] Acquiring the partition lock for write op
0624 07:56:43.277921 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0624 07:56:43.277926 (+ 5us) tablet.cc:683] Acquiring locks for 50 operations
0624 07:56:43.278237 (+ 311us) tablet.cc:699] Row locks acquired
0624 07:56:43.278239 (+ 2us) write_op.cc:260] PREPARE: finished
0624 07:56:43.278277 (+ 38us) write_op.cc:270] Start()
0624 07:56:43.278302 (+ 25us) write_op.cc:276] Timestamp: P: 1750751803278127 usec, L: 0
0624 07:56:43.278306 (+ 4us) op_driver.cc:348] REPLICATION: starting
0624 07:56:43.278573 (+ 267us) log.cc:844] Serialized 2467 byte log entry
0624 07:56:43.307613 (+ 29040us) op_driver.cc:464] REPLICATION: finished
0624 07:56:43.307704 (+ 91us) write_op.cc:301] APPLY: starting
0624 07:56:43.307722 (+ 18us) tablet.cc:1365] starting BulkCheckPresence
0624 07:56:43.307842 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0624 07:56:43.307885 (+ 43us) tablet.cc:1370] starting ApplyRowOperation cycle
0624 07:56:43.309042 (+ 1157us) tablet.cc:1381] finished ApplyRowOperation cycle
0624 07:56:43.309100 (+ 58us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0624 07:56:43.309108 (+ 8us) write_op.cc:312] APPLY: finished
0624 07:56:43.309271 (+ 163us) log.cc:844] Serialized 320 byte log entry
0624 07:56:43.309365 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0624 07:56:43.309417 (+ 52us) write_op.cc:454] Released schema lock
0624 07:56:43.309426 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":2581760,"child_traces":[["op",{"apply.queue_time_us":34,"num_ops":50,"prepare.queue_time_us":1234116,"prepare.run_cpu_time_us":1251,"prepare.run_wall_time_us":2810,"replication_time_us":29286,"spinlock_wait_cycles":142464}]]}
I20250624 07:56:43.311519 20513 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46736 (ReqId={client: d9d79f4177644471b1eee5e36c5e8d49, seq_no=9119, attempt_no=1}) took 1271 ms. Trace:
I20250624 07:56:43.311709 20513 rpcz_store.cc:276] 0624 07:56:42.039745 (+ 0us) service_pool.cc:168] Inserting onto call queue
0624 07:56:42.039800 (+ 55us) service_pool.cc:225] Handling call
0624 07:56:43.311503 (+1271703us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0624 07:56:43.274419 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 5ca5ea09d51f4eb390c4433fec8ed9fa
0624 07:56:43.274516 (+ 97us) write_op.cc:432] Acquiring schema lock in shared mode
0624 07:56:43.274521 (+ 5us) write_op.cc:435] Acquired schema lock
0624 07:56:43.274524 (+ 3us) tablet.cc:660] Decoding operations
0624 07:56:43.274904 (+ 380us) write_op.cc:620] Acquiring the partition lock for write op
0624 07:56:43.274912 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0624 07:56:43.274917 (+ 5us) tablet.cc:683] Acquiring locks for 50 operations
0624 07:56:43.275271 (+ 354us) tablet.cc:699] Row locks acquired
0624 07:56:43.275273 (+ 2us) write_op.cc:260] PREPARE: finished
0624 07:56:43.275314 (+ 41us) write_op.cc:270] Start()
0624 07:56:43.275342 (+ 28us) write_op.cc:276] Timestamp: P: 1750751803275164 usec, L: 0
0624 07:56:43.275346 (+ 4us) op_driver.cc:348] REPLICATION: starting
0624 07:56:43.275717 (+ 371us) log.cc:844] Serialized 2467 byte log entry
0624 07:56:43.307511 (+ 31794us) op_driver.cc:464] REPLICATION: finished
0624 07:56:43.307643 (+ 132us) write_op.cc:301] APPLY: starting
0624 07:56:43.307837 (+ 194us) tablet.cc:1365] starting BulkCheckPresence
0624 07:56:43.307979 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0624 07:56:43.308054 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0624 07:56:43.310998 (+ 2944us) tablet.cc:1381] finished ApplyRowOperation cycle
0624 07:56:43.311051 (+ 53us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0624 07:56:43.311060 (+ 9us) write_op.cc:312] APPLY: finished
0624 07:56:43.311294 (+ 234us) log.cc:844] Serialized 320 byte log entry
0624 07:56:43.311377 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0624 07:56:43.311418 (+ 41us) write_op.cc:454] Released schema lock
0624 07:56:43.311428 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63,"mutex_wait_us":257,"num_ops":50,"prepare.queue_time_us":1233276,"prepare.run_cpu_time_us":1178,"prepare.run_wall_time_us":1363,"replication_time_us":32124}]]}
I20250624 07:56:44.111559 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:44.111819 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:56:46.391691 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:46.391949 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:56:50.251324 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:50.251555 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:56:52.709460 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:52.709738 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:56:53.245944 17297 maintenance_manager.cc:419] P b182c77196c04b549d02ed8a4be16054: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033402
I20250624 07:56:53.358340 17296 maintenance_manager.cc:643] P b182c77196c04b549d02ed8a4be16054: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.112s user 0.028s sys 0.004s Metrics: {"bytes_written":7340,"cfile_init":1,"drs_written":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1140,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":14,"thread_start_us":9163,"threads_started":1,"wal-append.queue_time_us":8893}
I20250624 07:56:53.360149 17297 maintenance_manager.cc:419] P b182c77196c04b549d02ed8a4be16054: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 672 bytes on disk
I20250624 07:56:53.361204 17296 maintenance_manager.cc:643] P b182c77196c04b549d02ed8a4be16054: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20250624 07:56:56.246232 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:56.246590 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 2 scanners
I20250624 07:56:57.221091 18544 maintenance_manager.cc:419] P 6cbd668c726a4b1298e789270a0031e2: Scheduling FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa): perf score=0.053253
I20250624 07:56:59.127175 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:56:59.127447 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 2 scanners
W20250624 07:57:02.446323 18555 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.69:35369, user_credentials={real_user=slave}} blocked reactor thread for 52650.9us
I20250624 07:57:03.689997 18478 tablet_service.cc:1423] Tablet server 6cbd668c726a4b1298e789270a0031e2 set to quiescing
I20250624 07:57:03.690250 18478 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:05.385871 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18284
I20250624 07:57:05.472677 18406 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:57:05.898599 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18284
I20250624 07:57:05.949643 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.72:40741
--local_ip_for_outbound_sockets=127.16.221.72
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=34785
--webserver_interface=127.16.221.72
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:57:06.856506 20806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:57:06.857025 20806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:57:06.857223 20806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:57:06.869331 20806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:57:06.869617 20806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.72
I20250624 07:57:06.884291 20806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.72:40741
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
--webserver_interface=127.16.221.72
--webserver_port=34785
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.72
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:57:06.886678 20806 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:57:06.889263 20806 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:57:07.010713 20813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:57:07.022795 20806 server_base.cc:1048] running on GCE node
W20250624 07:57:07.026664 20815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:57:07.029291 20812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:57:07.031164 20806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:57:07.067828 20806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:57:07.076656 20806 hybrid_clock.cc:648] HybridClock initialized: now 1750751827076549 us; error 95 us; skew 500 ppm
I20250624 07:57:07.077662 20806 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:57:07.115590 20806 webserver.cc:469] Webserver started at http://127.16.221.72:34785/ using document root <none> and password file <none>
I20250624 07:57:07.117017 20806 fs_manager.cc:362] Metadata directory not provided
I20250624 07:57:07.117255 20806 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:57:07.200380 20806 fs_manager.cc:714] Time spent opening directory manager: real 0.080s user 0.005s sys 0.000s
I20250624 07:57:07.239351 20824 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:57:07.250330 20806 fs_manager.cc:730] Time spent opening block manager: real 0.048s user 0.000s sys 0.002s
I20250624 07:57:07.250823 20806 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
uuid: "8150d3eeb1c84234a13484098d17ce8f"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:57:07.252048 20806 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:57:07.525810 20806 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:57:07.531946 20806 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:57:07.540673 20806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:57:07.559005 20806 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:57:07.569844 20806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250624 07:57:07.570024 20806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20250624 07:57:07.570206 20806 ts_tablet_manager.cc:610] Registered 0 tablets
I20250624 07:57:07.570298 20806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20250624 07:57:08.746347 18583 meta_cache.cc:1261] Time spent looking up entry by key: real 0.066s user 0.000s sys 0.000s
I20250624 07:57:09.026932 20938 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.72:40741 every 8 connection(s)
I20250624 07:57:09.027271 20806 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.72:40741
I20250624 07:57:09.030591 20806 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb
I20250624 07:57:09.033039 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 20806
I20250624 07:57:09.033325 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18415
I20250624 07:57:09.094651 18538 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:57:09.107537 20939 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:57:09.107985 20939 heartbeater.cc:461] Registering TS with master...
I20250624 07:57:09.109172 20939 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:57:09.112003 17300 ts_manager.cc:194] Re-registered known tserver with Master: 8150d3eeb1c84234a13484098d17ce8f (127.16.221.72:40741)
I20250624 07:57:09.113299 17300 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.72:51667
I20250624 07:57:09.764675 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18415
W20250624 07:57:09.856869 17292 connection.cc:537] server connection from 127.16.221.73:37481 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250624 07:57:09.856921 20094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.221.73:38265 (error 108)
I20250624 07:57:09.858296 17269 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
/tmp/dist-test-taskaI4uEx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.221.73:38265
--local_ip_for_outbound_sockets=127.16.221.73
--tserver_master_addrs=127.16.221.126:44487
--webserver_port=35519
--webserver_interface=127.16.221.73
--builtin_ntp_servers=127.16.221.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250624 07:57:09.867305 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250624 07:57:09.991641 18555 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.69:35369, user_credentials={real_user=slave}} blocked reactor thread for 87045.7us
I20250624 07:57:10.116508 20939 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:57:10.294816 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250624 07:57:10.504434 20953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:57:10.504936 20953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250624 07:57:10.505131 20953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:57:10.518311 20953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250624 07:57:10.518702 20953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.221.73
I20250624 07:57:10.533255 20953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.221.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.221.73:38265
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
--webserver_interface=127.16.221.73
--webserver_port=35519
--tserver_master_addrs=127.16.221.126:44487
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.221.73
--log_dir=/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 6d72d4a242076468501f3430b9a2cd050c634be2
build type FASTDEBUG
built by None at 24 Jun 2025 07:43:20 UTC on 24a791456cd2
build id 6744
ASAN enabled
I20250624 07:57:10.535719 20953 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250624 07:57:10.538547 20953 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250624 07:57:10.613889 20963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:57:10.614866 20962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250624 07:57:10.621734 20965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250624 07:57:10.629446 20953 server_base.cc:1048] running on GCE node
I20250624 07:57:10.630412 20953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250624 07:57:10.631982 20953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250624 07:57:10.635519 20953 hybrid_clock.cc:648] HybridClock initialized: now 1750751830635259 us; error 216 us; skew 500 ppm
I20250624 07:57:10.635936 20953 server_base.cc:848] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250624 07:57:10.692013 20953 webserver.cc:469] Webserver started at http://127.16.221.73:35519/ using document root <none> and password file <none>
I20250624 07:57:10.693310 20953 fs_manager.cc:362] Metadata directory not provided
I20250624 07:57:10.693562 20953 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250624 07:57:10.704927 20953 fs_manager.cc:714] Time spent opening directory manager: real 0.008s user 0.000s sys 0.003s
I20250624 07:57:10.733240 20973 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250624 07:57:10.734915 20953 fs_manager.cc:730] Time spent opening block manager: real 0.017s user 0.005s sys 0.000s
I20250624 07:57:10.735176 20953 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data,/tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
uuid: "6cbd668c726a4b1298e789270a0031e2"
format_stamp: "Formatted at 2025-06-24 07:54:56 on dist-test-slave-sl8c"
I20250624 07:57:10.736227 20953 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
metadata directory: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/wal
1 data directories: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 6 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250624 07:57:10.828236 20953 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250624 07:57:10.829428 20953 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250624 07:57:10.829806 20953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250624 07:57:10.831017 20953 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250624 07:57:10.833942 20980 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20250624 07:57:10.841060 20953 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250624 07:57:10.841195 20953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20250624 07:57:10.841370 20953 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
W20250624 07:57:10.844225 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250624 07:57:10.846009 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap starting.
I20250624 07:57:10.849335 20953 ts_tablet_manager.cc:610] Registered 1 tablets
I20250624 07:57:10.849478 20953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.008s user 0.003s sys 0.001s
W20250624 07:57:11.391472 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20250624 07:57:11.915465 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20250624 07:57:12.288991 20980 log.cc:826] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Log is configured to *not* fsync() on all Append() calls
W20250624 07:57:12.341750 20092 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.16.221.67:41189, user_credentials={real_user=slave}} blocked reactor thread for 49998.3us
W20250624 07:57:12.570487 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.73:38265: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20250624 07:57:12.778349 20953 rpc_server.cc:307] RPC server started. Bound to: 127.16.221.73:38265
I20250624 07:57:12.778398 21092 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.16.221.73:38265 every 8 connection(s)
I20250624 07:57:12.780345 20953 server_base.cc:1180] Dumped server information to /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb
I20250624 07:57:12.788367 17269 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu as pid 20953
I20250624 07:57:12.806581 21094 heartbeater.cc:344] Connected to a master server at 127.16.221.126:44487
I20250624 07:57:12.807016 21094 heartbeater.cc:461] Registering TS with master...
I20250624 07:57:12.808147 21094 heartbeater.cc:507] Master 127.16.221.126:44487 requested a full tablet report, sending...
I20250624 07:57:12.811502 17300 ts_manager.cc:194] Re-registered known tserver with Master: 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:57:12.812891 17300 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.221.73:55985
W20250624 07:57:13.196870 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
W20250624 07:57:13.737522 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250624 07:57:13.783970 21100 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3 ops behind the committed index
I20250624 07:57:13.830348 21094 heartbeater.cc:499] Master 127.16.221.126:44487 was elected leader, sending a full tablet report...
W20250624 07:57:14.245223 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
I20250624 07:57:14.562834 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:14.581486 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:14.613868 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:14.671705 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:14.760201 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:14.796257 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
I20250624 07:57:14.806548 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:14.811553 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:14.831017 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:14.861722 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:15.403992 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
W20250624 07:57:15.974305 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
W20250624 07:57:16.434271 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
I20250624 07:57:16.788440 21055 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 322 ops behind the committed index [suppressed 27 similar messages]
W20250624 07:57:17.041981 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
I20250624 07:57:17.332319 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 1/3 (1.28M/8.00M this segment, stats: ops{read=738 overwritten=0 applied=737 ignored=0} inserts{seen=36800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:17.572840 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
W20250624 07:57:18.121419 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250624 07:57:18.622447 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250624 07:57:19.155957 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250624 07:57:19.654677 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250624 07:57:19.873926 20783 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 677 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:57:20.088678 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
W20250624 07:57:20.651325 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
W20250624 07:57:21.147823 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
W20250624 07:57:21.467799 18555 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.69:35369, user_credentials={real_user=slave}} blocked reactor thread for 113605us
W20250624 07:57:21.768891 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
I20250624 07:57:21.882241 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:21.926501 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:21.982360 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:21.983093 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:22.074227 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:22.102751 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:22.134279 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:22.151381 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:22.179451 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:22.302631 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
I20250624 07:57:22.340507 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 1/3 (3.00M/8.00M this segment, stats: ops{read=1733 overwritten=0 applied=1733 ignored=0} inserts{seen=86600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:22.758671 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250624 07:57:22.951268 21150 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 956 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:57:23.317153 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250624 07:57:23.828409 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
W20250624 07:57:24.395419 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 136: this message will repeat every 5th retry.
W20250624 07:57:24.441659 18555 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.69:35369, user_credentials={real_user=slave}} blocked reactor thread for 44345.4us
W20250624 07:57:24.859166 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 141: this message will repeat every 5th retry.
W20250624 07:57:25.350497 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 146: this message will repeat every 5th retry.
W20250624 07:57:25.947808 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 151: this message will repeat every 5th retry.
I20250624 07:57:26.013521 21212 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 1298 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:57:26.432814 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 156: this message will repeat every 5th retry.
W20250624 07:57:26.944476 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 161: this message will repeat every 5th retry.
I20250624 07:57:27.343711 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 1/3 (4.97M/8.00M this segment, stats: ops{read=2870 overwritten=0 applied=2870 ignored=0} inserts{seen=143450 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:27.458658 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 166: this message will repeat every 5th retry.
W20250624 07:57:28.017202 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 171: this message will repeat every 5th retry.
W20250624 07:57:28.637857 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 176: this message will repeat every 5th retry.
I20250624 07:57:29.056777 21212 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 1590 ops behind the committed index [suppressed 27 similar messages]
W20250624 07:57:29.184559 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 181: this message will repeat every 5th retry.
I20250624 07:57:29.336338 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.365394 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.437319 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:29.445606 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:29.521965 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.614492 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.639194 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.642519 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:29.672473 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:29.761946 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 186: this message will repeat every 5th retry.
W20250624 07:57:30.217777 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 191: this message will repeat every 5th retry.
W20250624 07:57:30.771082 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 196: this message will repeat every 5th retry.
W20250624 07:57:31.226460 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 201: this message will repeat every 5th retry.
W20250624 07:57:31.688264 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 206: this message will repeat every 5th retry.
I20250624 07:57:32.092854 21215 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 1876 ops behind the committed index [suppressed 29 similar messages]
W20250624 07:57:32.224078 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 211: this message will repeat every 5th retry.
I20250624 07:57:32.347193 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 1/3 (6.54M/8.00M this segment, stats: ops{read=3778 overwritten=0 applied=3778 ignored=0} inserts{seen=188850 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:32.690575 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 216: this message will repeat every 5th retry.
W20250624 07:57:33.201184 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 221: this message will repeat every 5th retry.
W20250624 07:57:33.781742 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 226: this message will repeat every 5th retry.
W20250624 07:57:34.302470 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 231: this message will repeat every 5th retry.
W20250624 07:57:34.794584 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 236: this message will repeat every 5th retry.
I20250624 07:57:35.187316 21275 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 2187 ops behind the committed index [suppressed 29 similar messages]
W20250624 07:57:35.340653 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 241: this message will repeat every 5th retry.
W20250624 07:57:35.770581 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 246: this message will repeat every 5th retry.
W20250624 07:57:36.336863 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 251: this message will repeat every 5th retry.
I20250624 07:57:36.462666 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replayed 1/3 log segments. Stats: ops{read=4621 overwritten=0 applied=4620 ignored=0} inserts{seen=230950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250624 07:57:36.758991 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:36.836000 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:36.838231 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:36.887039 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:36.916716 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:36.936705 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:36.953502 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:36.980580 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:37.008137 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 256: this message will repeat every 5th retry.
I20250624 07:57:37.043614 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:37.353953 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (120.8K/8.00M this segment, stats: ops{read=4689 overwritten=0 applied=4689 ignored=0} inserts{seen=234400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:37.423784 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 261: this message will repeat every 5th retry.
W20250624 07:57:38.036214 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 266: this message will repeat every 5th retry.
I20250624 07:57:38.201786 21294 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 2418 ops behind the committed index [suppressed 27 similar messages]
W20250624 07:57:38.681712 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 271: this message will repeat every 5th retry.
W20250624 07:57:39.217955 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 276: this message will repeat every 5th retry.
W20250624 07:57:39.719909 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 281: this message will repeat every 5th retry.
W20250624 07:57:40.299170 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 286: this message will repeat every 5th retry.
W20250624 07:57:40.826598 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 291: this message will repeat every 5th retry.
I20250624 07:57:41.267115 21337 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 2668 ops behind the committed index [suppressed 26 similar messages]
W20250624 07:57:41.406824 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 296: this message will repeat every 5th retry.
W20250624 07:57:41.988520 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 301: this message will repeat every 5th retry.
I20250624 07:57:42.355602 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (1.45M/8.00M this segment, stats: ops{read=5459 overwritten=0 applied=5459 ignored=0} inserts{seen=272900 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:42.443428 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 306: this message will repeat every 5th retry.
W20250624 07:57:43.077098 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 311: this message will repeat every 5th retry.
W20250624 07:57:43.722702 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 316: this message will repeat every 5th retry.
I20250624 07:57:44.281345 21338 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 2901 ops behind the committed index [suppressed 25 similar messages]
W20250624 07:57:44.330849 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 321: this message will repeat every 5th retry.
I20250624 07:57:44.689836 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.733134 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:44.733503 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.755018 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.765615 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.771374 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:44.773180 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.851486 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:44.916250 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:45.126116 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 326: this message will repeat every 5th retry.
I20250624 07:57:45.604574 19772 maintenance_manager.cc:419] P 439cf2177bdb4c19aa8449b6cbd440a7: Scheduling FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa): perf score=0.073601
W20250624 07:57:45.749255 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 331: this message will repeat every 5th retry.
W20250624 07:57:46.306492 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 336: this message will repeat every 5th retry.
W20250624 07:57:46.845222 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 341: this message will repeat every 5th retry.
I20250624 07:57:47.287719 21336 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3099 ops behind the committed index [suppressed 22 similar messages]
I20250624 07:57:47.356309 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (2.65M/8.00M this segment, stats: ops{read=6150 overwritten=0 applied=6150 ignored=0} inserts{seen=307450 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:47.746739 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 346: this message will repeat every 5th retry.
W20250624 07:57:48.392786 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 351: this message will repeat every 5th retry.
W20250624 07:57:49.127678 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 356: this message will repeat every 5th retry.
W20250624 07:57:49.951349 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 361: this message will repeat every 5th retry.
I20250624 07:57:50.459390 21337 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3284 ops behind the committed index [suppressed 19 similar messages]
W20250624 07:57:50.828524 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 366: this message will repeat every 5th retry.
W20250624 07:57:51.670640 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 371: this message will repeat every 5th retry.
I20250624 07:57:52.361281 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (4.01M/8.00M this segment, stats: ops{read=6939 overwritten=0 applied=6937 ignored=0} inserts{seen=346800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:52.573009 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 376: this message will repeat every 5th retry.
I20250624 07:57:53.062987 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:53.081286 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:53.176070 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:57:53.176575 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:57:53.241873 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:53.301083 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:53.305543 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:53.320252 17273 kernel_stack_watchdog.cc:198] Thread 18555 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 178ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f53b20b7980 (unknown) at ??:0
@ 0x36853c __sanitizer::Quarantine<>::DoRecycle() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_atomic_clang_x86.h:81
@ 0x3696de __sanitizer::Quarantine<>::Recycle() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_quarantine.h:181
@ 0x36b77b __asan::Allocator::Deallocate() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_quarantine.h:112
@ 0x436ea1 operator delete() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:160 (discriminator 6)
@ 0x7f53ac00c63e google::protobuf::internal::ArenaStringPtr::Destroy() at ??:0
@ 0x7f53b0318d32 kudu::RowOperationsPB::SharedDtor() at ??:0
@ 0x7f53b0316345 kudu::RowOperationsPB::~RowOperationsPB() at ??:0
@ 0x7f53b82f1374 kudu::tserver::WriteRequestPB::SharedDtor() at ??:0
@ 0x7f53b82b2d05 kudu::tserver::WriteRequestPB::~WriteRequestPB() at ??:0
@ 0x7f53bcf0c432 kudu::rpc::RetriableRpc<>::~RetriableRpc() at ??:0
@ 0x7f53bcef08e9 kudu::client::internal::WriteRpc::~WriteRpc() at ??:0
@ 0x7f53bcef0fb3 kudu::client::internal::WriteRpc::~WriteRpc() at ??:0
@ 0x7f53bcf0d3e2 std::default_delete<>::operator()() at ??:0
@ 0x7f53bcefef5c std::unique_ptr<>::~unique_ptr() at ??:0
@ 0x7f53bcef1f4c kudu::client::internal::WriteRpc::Finish() at ??:0
I20250624 07:57:53.328488 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:57:53.330971 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 381: this message will repeat every 5th retry.
W20250624 07:57:53.321779 18555 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.16.221.69:35369, user_credentials={real_user=slave}} blocked reactor thread for 433250us
I20250624 07:57:53.399091 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:57:53.565456 21403 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3403 ops behind the committed index [suppressed 18 similar messages]
W20250624 07:57:54.014698 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 386: this message will repeat every 5th retry.
W20250624 07:57:54.680402 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 391: this message will repeat every 5th retry.
W20250624 07:57:55.265010 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 396: this message will repeat every 5th retry.
W20250624 07:57:55.820648 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 401: this message will repeat every 5th retry.
W20250624 07:57:56.315848 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 406: this message will repeat every 5th retry.
I20250624 07:57:56.676609 21488 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3580 ops behind the committed index [suppressed 25 similar messages]
W20250624 07:57:57.013134 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 411: this message will repeat every 5th retry.
I20250624 07:57:57.365793 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (5.26M/8.00M this segment, stats: ops{read=7657 overwritten=0 applied=7657 ignored=0} inserts{seen=382800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:57:57.616410 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 416: this message will repeat every 5th retry.
W20250624 07:57:58.389322 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 421: this message will repeat every 5th retry.
W20250624 07:57:59.064435 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 426: this message will repeat every 5th retry.
W20250624 07:57:59.699657 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 431: this message will repeat every 5th retry.
I20250624 07:57:59.789613 21491 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3736 ops behind the committed index [suppressed 22 similar messages]
W20250624 07:58:00.409368 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 436: this message will repeat every 5th retry.
W20250624 07:58:01.094687 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 441: this message will repeat every 5th retry.
W20250624 07:58:01.613554 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 446: this message will repeat every 5th retry.
I20250624 07:58:02.072474 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:58:02.128767 18582 meta_cache.cc:1261] Time spent looking up entry by key: real 0.061s user 0.000s sys 0.000s
W20250624 07:58:02.241952 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 451: this message will repeat every 5th retry.
I20250624 07:58:02.252569 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:02.277323 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:58:02.306596 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:58:02.306887 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:02.331604 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:02.334137 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:02.367842 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (6.28M/8.00M this segment, stats: ops{read=8250 overwritten=0 applied=8249 ignored=0} inserts{seen=412400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250624 07:58:02.397267 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:02.424947 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:58:02.807703 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 456: this message will repeat every 5th retry.
I20250624 07:58:02.875393 21500 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 3901 ops behind the committed index [suppressed 24 similar messages]
W20250624 07:58:03.409870 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 461: this message will repeat every 5th retry.
W20250624 07:58:03.925297 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 466: this message will repeat every 5th retry.
W20250624 07:58:04.431172 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 471: this message will repeat every 5th retry.
W20250624 07:58:05.032522 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 476: this message will repeat every 5th retry.
W20250624 07:58:05.674633 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 481: this message will repeat every 5th retry.
I20250624 07:58:05.932451 21530 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4070 ops behind the committed index [suppressed 26 similar messages]
W20250624 07:58:06.226760 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 486: this message will repeat every 5th retry.
W20250624 07:58:06.726972 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 491: this message will repeat every 5th retry.
W20250624 07:58:07.317243 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 496: this message will repeat every 5th retry.
I20250624 07:58:07.369369 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 2/3 (7.51M/8.00M this segment, stats: ops{read=8960 overwritten=0 applied=8960 ignored=0} inserts{seen=447950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:58:07.889621 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 501: this message will repeat every 5th retry.
W20250624 07:58:08.506680 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 506: this message will repeat every 5th retry.
I20250624 07:58:09.135968 21566 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4270 ops behind the committed index [suppressed 25 similar messages]
W20250624 07:58:09.449486 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 511: this message will repeat every 5th retry.
I20250624 07:58:09.992425 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replayed 2/3 log segments. Stats: ops{read=9245 overwritten=0 applied=9244 ignored=0} inserts{seen=462100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250624 07:58:10.123003 19561 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:58:10.124534 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 516: this message will repeat every 5th retry.
I20250624 07:58:10.160375 19063 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:10.287931 20142 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250624 07:58:10.360219 21022 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:10.363989 18782 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:10.384850 20874 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:10.387040 19421 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250624 07:58:10.410995 19703 tablet_service.cc:1430] Tablet server has 0 leaders and 3 scanners
I20250624 07:58:10.462908 18924 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250624 07:58:10.686571 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 521: this message will repeat every 5th retry.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:753: Failure
Failed
Bad status: Timed out: cluster didn't become healthy: Running 'cluster ksck 127.16.221.126:44487' did not succeed in 60 seconds: Runtime error: error running 'cluster ksck 127.16.221.126:44487': stdout: Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
b182c77196c04b549d02ed8a4be16054 | 127.16.221.126:44487 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.16.221.84:34517 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server States
Server | State
----------------------------------+------------------
6cbd668c726a4b1298e789270a0031e2 | MAINTENANCE_MODE
8150d3eeb1c84234a13484098d17ce8f | MAINTENANCE_MODE
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
1c0cb4e9f1d94dfbbfbaf9138aa4e655 | 127.16.221.65:45529 | HEALTHY | /L0 | 0 | 0
1fe60cc8e38948b694506041d7734713 | 127.16.221.71:44909 | HEALTHY | /L1 | 0 | 0
439cf2177bdb4c19aa8449b6cbd440a7 | 127.16.221.67:41189 | HEALTHY | /L0 | 0 | 3
4f219d2e8b954461973a8ca8680a6383 | 127.16.221.69:35369 | HEALTHY | /L1 | 1 | 0
6cbd668c726a4b1298e789270a0031e2 | 127.16.221.73:38265 | HEALTHY | /L2 | 0 | 0
8150d3eeb1c84234a13484098d17ce8f | 127.16.221.72:40741 | HEALTHY | /L2 | 0 | 0
bede5abcb6174794bd2283843157bcef | 127.16.221.70:41185 | HEALTHY | /L1 | 0 | 0
d33fd96ef7d44ad8805718466ca9447f | 127.16.221.68:42243 | HEALTHY | /L1 | 0 | 0
e1bf694ea1724daaa49f0f0746a03eeb | 127.16.221.66:42593 | HEALTHY | /L0 | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
/L2 | 2
/L0 | 3
/L1 | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.16.221.65 | experimental | 127.16.221.65:45529
local_ip_for_outbound_sockets | 127.16.221.66 | experimental | 127.16.221.66:42593
local_ip_for_outbound_sockets | 127.16.221.67 | experimental | 127.16.221.67:41189
local_ip_for_outbound_sockets | 127.16.221.68 | experimental | 127.16.221.68:42243
local_ip_for_outbound_sockets | 127.16.221.69 | experimental | 127.16.221.69:35369
local_ip_for_outbound_sockets | 127.16.221.70 | experimental | 127.16.221.70:41185
local_ip_for_outbound_sockets | 127.16.221.71 | experimental | 127.16.221.71:44909
local_ip_for_outbound_sockets | 127.16.221.72 | experimental | 127.16.221.72:40741
local_ip_for_outbound_sockets | 127.16.221.73 | experimental | 127.16.221.73:38265
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-0/data/info.pb | hidden | 127.16.221.65:45529
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-1/data/info.pb | hidden | 127.16.221.66:42593
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/data/info.pb | hidden | 127.16.221.67:41189
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-3/data/info.pb | hidden | 127.16.221.68:42243
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-4/data/info.pb | hidden | 127.16.221.69:35369
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-5/data/info.pb | hidden | 127.16.221.70:41185
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-6/data/info.pb | hidden | 127.16.221.71:44909
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-7/data/info.pb | hidden | 127.16.221.72:40741
server_dump_info_path | /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-8/data/info.pb | hidden | 127.16.221.73:38265
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.16.221.84:34517 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
Tablet 5ca5ea09d51f4eb390c4433fec8ed9fa of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): RUNNING
4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): RUNNING [LEADER]
6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap replayed 2/3 log segments. Stats: ops{read=9245 overwritten=0 applied=9244 ignored=0} inserts{seen=462100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
All reported replicas are:
A = 439cf2177bdb4c19aa8449b6cbd440a7
B = 4f219d2e8b954461973a8ca8680a6383
C = 6cbd668c726a4b1298e789270a0031e2
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
, stderr: W20250624 07:58:09.168478 21569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250624 07:58:09.168990 21569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250624 07:58:09.187798 21569 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Runtime error: ksck discovered errors
: /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu: process exited with non-zero status 1
W20250624 07:58:11.427768 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 526: this message will repeat every 5th retry.
W20250624 07:58:11.926002 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 531: this message will repeat every 5th retry.
I20250624 07:58:12.205101 21576 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 23 similar messages]
I20250624 07:58:12.376014 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 3/3 (572.4K/3.09M this segment, stats: ops{read=9572 overwritten=0 applied=9572 ignored=0} inserts{seen=478500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:58:12.446153 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 536: this message will repeat every 5th retry.
W20250624 07:58:12.963755 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 541: this message will repeat every 5th retry.
W20250624 07:58:13.516028 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 546: this message will repeat every 5th retry.
W20250624 07:58:14.070739 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 551: this message will repeat every 5th retry.
W20250624 07:58:14.551173 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 556: this message will repeat every 5th retry.
W20250624 07:58:15.063230 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 561: this message will repeat every 5th retry.
I20250624 07:58:15.232630 21628 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 28 similar messages]
W20250624 07:58:15.606241 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 566: this message will repeat every 5th retry.
W20250624 07:58:16.074327 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 571: this message will repeat every 5th retry.
W20250624 07:58:16.568455 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 576: this message will repeat every 5th retry.
W20250624 07:58:17.070832 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 581: this message will repeat every 5th retry.
I20250624 07:58:17.376118 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replaying log segment 3/3 (2.36M/3.09M this segment, stats: ops{read=10626 overwritten=0 applied=10625 ignored=0} inserts{seen=531150 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250624 07:58:17.632210 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 586: this message will repeat every 5th retry.
W20250624 07:58:18.142153 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 591: this message will repeat every 5th retry.
I20250624 07:58:18.269800 21631 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 4391 ops behind the committed index [suppressed 29 similar messages]
W20250624 07:58:18.641099 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 596: this message will repeat every 5th retry.
W20250624 07:58:19.116403 20094 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 -> Peer 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265): Couldn't send request to peer 6cbd668c726a4b1298e789270a0031e2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 601: this message will repeat every 5th retry.
I20250624 07:58:19.437165 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap replayed 3/3 log segments. Stats: ops{read=11055 overwritten=0 applied=11055 ignored=0} inserts{seen=552650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250624 07:58:19.438315 20980 tablet_bootstrap.cc:492] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Bootstrap complete.
I20250624 07:58:19.445492 20980 ts_tablet_manager.cc:1397] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent bootstrapping tablet: real 68.600s user 46.265s sys 1.569s
I20250624 07:58:19.451423 20980 raft_consensus.cc:357] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.452657 20980 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Initialized, Role: FOLLOWER
I20250624 07:58:19.453598 20980 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11055, Last appended: 2.11055, Last appended by leader: 11055, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.462325 20980 ts_tablet_manager.cc:1428] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
I20250624 07:58:19.818593 21632 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:19.819025 21632 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:19.821497 21632 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:19.856235 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 11480 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:19.873847 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 11480 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:19.875397 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:20.170615 21632 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:21.375931 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:21.376569 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:21.378225 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:21.379071 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 13605 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:21.380018 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 13605 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:21.381042 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:21.381672 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:21.389463 21631 consensus_queue.cc:786] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer 6cbd668c726a4b1298e789270a0031e2 is lagging by at least 1841 ops behind the committed index [suppressed 17 similar messages]
I20250624 07:58:21.731043 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:21.731282 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:21.732007 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:21.732952 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14030 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:21.732983 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14030 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:21.735353 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:21.736313 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:22.102583 21646 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:22.102818 21646 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:22.103601 21646 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:22.104065 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14455 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:22.104283 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 14455 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:22.104905 20977 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:22.106537 21646 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:22.742956 21648 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:22.743311 21648 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:22.744582 21648 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:22.745646 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15305 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:22.750190 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15305 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:22.751720 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:22.755040 21648 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:23.308735 21648 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:23.308986 21648 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:23.309844 21648 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:23.311044 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:23.312011 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:23.312705 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6cbd668c726a4b1298e789270a0031e2; no voters: 439cf2177bdb4c19aa8449b6cbd440a7, 4f219d2e8b954461973a8ca8680a6383
I20250624 07:58:23.313453 21648 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250624 07:58:23.394874 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 37.790s user 24.589s sys 0.426s Metrics: {"bytes_written":20043962,"cfile_init":1,"dirs.queue_time_us":14808,"dirs.run_cpu_time_us":274,"dirs.run_wall_time_us":1030,"drs_written":1,"lbm_read_time_us":115,"lbm_reads_lt_1ms":4,"lbm_write_time_us":172663,"lbm_writes_1-10_ms":14,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":859,"peak_mem_usage":0,"rows_written":722800,"thread_start_us":21608,"threads_started":2,"wal-append.queue_time_us":6873}
I20250624 07:58:23.397328 19772 maintenance_manager.cc:419] P 439cf2177bdb4c19aa8449b6cbd440a7: Scheduling UndoDeltaBlockGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa): 8294353 bytes on disk
I20250624 07:58:23.398746 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: UndoDeltaBlockGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":121,"lbm_reads_lt_1ms":4}
I20250624 07:58:26.406147 19772 maintenance_manager.cc:419] P 439cf2177bdb4c19aa8449b6cbd440a7: Scheduling LogGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa): free 25168025 bytes of WAL
I20250624 07:58:26.406726 19659 log_reader.cc:385] T 5ca5ea09d51f4eb390c4433fec8ed9fa: removed 3 log segments from log reader
I20250624 07:58:26.406890 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000001 (ops 1-4937)
I20250624 07:58:26.407029 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000002 (ops 4938-9696)
I20250624 07:58:26.407121 19659 log.cc:1079] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7: Deleting log segment in path: /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0/minicluster-data/ts-2/wal/wals/5ca5ea09d51f4eb390c4433fec8ed9fa/wal-000000003 (ops 9697-14400)
I20250624 07:58:26.416723 19659 maintenance_manager.cc:643] P 439cf2177bdb4c19aa8449b6cbd440a7: LogGCOp(5ca5ea09d51f4eb390c4433fec8ed9fa) complete. Timing: real 0.010s user 0.000s sys 0.007s Metrics: {}
I20250624 07:58:26.475273 17269 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250624 07:58:26.475557 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 1c0cb4e9f1d94dfbbfbaf9138aa4e655 and pid 19496
************************ BEGIN STACKS **************************
I20250624 07:58:27.180711 20211 maintenance_manager.cc:419] P 4f219d2e8b954461973a8ca8680a6383: Scheduling FlushMRSOp(5ca5ea09d51f4eb390c4433fec8ed9fa): perf score=0.080383
[New LWP 19498]
[New LWP 19499]
[New LWP 19500]
[New LWP 19501]
[New LWP 19508]
[New LWP 19509]
[New LWP 19510]
[New LWP 19513]
[New LWP 19514]
[New LWP 19515]
[New LWP 19517]
[New LWP 19518]
[New LWP 19519]
[New LWP 19520]
[New LWP 19521]
[New LWP 19522]
[New LWP 19523]
[New LWP 19524]
[New LWP 19525]
[New LWP 19526]
[New LWP 19527]
[New LWP 19528]
[New LWP 19529]
[New LWP 19530]
[New LWP 19531]
[New LWP 19532]
[New LWP 19533]
[New LWP 19534]
[New LWP 19535]
[New LWP 19536]
[New LWP 19537]
[New LWP 19538]
[New LWP 19539]
[New LWP 19540]
[New LWP 19541]
[New LWP 19542]
[New LWP 19543]
[New LWP 19544]
[New LWP 19545]
[New LWP 19546]
[New LWP 19547]
[New LWP 19548]
[New LWP 19549]
[New LWP 19550]
[New LWP 19551]
[New LWP 19552]
[New LWP 19553]
[New LWP 19554]
[New LWP 19555]
[New LWP 19556]
[New LWP 19557]
[New LWP 19558]
[New LWP 19559]
[New LWP 19560]
[New LWP 19561]
[New LWP 19562]
[New LWP 19563]
[New LWP 19564]
[New LWP 19565]
[New LWP 19566]
[New LWP 19567]
[New LWP 19568]
[New LWP 19569]
[New LWP 19570]
[New LWP 19571]
[New LWP 19572]
[New LWP 19573]
[New LWP 19574]
[New LWP 19575]
[New LWP 19576]
[New LWP 19577]
[New LWP 19578]
[New LWP 19579]
[New LWP 19580]
[New LWP 19581]
[New LWP 19582]
[New LWP 19583]
[New LWP 19584]
[New LWP 19585]
[New LWP 19586]
[New LWP 19587]
[New LWP 19588]
[New LWP 19589]
[New LWP 19590]
[New LWP 19591]
[New LWP 19592]
[New LWP 19593]
[New LWP 19594]
[New LWP 19595]
[New LWP 19596]
[New LWP 19597]
[New LWP 19598]
[New LWP 19599]
[New LWP 19600]
[New LWP 19601]
[New LWP 19602]
[New LWP 19603]
[New LWP 19604]
[New LWP 19605]
[New LWP 19606]
[New LWP 19607]
[New LWP 19608]
[New LWP 19609]
[New LWP 19610]
[New LWP 19611]
[New LWP 19612]
[New LWP 19613]
[New LWP 19614]
[New LWP 19615]
[New LWP 19616]
[New LWP 19617]
[New LWP 19618]
[New LWP 19619]
[New LWP 19620]
[New LWP 19621]
[New LWP 19622]
[New LWP 19623]
[New LWP 19624]
[New LWP 19625]
[New LWP 19626]
[New LWP 19634]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f544da40d50 in ?? ()
Id Target Id Frame
* 1 LWP 19496 "kudu" 0x00007f544da40d50 in ?? ()
2 LWP 19498 "kudu" 0x00007f544da3cfb9 in ?? ()
3 LWP 19499 "kudu" 0x00007f544da3cfb9 in ?? ()
4 LWP 19500 "kudu" 0x00007f544da3cfb9 in ?? ()
5 LWP 19501 "kernel-watcher-" 0x00007f544da3cfb9 in ?? ()
6 LWP 19508 "ntp client-1950" 0x00007f544da409e2 in ?? ()
7 LWP 19509 "file cache-evic" 0x00007f544da3cfb9 in ?? ()
8 LWP 19510 "sq_acceptor" 0x00007f5446762cb9 in ?? ()
9 LWP 19513 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
10 LWP 19514 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
11 LWP 19515 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
12 LWP 19517 "rpc reactor-195" 0x00007f544676fa47 in ?? ()
13 LWP 19518 "MaintenanceMgr " 0x00007f544da3cad3 in ?? ()
14 LWP 19519 "txn-status-mana" 0x00007f544da3cfb9 in ?? ()
15 LWP 19520 "collect_and_rem" 0x00007f544da3cfb9 in ?? ()
16 LWP 19521 "tc-session-exp-" 0x00007f544da3cfb9 in ?? ()
17 LWP 19522 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
18 LWP 19523 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
19 LWP 19524 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
20 LWP 19525 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
21 LWP 19526 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
22 LWP 19527 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
23 LWP 19528 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
24 LWP 19529 "rpc worker-1952" 0x00007f544da3cad3 in ?? ()
25 LWP 19530 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
26 LWP 19531 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
27 LWP 19532 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
28 LWP 19533 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
29 LWP 19534 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
30 LWP 19535 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
31 LWP 19536 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
32 LWP 19537 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
33 LWP 19538 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
34 LWP 19539 "rpc worker-1953" 0x00007f544da3cad3 in ?? ()
35 LWP 19540 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
36 LWP 19541 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
37 LWP 19542 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
38 LWP 19543 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
39 LWP 19544 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
40 LWP 19545 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
41 LWP 19546 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
42 LWP 19547 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
43 LWP 19548 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
44 LWP 19549 "rpc worker-1954" 0x00007f544da3cad3 in ?? ()
45 LWP 19550 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
46 LWP 19551 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
47 LWP 19552 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
48 LWP 19553 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
49 LWP 19554 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
50 LWP 19555 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
51 LWP 19556 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
52 LWP 19557 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
53 LWP 19558 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
54 LWP 19559 "rpc worker-1955" 0x00007f544da3cad3 in ?? ()
55 LWP 19560 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
56 LWP 19561 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
57 LWP 19562 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
58 LWP 19563 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
59 LWP 19564 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
60 LWP 19565 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
61 LWP 19566 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
62 LWP 19567 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
63 LWP 19568 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
64 LWP 19569 "rpc worker-1956" 0x00007f544da3cad3 in ?? ()
65 LWP 19570 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
66 LWP 19571 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
67 LWP 19572 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
68 LWP 19573 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
69 LWP 19574 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
70 LWP 19575 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
71 LWP 19576 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
72 LWP 19577 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
73 LWP 19578 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
74 LWP 19579 "rpc worker-1957" 0x00007f544da3cad3 in ?? ()
75 LWP 19580 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
76 LWP 19581 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
77 LWP 19582 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
78 LWP 19583 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
79 LWP 19584 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
80 LWP 19585 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
81 LWP 19586 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
82 LWP 19587 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
83 LWP 19588 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
84 LWP 19589 "rpc worker-1958" 0x00007f544da3cad3 in ?? ()
85 LWP 19590 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
86 LWP 19591 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
87 LWP 19592 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
88 LWP 19593 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
89 LWP 19594 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
90 LWP 19595 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
91 LWP 19596 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
92 LWP 19597 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
93 LWP 19598 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
94 LWP 19599 "rpc worker-1959" 0x00007f544da3cad3 in ?? ()
95 LWP 19600 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
96 LWP 19601 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
97 LWP 19602 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
98 LWP 19603 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
99 LWP 19604 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
100 LWP 19605 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
101 LWP 19606 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
102 LWP 19607 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
103 LWP 19608 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
104 LWP 19609 "rpc worker-1960" 0x00007f544da3cad3 in ?? ()
105 LWP 19610 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
106 LWP 19611 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
107 LWP 19612 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
108 LWP 19613 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
109 LWP 19614 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
110 LWP 19615 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
111 LWP 19616 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
112 LWP 19617 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
113 LWP 19618 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
114 LWP 19619 "rpc worker-1961" 0x00007f544da3cad3 in ?? ()
115 LWP 19620 "rpc worker-1962" 0x00007f544da3cad3 in ?? ()
116 LWP 19621 "rpc worker-1962" 0x00007f544da3cad3 in ?? ()
117 LWP 19622 "diag-logger-196" 0x00007f544da3cfb9 in ?? ()
118 LWP 19623 "result-tracker-" 0x00007f544da3cfb9 in ?? ()
119 LWP 19624 "excess-log-dele" 0x00007f544da3cfb9 in ?? ()
120 LWP 19625 "acceptor-19625" 0x00007f54467710c7 in ?? ()
121 LWP 19626 "heartbeat-19626" 0x00007f54513041e1 in ?? ()
122 LWP 19634 "maintenance_sch" 0x00007f544da3cfb9 in ?? ()
Thread 122 (LWP 19634):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5402ad56e0 in ?? ()
#2 0x00000000000002c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f5402ad5730 in ?? ()
#6 0x0000000000000592 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19626):
#0 0x00007f54513041e1 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 120 (LWP 19625):
#0 0x00007f54467710c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f5404323d90 in ?? ()
#3 0x00007f5404323dc0 in ?? ()
#4 0x00007f5404323ec0 in ?? ()
#5 0x00007f5404323db0 in ?? ()
#6 0x00007f5404323e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f544a5f7980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19624):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5404b3df40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007fff340afc50 in ?? ()
#5 0x00007f5404b3df30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19623):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f5405356100 in ?? ()
#2 0x00000000000000b2 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f54053560f0 in ?? ()
#6 0x0000000000000164 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19622):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19621):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000024 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f5406385e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5406385eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 19620):
#0 0x00007f544da3cad3 in ?? ()
#1 0x000000000000001c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f5406b9de90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5406b9deb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 19619):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19618):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19617):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19616):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19615):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19614):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19613):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19612):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19611):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19610):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19609):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19608):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19607):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19606):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19605):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19604):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19603):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19602):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19601):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19600):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19599):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19598):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19597):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19596):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19595):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19594):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19593):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19592):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19591):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19590):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19589):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19588):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19587):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19586):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19585):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19584):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19583):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19582):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19581):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f5419f3ee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5419f3eeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19580):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19579):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19578):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19577):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19576):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19575):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19574):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19573):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19572):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19571):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19570):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19569):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19568):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19567):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19566):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19565):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19564):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19563):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19562):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19561):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f542411fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f542411feb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19560):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19559):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19558):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19557):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19556):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19555):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19554):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19553):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19552):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19551):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19550):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19549):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19548):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19547):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19546):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19545):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19544):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19543):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19542):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19541):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f542e300e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f542e300eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 19540):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19539):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19538):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19537):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19536):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19535):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19534):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19533):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19532):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19531):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19530):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19529):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19528):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19527):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19526):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19525):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19524):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19523):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19522):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19521):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f54383c2cc0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f54383c2cb0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19520):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19519):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f54393c4240 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f54393c4230 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19518):
#0 0x00007f544da3cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19517):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543a3c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f543a3c7310 in ?? ()
#4 0x00007f543a3c7520 in ?? ()
#5 0x00007f543a3c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f543a3c73e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3f50624dd2f1a9fc in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x000000007c900000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000feb07470e7c in ?? ()
#17 0x00007f543a3c73c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543a3c7390 in ?? ()
#20 0x3f50624dd2f1a9fc in ?? ()
#21 0x000000003a3c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3f50624dd2f1a9fc in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19515):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543abc8320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f543abc8310 in ?? ()
#4 0x00007f543abc8520 in ?? ()
#5 0x00007f543abc8360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f543abc83e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb95cd615b3e000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000000c000000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000feb0757107c in ?? ()
#17 0x00007f543abc83c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543abc8390 in ?? ()
#20 0x3fb95cd615b3e000 in ?? ()
#21 0x000000003abc83e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb95cd615b3e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19514):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543b3c9320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f543b3c9310 in ?? ()
#4 0x00007f543b3c9520 in ?? ()
#5 0x00007f543b3c9360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f543b3c93e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb97a117a63e000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000d5000000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000feb0767127c in ?? ()
#17 0x00007f543b3c93c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x00007f543b3c9390 in ?? ()
#20 0x3fb97a117a63e000 in ?? ()
#21 0x000000003b3c93e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb97a117a63e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19513):
#0 0x00007f544676fa47 in ?? ()
#1 0x00007f543cfb3320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f543cfb3310 in ?? ()
#4 0x00007f543cfb3520 in ?? ()
#5 0x00007f543cfb3360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f543cfb33e0 in ?? ()
#8 0x00007f5448de225d in ?? ()
#9 0x3fb9821e2c56a000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000010800000 in ?? ()
#14 0x00007f5400000000 in ?? ()
#15 0x41da1696258f83a5 in ?? ()
#16 0x00000feb079ee67c in ?? ()
#17 0x00007f543cfb33c0 in ?? ()
#18 0x00007f5448de6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19510):
#0 0x00007f5446762cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f543efb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19509):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f543e7b60c0 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f543e7b60b0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19508):
#0 0x00007f544da409e2 in ?? ()
#1 0x00007f543d7b2ba0 in ?? ()
#2 0x00007f543d7b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f543d7b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f543d7b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f543d7b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f5453353edc in ?? ()
#12 0x00007f54533438d9 in ?? ()
#13 0x00007f5453347ec6 in ?? ()
#14 0x00007f545334c900 in ?? ()
#15 0x00007f545334c729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f544a6184fe in ?? ()
#18 0x00007f544da366db in ?? ()
#19 0x00000feb07aee484 in ?? ()
#20 0x00007f543d7b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19501):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x00007f543ffb8c80 in ?? ()
#2 0x000000000000037e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f543ffb8c70 in ?? ()
#6 0x00000000000006fc in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19500):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19499):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19498):
#0 0x00007f544da3cfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f54417bc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19496):
#0 0x00007f544da40d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:27.932780 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID e1bf694ea1724daaa49f0f0746a03eeb and pid 19354
************************ BEGIN STACKS **************************
[New LWP 19359]
[New LWP 19360]
[New LWP 19361]
[New LWP 19362]
[New LWP 19368]
[New LWP 19369]
[New LWP 19370]
[New LWP 19373]
[New LWP 19374]
[New LWP 19375]
[New LWP 19376]
[New LWP 19377]
[New LWP 19378]
[New LWP 19379]
[New LWP 19380]
[New LWP 19381]
[New LWP 19382]
[New LWP 19383]
[New LWP 19384]
[New LWP 19385]
[New LWP 19386]
[New LWP 19388]
[New LWP 19389]
[New LWP 19390]
[New LWP 19391]
[New LWP 19392]
[New LWP 19393]
[New LWP 19394]
[New LWP 19395]
[New LWP 19396]
[New LWP 19397]
[New LWP 19398]
[New LWP 19399]
[New LWP 19400]
[New LWP 19401]
[New LWP 19402]
[New LWP 19403]
[New LWP 19404]
[New LWP 19405]
[New LWP 19406]
[New LWP 19407]
[New LWP 19408]
[New LWP 19409]
[New LWP 19410]
[New LWP 19411]
[New LWP 19412]
[New LWP 19413]
[New LWP 19414]
[New LWP 19415]
[New LWP 19416]
[New LWP 19417]
[New LWP 19418]
[New LWP 19419]
[New LWP 19420]
[New LWP 19421]
[New LWP 19422]
[New LWP 19423]
[New LWP 19424]
[New LWP 19425]
[New LWP 19426]
[New LWP 19427]
[New LWP 19428]
[New LWP 19429]
[New LWP 19430]
[New LWP 19431]
[New LWP 19432]
[New LWP 19433]
[New LWP 19434]
[New LWP 19435]
[New LWP 19436]
[New LWP 19437]
[New LWP 19438]
[New LWP 19439]
[New LWP 19440]
[New LWP 19441]
[New LWP 19442]
[New LWP 19443]
[New LWP 19444]
[New LWP 19445]
[New LWP 19446]
[New LWP 19447]
[New LWP 19448]
[New LWP 19449]
[New LWP 19450]
[New LWP 19451]
[New LWP 19452]
[New LWP 19453]
[New LWP 19454]
[New LWP 19455]
[New LWP 19456]
[New LWP 19457]
[New LWP 19458]
[New LWP 19459]
[New LWP 19460]
[New LWP 19461]
[New LWP 19462]
[New LWP 19463]
[New LWP 19464]
[New LWP 19465]
[New LWP 19466]
[New LWP 19467]
[New LWP 19468]
[New LWP 19469]
[New LWP 19470]
[New LWP 19471]
[New LWP 19472]
[New LWP 19473]
[New LWP 19474]
[New LWP 19475]
[New LWP 19476]
[New LWP 19477]
[New LWP 19478]
[New LWP 19479]
[New LWP 19480]
[New LWP 19481]
[New LWP 19482]
[New LWP 19483]
[New LWP 19484]
[New LWP 19485]
[New LWP 19486]
[New LWP 19487]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f120a610d50 in ?? ()
Id Target Id Frame
* 1 LWP 19354 "kudu" 0x00007f120a610d50 in ?? ()
2 LWP 19359 "kudu" 0x00007f120a60cfb9 in ?? ()
3 LWP 19360 "kudu" 0x00007f120a60cfb9 in ?? ()
4 LWP 19361 "kudu" 0x00007f120a60cfb9 in ?? ()
5 LWP 19362 "kernel-watcher-" 0x00007f120a60cfb9 in ?? ()
6 LWP 19368 "ntp client-1936" 0x00007f120a6109e2 in ?? ()
7 LWP 19369 "file cache-evic" 0x00007f120a60cfb9 in ?? ()
8 LWP 19370 "sq_acceptor" 0x00007f1203332cb9 in ?? ()
9 LWP 19373 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
10 LWP 19374 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
11 LWP 19375 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
12 LWP 19376 "rpc reactor-193" 0x00007f120333fa47 in ?? ()
13 LWP 19377 "MaintenanceMgr " 0x00007f120a60cad3 in ?? ()
14 LWP 19378 "txn-status-mana" 0x00007f120a60cfb9 in ?? ()
15 LWP 19379 "collect_and_rem" 0x00007f120a60cfb9 in ?? ()
16 LWP 19380 "tc-session-exp-" 0x00007f120a60cfb9 in ?? ()
17 LWP 19381 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
18 LWP 19382 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
19 LWP 19383 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
20 LWP 19384 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
21 LWP 19385 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
22 LWP 19386 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
23 LWP 19388 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
24 LWP 19389 "rpc worker-1938" 0x00007f120a60cad3 in ?? ()
25 LWP 19390 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
26 LWP 19391 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
27 LWP 19392 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
28 LWP 19393 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
29 LWP 19394 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
30 LWP 19395 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
31 LWP 19396 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
32 LWP 19397 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
33 LWP 19398 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
34 LWP 19399 "rpc worker-1939" 0x00007f120a60cad3 in ?? ()
35 LWP 19400 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
36 LWP 19401 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
37 LWP 19402 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
38 LWP 19403 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
39 LWP 19404 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
40 LWP 19405 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
41 LWP 19406 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
42 LWP 19407 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
43 LWP 19408 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
44 LWP 19409 "rpc worker-1940" 0x00007f120a60cad3 in ?? ()
45 LWP 19410 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
46 LWP 19411 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
47 LWP 19412 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
48 LWP 19413 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
49 LWP 19414 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
50 LWP 19415 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
51 LWP 19416 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
52 LWP 19417 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
53 LWP 19418 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
54 LWP 19419 "rpc worker-1941" 0x00007f120a60cad3 in ?? ()
55 LWP 19420 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
56 LWP 19421 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
57 LWP 19422 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
58 LWP 19423 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
59 LWP 19424 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
60 LWP 19425 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
61 LWP 19426 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
62 LWP 19427 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
63 LWP 19428 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
64 LWP 19429 "rpc worker-1942" 0x00007f120a60cad3 in ?? ()
65 LWP 19430 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
66 LWP 19431 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
67 LWP 19432 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
68 LWP 19433 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
69 LWP 19434 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
70 LWP 19435 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
71 LWP 19436 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
72 LWP 19437 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
73 LWP 19438 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
74 LWP 19439 "rpc worker-1943" 0x00007f120a60cad3 in ?? ()
75 LWP 19440 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
76 LWP 19441 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
77 LWP 19442 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
78 LWP 19443 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
79 LWP 19444 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
80 LWP 19445 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
81 LWP 19446 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
82 LWP 19447 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
83 LWP 19448 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
84 LWP 19449 "rpc worker-1944" 0x00007f120a60cad3 in ?? ()
85 LWP 19450 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
86 LWP 19451 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
87 LWP 19452 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
88 LWP 19453 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
89 LWP 19454 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
90 LWP 19455 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
91 LWP 19456 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
92 LWP 19457 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
93 LWP 19458 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
94 LWP 19459 "rpc worker-1945" 0x00007f120a60cad3 in ?? ()
95 LWP 19460 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
96 LWP 19461 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
97 LWP 19462 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
98 LWP 19463 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
99 LWP 19464 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
100 LWP 19465 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
101 LWP 19466 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
102 LWP 19467 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
103 LWP 19468 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
104 LWP 19469 "rpc worker-1946" 0x00007f120a60cad3 in ?? ()
105 LWP 19470 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
106 LWP 19471 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
107 LWP 19472 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
108 LWP 19473 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
109 LWP 19474 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
110 LWP 19475 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
111 LWP 19476 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
112 LWP 19477 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
113 LWP 19478 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
114 LWP 19479 "rpc worker-1947" 0x00007f120a60cad3 in ?? ()
115 LWP 19480 "rpc worker-1948" 0x00007f120a60cad3 in ?? ()
116 LWP 19481 "rpc worker-1948" 0x00007f120a60cad3 in ?? ()
117 LWP 19482 "diag-logger-194" 0x00007f120a60cfb9 in ?? ()
118 LWP 19483 "result-tracker-" 0x00007f120a60cfb9 in ?? ()
119 LWP 19484 "excess-log-dele" 0x00007f120a60cfb9 in ?? ()
120 LWP 19485 "acceptor-19485" 0x00007f12033410c7 in ?? ()
121 LWP 19486 "heartbeat-19486" 0x00007f120a60cfb9 in ?? ()
122 LWP 19487 "maintenance_sch" 0x00007f120a60cfb9 in ?? ()
Thread 122 (LWP 19487):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11bfeb46e0 in ?? ()
#2 0x00000000000002d7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f11bfeb4730 in ?? ()
#6 0x00000000000005ae in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19486):
#0 0x00007f120a60cfb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000b5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f11c06cc5f0 in ?? ()
#6 0x000000000000016a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19485):
#0 0x00007f12033410c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f11c0ef3d90 in ?? ()
#3 0x00007f11c0ef3dc0 in ?? ()
#4 0x00007f11c0ef3ec0 in ?? ()
#5 0x00007f11c0ef3db0 in ?? ()
#6 0x00007f11c0ef3e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f12071c7980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19484):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11c170df40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffd064c3530 in ?? ()
#5 0x00007f11c170df30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19483):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11c1f26100 in ?? ()
#2 0x00000000000000b6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f11c1f260f0 in ?? ()
#6 0x000000000000016c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19482):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19481):
#0 0x00007f120a60cad3 in ?? ()
#1 0x000000000000002b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f11c2f55e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11c2f55eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f120a60c770 in ?? ()
#10 0x00007f11c2f55eb0 in ?? ()
#11 0x00000fe2385eabd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19480):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000015 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f11c376de90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11c376deb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f120a60c770 in ?? ()
#10 0x00007f11c376deb0 in ?? ()
#11 0x00000fe2386edbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19479):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19478):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19477):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19476):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19475):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19474):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19473):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19472):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19471):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19470):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19469):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19468):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19467):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19466):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19465):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19464):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19463):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19462):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19461):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19460):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19459):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19458):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19457):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19456):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19455):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19454):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19453):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19452):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19451):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19450):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19449):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19448):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19447):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19446):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19445):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19444):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19443):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19442):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19441):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f11d6b0ee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11d6b0eeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19440):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19439):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19438):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19437):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19436):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19435):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19434):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19433):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19432):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19431):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19430):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19429):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19428):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19427):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19426):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19425):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19424):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19423):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19422):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19421):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f11e0cefe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11e0cefeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19420):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19419):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19418):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19417):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19416):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19415):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19414):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19413):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19412):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19411):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19410):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19409):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19408):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19407):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19406):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19405):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19404):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19403):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19402):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19401):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f11eaed0e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f11eaed0eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 19400):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19399):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19398):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19397):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19396):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19395):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19394):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19393):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19392):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19391):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19390):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19389):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19388):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19386):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19385):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19384):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19383):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19382):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19381):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19380):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11f50d3cc0 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f11f50d3cb0 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19379):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19378):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11f6111240 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f11f6111230 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19377):
#0 0x00007f120a60cad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19376):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f7160320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f11f7160310 in ?? ()
#4 0x00007f11f7160520 in ?? ()
#5 0x00007f11f7160360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f11f71603e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb963499566c000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000c4400000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe2bee2407c in ?? ()
#17 0x00007f11f71603c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f7160390 in ?? ()
#20 0x3fb963499566c000 in ?? ()
#21 0x00000000f71603e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb963499566c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19375):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f7977320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f11f7977310 in ?? ()
#4 0x00007f11f7977520 in ?? ()
#5 0x00007f11f7977360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f11f79773e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb94bc99c076000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000003c000000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fe2bef26e7c in ?? ()
#17 0x00007f11f79773c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f7977390 in ?? ()
#20 0x3fb94bc99c076000 in ?? ()
#21 0x00000000f79773e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb94bc99c076000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19374):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11f818e320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f11f818e310 in ?? ()
#4 0x00007f11f818e520 in ?? ()
#5 0x00007f11f818e360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f11f818e3e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb94961f3e8a000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x0000000093c00000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fe2bf029c7c in ?? ()
#17 0x00007f11f818e3c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x00007f11f818e390 in ?? ()
#20 0x3fb94961f3e8a000 in ?? ()
#21 0x00000000f818e3e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb94961f3e8a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19373):
#0 0x00007f120333fa47 in ?? ()
#1 0x00007f11fa1ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f11fa1ad310 in ?? ()
#4 0x00007f11fa1ad520 in ?? ()
#5 0x00007f11fa1ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f11fa1ad3e0 in ?? ()
#8 0x00007f12059b225d in ?? ()
#9 0x3fb98aab10228000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000009a800000 in ?? ()
#14 0x00007f1100000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe2bf42da7c in ?? ()
#17 0x00007f11fa1ad3c0 in ?? ()
#18 0x00007f12059b6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19370):
#0 0x00007f1203332cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f11fbbb77c8 in ?? ()
#3 0x000060200001ca10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19369):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11fb3b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f11fb3b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19368):
#0 0x00007f120a6109e2 in ?? ()
#1 0x00007f11fabb3ba0 in ?? ()
#2 0x00007f11fabb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f11fabb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f11fabb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f11fabb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f120ff23edc in ?? ()
#12 0x00007f120ff138d9 in ?? ()
#13 0x00007f120ff17ec6 in ?? ()
#14 0x00007f120ff1c900 in ?? ()
#15 0x00007f120ff1c729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f12071e84fe in ?? ()
#18 0x00007f120a6066db in ?? ()
#19 0x00000fe2bf56e684 in ?? ()
#20 0x00007f11fabb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19362):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x00007f11fcbb8c80 in ?? ()
#2 0x000000000000038f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f11fcbb8c70 in ?? ()
#6 0x000000000000071e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19361):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19360):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19359):
#0 0x00007f120a60cfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f11fe3bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19354):
#0 0x00007f120a610d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:29.398350 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID 439cf2177bdb4c19aa8449b6cbd440a7 and pid 19638
************************ BEGIN STACKS **************************
[New LWP 19641]
[New LWP 19642]
[New LWP 19643]
[New LWP 19644]
[New LWP 19650]
[New LWP 19651]
[New LWP 19652]
[New LWP 19655]
[New LWP 19656]
[New LWP 19657]
[New LWP 19658]
[New LWP 19659]
[New LWP 19660]
[New LWP 19662]
[New LWP 19663]
[New LWP 19664]
[New LWP 19665]
[New LWP 19666]
[New LWP 19667]
[New LWP 19668]
[New LWP 19669]
[New LWP 19670]
[New LWP 19671]
[New LWP 19672]
[New LWP 19673]
[New LWP 19674]
[New LWP 19675]
[New LWP 19676]
[New LWP 19677]
[New LWP 19678]
[New LWP 19679]
[New LWP 19680]
[New LWP 19681]
[New LWP 19682]
[New LWP 19683]
[New LWP 19684]
[New LWP 19685]
[New LWP 19686]
[New LWP 19687]
[New LWP 19688]
[New LWP 19689]
[New LWP 19690]
[New LWP 19691]
[New LWP 19692]
[New LWP 19693]
[New LWP 19694]
[New LWP 19695]
[New LWP 19696]
[New LWP 19697]
[New LWP 19698]
[New LWP 19699]
[New LWP 19700]
[New LWP 19701]
[New LWP 19702]
[New LWP 19703]
[New LWP 19704]
[New LWP 19705]
[New LWP 19706]
[New LWP 19707]
[New LWP 19708]
[New LWP 19709]
[New LWP 19712]
[New LWP 19713]
[New LWP 19714]
[New LWP 19715]
[New LWP 19716]
[New LWP 19717]
[New LWP 19718]
[New LWP 19719]
[New LWP 19720]
[New LWP 19721]
[New LWP 19722]
[New LWP 19723]
[New LWP 19724]
[New LWP 19725]
[New LWP 19726]
[New LWP 19727]
[New LWP 19728]
[New LWP 19729]
[New LWP 19730]
[New LWP 19731]
[New LWP 19732]
[New LWP 19733]
[New LWP 19734]
[New LWP 19735]
[New LWP 19736]
[New LWP 19737]
[New LWP 19738]
[New LWP 19739]
[New LWP 19740]
[New LWP 19741]
[New LWP 19742]
[New LWP 19743]
[New LWP 19744]
[New LWP 19745]
[New LWP 19746]
[New LWP 19747]
[New LWP 19748]
[New LWP 19749]
[New LWP 19750]
[New LWP 19751]
[New LWP 19752]
[New LWP 19753]
[New LWP 19754]
[New LWP 19755]
[New LWP 19756]
[New LWP 19757]
[New LWP 19758]
[New LWP 19759]
[New LWP 19760]
[New LWP 19761]
[New LWP 19762]
[New LWP 19764]
[New LWP 19765]
[New LWP 19766]
[New LWP 19767]
[New LWP 19768]
[New LWP 19769]
[New LWP 19770]
[New LWP 19771]
[New LWP 19772]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f5e91e4ad50 in ?? ()
Id Target Id Frame
* 1 LWP 19638 "kudu" 0x00007f5e91e4ad50 in ?? ()
2 LWP 19641 "kudu" 0x00007f5e91e46fb9 in ?? ()
3 LWP 19642 "kudu" 0x00007f5e91e46fb9 in ?? ()
4 LWP 19643 "kudu" 0x00007f5e91e46fb9 in ?? ()
5 LWP 19644 "kernel-watcher-" 0x00007f5e91e46fb9 in ?? ()
6 LWP 19650 "ntp client-1965" 0x00007f5e91e4a9e2 in ?? ()
7 LWP 19651 "file cache-evic" 0x00007f5e91e46fb9 in ?? ()
8 LWP 19652 "sq_acceptor" 0x00007f5e8ab6ccb9 in ?? ()
9 LWP 19655 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
10 LWP 19656 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
11 LWP 19657 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
12 LWP 19658 "rpc reactor-196" 0x00007f5e8ab79a47 in ?? ()
13 LWP 19659 "MaintenanceMgr " 0x00007f5e91e46ad3 in ?? ()
14 LWP 19660 "txn-status-mana" 0x00007f5e91e46fb9 in ?? ()
15 LWP 19662 "collect_and_rem" 0x00007f5e91e46fb9 in ?? ()
16 LWP 19663 "tc-session-exp-" 0x00007f5e91e46fb9 in ?? ()
17 LWP 19664 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
18 LWP 19665 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
19 LWP 19666 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
20 LWP 19667 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
21 LWP 19668 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
22 LWP 19669 "rpc worker-1966" 0x00007f5e91e46ad3 in ?? ()
23 LWP 19670 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
24 LWP 19671 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
25 LWP 19672 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
26 LWP 19673 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
27 LWP 19674 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
28 LWP 19675 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
29 LWP 19676 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
30 LWP 19677 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
31 LWP 19678 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
32 LWP 19679 "rpc worker-1967" 0x00007f5e91e46ad3 in ?? ()
33 LWP 19680 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
34 LWP 19681 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
35 LWP 19682 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
36 LWP 19683 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
37 LWP 19684 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
38 LWP 19685 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
39 LWP 19686 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
40 LWP 19687 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
41 LWP 19688 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
42 LWP 19689 "rpc worker-1968" 0x00007f5e91e46ad3 in ?? ()
43 LWP 19690 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
44 LWP 19691 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
45 LWP 19692 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
46 LWP 19693 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
47 LWP 19694 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
48 LWP 19695 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
49 LWP 19696 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
50 LWP 19697 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
51 LWP 19698 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
52 LWP 19699 "rpc worker-1969" 0x00007f5e91e46ad3 in ?? ()
53 LWP 19700 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
54 LWP 19701 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
55 LWP 19702 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
56 LWP 19703 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
57 LWP 19704 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
58 LWP 19705 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
59 LWP 19706 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
60 LWP 19707 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
61 LWP 19708 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
62 LWP 19709 "rpc worker-1970" 0x00007f5e91e46ad3 in ?? ()
63 LWP 19712 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
64 LWP 19713 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
65 LWP 19714 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
66 LWP 19715 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
67 LWP 19716 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
68 LWP 19717 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
69 LWP 19718 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
70 LWP 19719 "rpc worker-1971" 0x00007f5e91e46ad3 in ?? ()
71 LWP 19720 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
72 LWP 19721 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
73 LWP 19722 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
74 LWP 19723 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
75 LWP 19724 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
76 LWP 19725 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
77 LWP 19726 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
78 LWP 19727 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
79 LWP 19728 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
80 LWP 19729 "rpc worker-1972" 0x00007f5e91e46ad3 in ?? ()
81 LWP 19730 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
82 LWP 19731 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
83 LWP 19732 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
84 LWP 19733 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
85 LWP 19734 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
86 LWP 19735 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
87 LWP 19736 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
88 LWP 19737 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
89 LWP 19738 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
90 LWP 19739 "rpc worker-1973" 0x00007f5e91e46ad3 in ?? ()
91 LWP 19740 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
92 LWP 19741 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
93 LWP 19742 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
94 LWP 19743 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
95 LWP 19744 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
96 LWP 19745 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
97 LWP 19746 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
98 LWP 19747 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
99 LWP 19748 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
100 LWP 19749 "rpc worker-1974" 0x00007f5e91e46ad3 in ?? ()
101 LWP 19750 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
102 LWP 19751 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
103 LWP 19752 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
104 LWP 19753 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
105 LWP 19754 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
106 LWP 19755 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
107 LWP 19756 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
108 LWP 19757 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
109 LWP 19758 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
110 LWP 19759 "rpc worker-1975" 0x00007f5e91e46ad3 in ?? ()
111 LWP 19760 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
112 LWP 19761 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
113 LWP 19762 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
114 LWP 19764 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
115 LWP 19765 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
116 LWP 19766 "rpc worker-1976" 0x00007f5e91e46ad3 in ?? ()
117 LWP 19767 "diag-logger-197" 0x00007f5e91e46fb9 in ?? ()
118 LWP 19768 "result-tracker-" 0x00007f5e91e46fb9 in ?? ()
119 LWP 19769 "excess-log-dele" 0x00007f5e91e46fb9 in ?? ()
120 LWP 19770 "acceptor-19770" 0x00007f5e8ab7b0c7 in ?? ()
121 LWP 19771 "heartbeat-19771" 0x00007f5e91e46fb9 in ?? ()
122 LWP 19772 "maintenance_sch" 0x00007f5e91e46fb9 in ?? ()
Thread 122 (LWP 19772):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e46e966e0 in ?? ()
#2 0x00000000000002cc in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00006160000157a4 in ?? ()
#5 0x00007f5e46e96730 in ?? ()
#6 0x0000000000000599 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f5e46e96750 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f5e8e8d6380 in ?? ()
#12 0x00007f5e46e970d0 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 121 (LWP 19771):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000b3 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f5e476ae5f0 in ?? ()
#6 0x0000000000000167 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f5e476ae610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f5e476ae690 in ?? ()
#12 0x00000febc8ed5cd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 19770):
#0 0x00007f5e8ab7b0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f5e47ed5d90 in ?? ()
#3 0x00007f5e47ed5dc0 in ?? ()
#4 0x00007f5e47ed5ec0 in ?? ()
#5 0x00007f5e47ed5db0 in ?? ()
#6 0x00007f5e47ed5e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f5e8ea01980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19769):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e486eef40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe3466e6d0 in ?? ()
#5 0x00007f5e486eef30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19768):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e48f08100 in ?? ()
#2 0x00000000000000b3 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f5e48f080f0 in ?? ()
#6 0x0000000000000166 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19767):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19766):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000001b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b684c in ?? ()
#4 0x00007f5e49f37e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e49f37eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b6800 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e49f37eb0 in ?? ()
#11 0x00000febc93e6fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19765):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f5e4a74fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e4a74feb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e4a74feb0 in ?? ()
#11 0x00000febc94e9fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19764):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19762):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19761):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19760):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19759):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19758):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19757):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19756):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19755):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19754):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19753):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19752):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19751):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19750):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19749):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19748):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19747):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19746):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19745):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19744):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19743):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19742):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19741):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19740):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19739):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19738):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19737):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19736):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19735):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19734):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19733):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19732):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19731):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19730):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19729):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19728):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19727):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19726):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19725):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000196d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f5e5e308e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5e308eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5e308eb0 in ?? ()
#11 0x00000febcbc611d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 19724):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000203f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f5e5eb20e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5eb20eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5eb20eb0 in ?? ()
#11 0x00000febcbd641d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 19723):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x00000000000005e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a008c in ?? ()
#4 0x00007f5e5f338e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e5f338eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a0040 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e5f338eb0 in ?? ()
#11 0x00000febcbe671d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 19722):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19721):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19720):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19719):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19718):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19717):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19716):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19715):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19714):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19713):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19712):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19709):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19708):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19707):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19706):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19705):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19704):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19703):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f5e67cd1e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e67cd1eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 19702):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19701):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19700):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19699):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19698):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19697):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19696):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19695):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19694):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19693):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19692):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19691):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19690):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19689):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19688):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19687):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19686):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19685):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19684):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19683):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000001c5d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f5e71eb2e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e71eb2eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e71eb2eb0 in ?? ()
#11 0x00000febce3d65d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 19682):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000294 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f5e726cae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e726caeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 19681):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000002297 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f5e72ee2e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e72ee2eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e72ee2eb0 in ?? ()
#11 0x00000febce5dc5d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 19680):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000001846 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008fff8 in ?? ()
#4 0x00007f5e736fae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e736faeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 19679):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x000000000000196e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008d008 in ?? ()
#4 0x00007f5e73f12e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e73f12eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 31 (LWP 19678):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x00000000000020a6 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000086808 in ?? ()
#4 0x00007f5e7472ae90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e7472aeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 30 (LWP 19677):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000002341 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008000c in ?? ()
#4 0x00007f5e74f42e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e74f42eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00007ffc0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e74f42eb0 in ?? ()
#11 0x00000febce9e85d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 29 (LWP 19676):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19675):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19674):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19673):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19672):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19671):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19670):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19669):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19668):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19667):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19666):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19665):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19664):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19663):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e7c0b7cc0 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007f5e7c0b7cb0 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19662):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19660):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e7d94d240 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f5e7d94d230 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19659):
#0 0x00007f5e91e46ad3 in ?? ()
#1 0x0000000000000003 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x00007f5e7e174ba4 in ?? ()
#4 0x00007f5e7e1748b0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5e7e1748d0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00006160000159c0 in ?? ()
#9 0x00007f5e91e46770 in ?? ()
#10 0x00007f5e7e1748d0 in ?? ()
#11 0x00007f5e7e1748d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 12 (LWP 19658):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7e99c320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f5e7e99c310 in ?? ()
#4 0x00007f5e7e99c520 in ?? ()
#5 0x00007f5e7e99c360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f5e7e99c3e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb949ab08c8a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000055000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000fec4fd2b87c in ?? ()
#17 0x00007f5e7e99c3c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7e99c390 in ?? ()
#20 0x3fb949ab08c8a000 in ?? ()
#21 0x000000007e99c3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb949ab08c8a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19657):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7f1b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f5e7f1b3310 in ?? ()
#4 0x00007f5e7f1b3520 in ?? ()
#5 0x00007f5e7f1b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f5e7f1b33e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb960a45abfe000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000026c00000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fec4fe2e67c in ?? ()
#17 0x00007f5e7f1b33c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7f1b3390 in ?? ()
#20 0x3fb960a45abfe000 in ?? ()
#21 0x000000007f1b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb960a45abfe000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19656):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e7f9b4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f5e7f9b4310 in ?? ()
#4 0x00007f5e7f9b4520 in ?? ()
#5 0x00007f5e7f9b4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f5e7f9b43e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb50896d6e28000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000001a000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fec4ff2e87c in ?? ()
#17 0x00007f5e7f9b43c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x00007f5e7f9b4390 in ?? ()
#20 0x3fb50896d6e28000 in ?? ()
#21 0x000000007f9b43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb50896d6e28000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19655):
#0 0x00007f5e8ab79a47 in ?? ()
#1 0x00007f5e819ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f5e819ad310 in ?? ()
#4 0x00007f5e819ad520 in ?? ()
#5 0x00007f5e819ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f5e819ad3e0 in ?? ()
#8 0x00007f5e8d1ec25d in ?? ()
#9 0x3fb9387ddc372000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000003000000 in ?? ()
#14 0x00007f5e00000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fec5032da7c in ?? ()
#17 0x00007f5e819ad3c0 in ?? ()
#18 0x00007f5e8d1f0ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19652):
#0 0x00007f5e8ab6ccb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f5e833b77c8 in ?? ()
#3 0x000060200001ccd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19651):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e82bb60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f5e82bb60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19650):
#0 0x00007f5e91e4a9e2 in ?? ()
#1 0x00007f5e823b3ba0 in ?? ()
#2 0x00007f5e823b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f5e823b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f5e823b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f5e823b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f5e9775dedc in ?? ()
#12 0x00007f5e9774d8d9 in ?? ()
#13 0x00007f5e97751ec6 in ?? ()
#14 0x00007f5e97756900 in ?? ()
#15 0x00007f5e97756729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f5e8ea224fe in ?? ()
#18 0x00007f5e91e406db in ?? ()
#19 0x00000fec5046e684 in ?? ()
#20 0x00007f5e823b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19644):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x00007f5e843b8c80 in ?? ()
#2 0x0000000000000383 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f5e843b8c70 in ?? ()
#6 0x0000000000000706 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19643):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19642):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19641):
#0 0x00007f5e91e46fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f5e85bbc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19638):
#0 0x00007f5e91e4ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:30.931772 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 3 with UUID d33fd96ef7d44ad8805718466ca9447f and pid 18856
************************ BEGIN STACKS **************************
[New LWP 18859]
[New LWP 18860]
[New LWP 18862]
[New LWP 18863]
[New LWP 18870]
[New LWP 18871]
[New LWP 18872]
[New LWP 18875]
[New LWP 18876]
[New LWP 18877]
[New LWP 18878]
[New LWP 18879]
[New LWP 18880]
[New LWP 18881]
[New LWP 18884]
[New LWP 18885]
[New LWP 18886]
[New LWP 18887]
[New LWP 18888]
[New LWP 18889]
[New LWP 18890]
[New LWP 18891]
[New LWP 18892]
[New LWP 18893]
[New LWP 18894]
[New LWP 18895]
[New LWP 18896]
[New LWP 18897]
[New LWP 18898]
[New LWP 18899]
[New LWP 18900]
[New LWP 18901]
[New LWP 18902]
[New LWP 18903]
[New LWP 18904]
[New LWP 18905]
[New LWP 18906]
[New LWP 18907]
[New LWP 18908]
[New LWP 18909]
[New LWP 18910]
[New LWP 18911]
[New LWP 18912]
[New LWP 18913]
[New LWP 18914]
[New LWP 18915]
[New LWP 18916]
[New LWP 18917]
[New LWP 18918]
[New LWP 18919]
[New LWP 18920]
[New LWP 18921]
[New LWP 18922]
[New LWP 18923]
[New LWP 18924]
[New LWP 18925]
[New LWP 18926]
[New LWP 18927]
[New LWP 18928]
[New LWP 18929]
[New LWP 18930]
[New LWP 18931]
[New LWP 18932]
[New LWP 18933]
[New LWP 18934]
[New LWP 18935]
[New LWP 18936]
[New LWP 18937]
[New LWP 18938]
[New LWP 18939]
[New LWP 18940]
[New LWP 18941]
[New LWP 18942]
[New LWP 18943]
[New LWP 18944]
[New LWP 18945]
[New LWP 18946]
[New LWP 18947]
[New LWP 18948]
[New LWP 18949]
[New LWP 18950]
[New LWP 18951]
[New LWP 18952]
[New LWP 18953]
[New LWP 18954]
[New LWP 18955]
[New LWP 18956]
[New LWP 18957]
[New LWP 18959]
[New LWP 18960]
[New LWP 18961]
[New LWP 18962]
[New LWP 18963]
[New LWP 18964]
[New LWP 18967]
[New LWP 18968]
[New LWP 18969]
[New LWP 18970]
[New LWP 18971]
[New LWP 18972]
[New LWP 18973]
[New LWP 18974]
[New LWP 18975]
[New LWP 18976]
[New LWP 18977]
[New LWP 18978]
[New LWP 18979]
[New LWP 18980]
[New LWP 18981]
[New LWP 18982]
[New LWP 18983]
[New LWP 18984]
[New LWP 18985]
[New LWP 18986]
[New LWP 18987]
[New LWP 18988]
[New LWP 18989]
[New LWP 18991]
[New LWP 18992]
[New LWP 18993]
[New LWP 18997]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f148abc7d50 in ?? ()
Id Target Id Frame
* 1 LWP 18856 "kudu" 0x00007f148abc7d50 in ?? ()
2 LWP 18859 "kudu" 0x00007f148abc3fb9 in ?? ()
3 LWP 18860 "kudu" 0x00007f148abc3fb9 in ?? ()
4 LWP 18862 "kudu" 0x00007f148abc3fb9 in ?? ()
5 LWP 18863 "kernel-watcher-" 0x00007f148abc3fb9 in ?? ()
6 LWP 18870 "ntp client-1887" 0x00007f148abc79e2 in ?? ()
7 LWP 18871 "file cache-evic" 0x00007f148abc3fb9 in ?? ()
8 LWP 18872 "sq_acceptor" 0x00007f14838e9cb9 in ?? ()
9 LWP 18875 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
10 LWP 18876 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
11 LWP 18877 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
12 LWP 18878 "rpc reactor-188" 0x00007f14838f6a47 in ?? ()
13 LWP 18879 "MaintenanceMgr " 0x00007f148abc3ad3 in ?? ()
14 LWP 18880 "txn-status-mana" 0x00007f148abc3fb9 in ?? ()
15 LWP 18881 "collect_and_rem" 0x00007f148abc3fb9 in ?? ()
16 LWP 18884 "tc-session-exp-" 0x00007f148abc3fb9 in ?? ()
17 LWP 18885 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
18 LWP 18886 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
19 LWP 18887 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
20 LWP 18888 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
21 LWP 18889 "rpc worker-1888" 0x00007f148abc3ad3 in ?? ()
22 LWP 18890 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
23 LWP 18891 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
24 LWP 18892 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
25 LWP 18893 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
26 LWP 18894 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
27 LWP 18895 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
28 LWP 18896 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
29 LWP 18897 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
30 LWP 18898 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
31 LWP 18899 "rpc worker-1889" 0x00007f148abc3ad3 in ?? ()
32 LWP 18900 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
33 LWP 18901 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
34 LWP 18902 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
35 LWP 18903 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
36 LWP 18904 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
37 LWP 18905 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
38 LWP 18906 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
39 LWP 18907 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
40 LWP 18908 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
41 LWP 18909 "rpc worker-1890" 0x00007f148abc3ad3 in ?? ()
42 LWP 18910 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
43 LWP 18911 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
44 LWP 18912 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
45 LWP 18913 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
46 LWP 18914 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
47 LWP 18915 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
48 LWP 18916 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
49 LWP 18917 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
50 LWP 18918 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
51 LWP 18919 "rpc worker-1891" 0x00007f148abc3ad3 in ?? ()
52 LWP 18920 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
53 LWP 18921 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
54 LWP 18922 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
55 LWP 18923 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
56 LWP 18924 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
57 LWP 18925 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
58 LWP 18926 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
59 LWP 18927 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
60 LWP 18928 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
61 LWP 18929 "rpc worker-1892" 0x00007f148abc3ad3 in ?? ()
62 LWP 18930 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
63 LWP 18931 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
64 LWP 18932 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
65 LWP 18933 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
66 LWP 18934 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
67 LWP 18935 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
68 LWP 18936 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
69 LWP 18937 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
70 LWP 18938 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
71 LWP 18939 "rpc worker-1893" 0x00007f148abc3ad3 in ?? ()
72 LWP 18940 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
73 LWP 18941 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
74 LWP 18942 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
75 LWP 18943 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
76 LWP 18944 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
77 LWP 18945 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
78 LWP 18946 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
79 LWP 18947 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
80 LWP 18948 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
81 LWP 18949 "rpc worker-1894" 0x00007f148abc3ad3 in ?? ()
82 LWP 18950 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
83 LWP 18951 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
84 LWP 18952 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
85 LWP 18953 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
86 LWP 18954 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
87 LWP 18955 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
88 LWP 18956 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
89 LWP 18957 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
90 LWP 18959 "rpc worker-1895" 0x00007f148abc3ad3 in ?? ()
91 LWP 18960 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
92 LWP 18961 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
93 LWP 18962 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
94 LWP 18963 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
95 LWP 18964 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
96 LWP 18967 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
97 LWP 18968 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
98 LWP 18969 "rpc worker-1896" 0x00007f148abc3ad3 in ?? ()
99 LWP 18970 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
100 LWP 18971 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
101 LWP 18972 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
102 LWP 18973 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
103 LWP 18974 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
104 LWP 18975 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
105 LWP 18976 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
106 LWP 18977 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
107 LWP 18978 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
108 LWP 18979 "rpc worker-1897" 0x00007f148abc3ad3 in ?? ()
109 LWP 18980 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
110 LWP 18981 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
111 LWP 18982 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
112 LWP 18983 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
113 LWP 18984 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
114 LWP 18985 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
115 LWP 18986 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
116 LWP 18987 "rpc worker-1898" 0x00007f148abc3ad3 in ?? ()
117 LWP 18988 "diag-logger-189" 0x00007f148abc3fb9 in ?? ()
118 LWP 18989 "result-tracker-" 0x00007f148abc3fb9 in ?? ()
119 LWP 18991 "excess-log-dele" 0x00007f148abc3fb9 in ?? ()
120 LWP 18992 "acceptor-18992" 0x00007f14838f80c7 in ?? ()
121 LWP 18993 "heartbeat-18993" 0x00007f148abc3fb9 in ?? ()
122 LWP 18997 "maintenance_sch" 0x00007f148abc3fb9 in ?? ()
Thread 122 (LWP 18997):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f144046b6e0 in ?? ()
#2 0x0000000000000326 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f144046b730 in ?? ()
#6 0x000000000000064c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18993):
#0 0x00007f148abc3fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f1440c835f0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18992):
#0 0x00007f14838f80c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f14414aad90 in ?? ()
#3 0x00007f14414aadc0 in ?? ()
#4 0x00007f14414aaec0 in ?? ()
#5 0x00007f14414aadb0 in ?? ()
#6 0x00007f14414aae20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f148777e980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18991):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f1441cc4f40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffff38c78f0 in ?? ()
#5 0x00007f1441cc4f30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18989):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14424dd100 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f14424dd0f0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18988):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18987):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x000000000000001f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f144350ce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f144350ceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f144350ceb0 in ?? ()
#11 0x00000fe2886a19d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 18986):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f1443d24e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1443d24eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f1443d24eb0 in ?? ()
#11 0x00000fe2887a49d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 18985):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18984):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18983):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18982):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18981):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18980):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18979):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18978):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18977):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18976):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18975):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18974):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18973):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18972):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18971):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18970):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18969):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18968):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18967):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18964):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18963):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18962):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18961):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18960):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18959):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18957):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18956):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18955):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18954):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18953):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18952):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18951):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18950):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18949):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18948):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18947):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18946):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18945):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18944):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f14578dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f14578ddeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f14578ddeb0 in ?? ()
#11 0x00000fe28af1bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18943):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 18942):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18941):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18940):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18939):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18938):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18937):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18936):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18935):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18934):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18933):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18932):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18931):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18930):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18929):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18928):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18927):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18926):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18925):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18924):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f1461abee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1461abeeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f1461abeeb0 in ?? ()
#11 0x00000fe28c357dd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18923):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18922):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18921):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18920):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18919):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18918):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18917):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18916):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18915):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18914):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18913):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18912):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18911):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18910):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18909):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18908):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18907):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18906):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18905):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18904):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f146bc99e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f146bc99eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f148abc3770 in ?? ()
#10 0x00007f146bc99eb0 in ?? ()
#11 0x00000fe28d7933d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18903):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18902):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18901):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18900):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18899):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18898):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18897):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18896):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18895):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18894):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18893):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18892):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18891):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18890):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18889):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18888):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18887):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18886):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18885):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18884):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14754c2cc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f14754c2cb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18881):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18880):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f14764c4240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f14764c4230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18879):
#0 0x00007f148abc3ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18878):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f14774c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f14774c7310 in ?? ()
#4 0x00007f14774c7520 in ?? ()
#5 0x00007f14774c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f14774c73e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb95fd068734000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000024000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f839e in ?? ()
#16 0x00000fe30ee90e7c in ?? ()
#17 0x00007f14774c73c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f14774c7390 in ?? ()
#20 0x3fb95fd068734000 in ?? ()
#21 0x00000000774c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb95fd068734000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18877):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f147a0b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f147a0b3310 in ?? ()
#4 0x00007f147a0b3520 in ?? ()
#5 0x00007f147a0b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f147a0b33e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb96bdd088c2000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000e6000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe30f40e67c in ?? ()
#17 0x00007f147a0b33c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f147a0b3390 in ?? ()
#20 0x3fb96bdd088c2000 in ?? ()
#21 0x000000007a0b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb96bdd088c2000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18876):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f1477cc8320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f1477cc8310 in ?? ()
#4 0x00007f1477cc8520 in ?? ()
#5 0x00007f1477cc8360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f1477cc83e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb9799b07f1c000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000c7800000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe30ef9107c in ?? ()
#17 0x00007f1477cc83c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x00007f1477cc8390 in ?? ()
#20 0x3fb9799b07f1c000 in ?? ()
#21 0x0000000077cc83e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb9799b07f1c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18875):
#0 0x00007f14838f6a47 in ?? ()
#1 0x00007f14784c9320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f14784c9310 in ?? ()
#4 0x00007f14784c9520 in ?? ()
#5 0x00007f14784c9360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f14784c93e0 in ?? ()
#8 0x00007f1485f6925d in ?? ()
#9 0x3fb952c959604000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000b1000000 in ?? ()
#14 0x00007f1400000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000fe30f09127c in ?? ()
#17 0x00007f14784c93c0 in ?? ()
#18 0x00007f1485f6dba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18872):
#0 0x00007f14838e9cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f147c0b77c8 in ?? ()
#3 0x000060200001dc90 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18871):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f147b8b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f147b8b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18870):
#0 0x00007f148abc79e2 in ?? ()
#1 0x00007f147a8b2ba0 in ?? ()
#2 0x00007f147a8b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f147a8b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f147a8b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f147a8b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f14904daedc in ?? ()
#12 0x00007f14904ca8d9 in ?? ()
#13 0x00007f14904ceec6 in ?? ()
#14 0x00007f14904d3900 in ?? ()
#15 0x00007f14904d3729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f148779f4fe in ?? ()
#18 0x00007f148abbd6db in ?? ()
#19 0x00000fe30f50e484 in ?? ()
#20 0x00007f147a8b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18863):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x00007f147d0b8c80 in ?? ()
#2 0x00000000000003f5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f147d0b8c70 in ?? ()
#6 0x00000000000007ea in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18862):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18860):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18859):
#0 0x00007f148abc3fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f147e8bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18856):
#0 0x00007f148abc7d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:31.823567 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 4 with UUID 4f219d2e8b954461973a8ca8680a6383 and pid 20075
************************ BEGIN STACKS **************************
[New LWP 20078]
[New LWP 20079]
[New LWP 20080]
[New LWP 20081]
[New LWP 20087]
[New LWP 20088]
[New LWP 20089]
[New LWP 20092]
[New LWP 20093]
[New LWP 20094]
[New LWP 20095]
[New LWP 20096]
[New LWP 20097]
[New LWP 20099]
[New LWP 20100]
[New LWP 20101]
[New LWP 20102]
[New LWP 20103]
[New LWP 20104]
[New LWP 20105]
[New LWP 20106]
[New LWP 20107]
[New LWP 20108]
[New LWP 20109]
[New LWP 20110]
[New LWP 20111]
[New LWP 20112]
[New LWP 20113]
[New LWP 20114]
[New LWP 20115]
[New LWP 20116]
[New LWP 20117]
[New LWP 20118]
[New LWP 20119]
[New LWP 20120]
[New LWP 20121]
[New LWP 20122]
[New LWP 20123]
[New LWP 20124]
[New LWP 20125]
[New LWP 20126]
[New LWP 20127]
[New LWP 20128]
[New LWP 20129]
[New LWP 20130]
[New LWP 20131]
[New LWP 20132]
[New LWP 20133]
[New LWP 20134]
[New LWP 20135]
[New LWP 20136]
[New LWP 20137]
[New LWP 20139]
[New LWP 20141]
[New LWP 20142]
[New LWP 20143]
[New LWP 20144]
[New LWP 20146]
[New LWP 20147]
[New LWP 20148]
[New LWP 20149]
[New LWP 20150]
[New LWP 20151]
[New LWP 20152]
[New LWP 20153]
[New LWP 20154]
[New LWP 20155]
[New LWP 20156]
[New LWP 20157]
[New LWP 20158]
[New LWP 20159]
[New LWP 20160]
[New LWP 20161]
[New LWP 20162]
[New LWP 20163]
[New LWP 20164]
[New LWP 20165]
[New LWP 20166]
[New LWP 20167]
[New LWP 20168]
[New LWP 20169]
[New LWP 20170]
[New LWP 20171]
[New LWP 20172]
[New LWP 20173]
[New LWP 20174]
[New LWP 20175]
[New LWP 20176]
[New LWP 20178]
[New LWP 20179]
[New LWP 20180]
[New LWP 20181]
[New LWP 20182]
[New LWP 20183]
[New LWP 20184]
[New LWP 20185]
[New LWP 20186]
[New LWP 20187]
[New LWP 20188]
[New LWP 20189]
[New LWP 20190]
[New LWP 20191]
[New LWP 20192]
[New LWP 20193]
[New LWP 20194]
[New LWP 20195]
[New LWP 20196]
[New LWP 20197]
[New LWP 20198]
[New LWP 20199]
[New LWP 20200]
[New LWP 20201]
[New LWP 20202]
[New LWP 20203]
[New LWP 20204]
[New LWP 20205]
[New LWP 20206]
[New LWP 20207]
[New LWP 20209]
[New LWP 20210]
[New LWP 20211]
[New LWP 21677]
[New LWP 21695]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007ff5b495dd50 in ?? ()
Id Target Id Frame
* 1 LWP 20075 "kudu" 0x00007ff5b495dd50 in ?? ()
2 LWP 20078 "kudu" 0x00007ff5b4959fb9 in ?? ()
3 LWP 20079 "kudu" 0x00007ff5b4959fb9 in ?? ()
4 LWP 20080 "kudu" 0x00007ff5b4959fb9 in ?? ()
5 LWP 20081 "kernel-watcher-" 0x00007ff5b4959fb9 in ?? ()
6 LWP 20087 "ntp client-2008" 0x00007ff5b495d9e2 in ?? ()
7 LWP 20088 "file cache-evic" 0x00007ff5b4959fb9 in ?? ()
8 LWP 20089 "sq_acceptor" 0x00007ff5ad67fcb9 in ?? ()
9 LWP 20092 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
10 LWP 20093 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
11 LWP 20094 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
12 LWP 20095 "rpc reactor-200" 0x00007ff5ad68ca47 in ?? ()
13 LWP 20096 "MaintenanceMgr " 0x000000000096b06f in __sanitizer::theDepot ()
14 LWP 20097 "txn-status-mana" 0x00007ff5b4959fb9 in ?? ()
15 LWP 20099 "collect_and_rem" 0x00007ff5b4959fb9 in ?? ()
16 LWP 20100 "tc-session-exp-" 0x00007ff5b4959fb9 in ?? ()
17 LWP 20101 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
18 LWP 20102 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
19 LWP 20103 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
20 LWP 20104 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
21 LWP 20105 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
22 LWP 20106 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
23 LWP 20107 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
24 LWP 20108 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
25 LWP 20109 "rpc worker-2010" 0x00007ff5b4959ad3 in ?? ()
26 LWP 20110 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
27 LWP 20111 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
28 LWP 20112 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
29 LWP 20113 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
30 LWP 20114 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
31 LWP 20115 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
32 LWP 20116 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
33 LWP 20117 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
34 LWP 20118 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
35 LWP 20119 "rpc worker-2011" 0x00007ff5b4959ad3 in ?? ()
36 LWP 20120 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
37 LWP 20121 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
38 LWP 20122 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
39 LWP 20123 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
40 LWP 20124 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
41 LWP 20125 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
42 LWP 20126 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
43 LWP 20127 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
44 LWP 20128 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
45 LWP 20129 "rpc worker-2012" 0x00007ff5b4959ad3 in ?? ()
46 LWP 20130 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
47 LWP 20131 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
48 LWP 20132 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
49 LWP 20133 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
50 LWP 20134 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
51 LWP 20135 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
52 LWP 20136 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
53 LWP 20137 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
54 LWP 20139 "rpc worker-2013" 0x00007ff5b4959ad3 in ?? ()
55 LWP 20141 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
56 LWP 20142 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
57 LWP 20143 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
58 LWP 20144 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
59 LWP 20146 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
60 LWP 20147 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
61 LWP 20148 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
62 LWP 20149 "rpc worker-2014" 0x00007ff5b4959ad3 in ?? ()
63 LWP 20150 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
64 LWP 20151 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
65 LWP 20152 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
66 LWP 20153 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
67 LWP 20154 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
68 LWP 20155 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
69 LWP 20156 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
70 LWP 20157 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
71 LWP 20158 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
72 LWP 20159 "rpc worker-2015" 0x00007ff5b4959ad3 in ?? ()
73 LWP 20160 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
74 LWP 20161 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
75 LWP 20162 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
76 LWP 20163 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
77 LWP 20164 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
78 LWP 20165 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
79 LWP 20166 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
80 LWP 20167 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
81 LWP 20168 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
82 LWP 20169 "rpc worker-2016" 0x00007ff5b4959ad3 in ?? ()
83 LWP 20170 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
84 LWP 20171 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
85 LWP 20172 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
86 LWP 20173 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
87 LWP 20174 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
88 LWP 20175 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
89 LWP 20176 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
90 LWP 20178 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
91 LWP 20179 "rpc worker-2017" 0x00007ff5b4959ad3 in ?? ()
92 LWP 20180 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
93 LWP 20181 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
94 LWP 20182 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
95 LWP 20183 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
96 LWP 20184 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
97 LWP 20185 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
98 LWP 20186 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
99 LWP 20187 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
100 LWP 20188 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
101 LWP 20189 "rpc worker-2018" 0x00007ff5b4959ad3 in ?? ()
102 LWP 20190 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
103 LWP 20191 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
104 LWP 20192 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
105 LWP 20193 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
106 LWP 20194 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
107 LWP 20195 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
108 LWP 20196 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
109 LWP 20197 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
110 LWP 20198 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
111 LWP 20199 "rpc worker-2019" 0x00007ff5b4959ad3 in ?? ()
112 LWP 20200 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
113 LWP 20201 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
114 LWP 20202 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
115 LWP 20203 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
116 LWP 20204 "rpc worker-2020" 0x00007ff5b4959ad3 in ?? ()
117 LWP 20205 "diag-logger-202" 0x00007ff5b4959fb9 in ?? ()
118 LWP 20206 "result-tracker-" 0x00007ff5b4959fb9 in ?? ()
119 LWP 20207 "excess-log-dele" 0x00007ff5b4959fb9 in ?? ()
120 LWP 20209 "acceptor-20209" 0x00007ff5ad68e0c7 in ?? ()
121 LWP 20210 "heartbeat-20210" 0x00007ff5b4959fb9 in ?? ()
122 LWP 20211 "maintenance_sch" 0x00007ff5b4959fb9 in ?? ()
123 LWP 21677 "raft [worker]-2" 0x00007ff5b4959fb9 in ?? ()
124 LWP 21695 "raft [worker]-2" 0x00007ff5b4959fb9 in ?? ()
Thread 124 (LWP 21695):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007ff55518fba4 in ?? ()
#5 0x00007ff55518f830 in ?? ()
#6 0x000000000000000f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ff55518f850 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 123 (LWP 21677):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ff555990ba0 in ?? ()
#5 0x00007ff555990830 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 20211):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56902e6e0 in ?? ()
#2 0x0000000000000257 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007ff56902e730 in ?? ()
#6 0x00000000000004ae in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 20210):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000099 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007ff5698465f0 in ?? ()
#6 0x0000000000000133 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ff569846610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007ff569846690 in ?? ()
#12 0x00000ffead308cd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 20209):
#0 0x00007ff5ad68e0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007ff56a05dd90 in ?? ()
#3 0x00007ff56a05ddc0 in ?? ()
#4 0x00007ff56a05dec0 in ?? ()
#5 0x00007ff56a05ddb0 in ?? ()
#6 0x00007ff56a05de20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007ff5b1514980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 20207):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56a876f40 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffff4670c90 in ?? ()
#5 0x00007ff56a876f30 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 20206):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff56b08f100 in ?? ()
#2 0x0000000000000096 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007ff56b08f0f0 in ?? ()
#6 0x000000000000012c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 20205):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 20204):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000000f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00034688c in ?? ()
#4 0x00007ff56c0cfe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff56c0cfeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000346840 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff56c0cfeb0 in ?? ()
#11 0x00000ffead819fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 20203):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0003367cc in ?? ()
#4 0x00007ff56c8e7e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff56c8e7eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000336780 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff56c8e7eb0 in ?? ()
#11 0x00000ffead91cfd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 20202):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 20201):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 20200):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 20199):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 20198):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 20197):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 20196):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 20195):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 20194):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 20193):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 20192):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 20191):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 20190):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 20189):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 20188):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 20187):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 20186):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 20185):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 20184):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 20183):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 20182):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 20181):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 20180):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 20179):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 20178):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 20176):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 20175):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 20174):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 20173):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 20172):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 20171):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 20170):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 20169):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 20168):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 20167):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 20166):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 20165):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 20164):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 20163):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000003c2 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007ff580604e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff580604eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 20162):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000002e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007ff580e1be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff580e1beb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff580e1beb0 in ?? ()
#11 0x00000ffeb01c37d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 20161):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 20160):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 20159):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 20158):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 20157):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 20156):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 20155):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 20154):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 20153):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 20152):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 20151):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 20150):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 20149):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 20148):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 20147):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 20146):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 20144):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 20143):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 20142):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000000d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007ff58a7e4e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff58a7e4eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff58a7e4eb0 in ?? ()
#11 0x00000ffeb14fc9d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 20141):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 20139):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 20137):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 20136):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 20135):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 20134):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 20133):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 20132):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 20131):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 20130):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 20129):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 20128):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 20127):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 20126):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 20125):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 20124):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 20123):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 20122):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 20121):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 20120):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x00000000000006c5 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007ff5949c5e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5949c5eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff5949c5eb0 in ?? ()
#11 0x00000ffeb2938bd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 20119):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x000000000000097e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007ff5951dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5951ddeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 20118):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000a29 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007ff5959f5e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ff5959f5eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007ff5b4959770 in ?? ()
#10 0x00007ff5959f5eb0 in ?? ()
#11 0x00000ffeb2b3ebd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 20117):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20116):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20115):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20114):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20113):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20112):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20111):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20110):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20109):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20108):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20107):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20106):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20105):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20104):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20103):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20102):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20101):
#0 0x00007ff5b4959ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20100):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff59ebc6cc0 in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007ff59ebc6cb0 in ?? ()
#6 0x000000000000001e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20099):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20097):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a045b240 in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007ff5a045b230 in ?? ()
#6 0x000000000000001e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20096):
#0 0x000000000096b06f in __sanitizer::theDepot ()
#1 0x00007ff5a0c7ea20 in ?? ()
#2 0x0000000000a5d66c in __sanitizer::theDepot ()
#3 0x0000000041b58ab3 in ?? ()
I20250624 07:58:32.829517 21699 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.829730 21699 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.830974 21699 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369), 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73:38265)
I20250624 07:58:32.852766 21043 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "6cbd668c726a4b1298e789270a0031e2" is_pre_election: true
I20250624 07:58:32.891990 21703 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.892247 21703 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.893267 21703 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:32.894335 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" is_pre_election: true
I20250624 07:58:32.894752 19723 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 2.
I20250624 07:58:32.895587 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:58:32.896028 21703 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250624 07:58:32.896157 21703 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4f219d2e8b954461973a8ca8680a6383)
I20250624 07:58:32.896262 21703 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:32.901054 21703 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.901824 21703 leader_election.cc:290] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 election: Requested vote from peers 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189), 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369)
I20250624 07:58:32.902967 19723 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "439cf2177bdb4c19aa8449b6cbd440a7"
I20250624 07:58:32.903340 19723 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:32.906291 19723 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 3.
I20250624 07:58:32.907157 20974 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7, 6cbd668c726a4b1298e789270a0031e2; no voters:
I20250624 07:58:32.907734 21703 raft_consensus.cc:2802] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 FOLLOWER]: Leader election won for term 3
I20250624 07:58:32.908365 21703 raft_consensus.cc:695] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 LEADER]: Becoming Leader. State: Replica: 6cbd668c726a4b1298e789270a0031e2, State: Running, Role: LEADER
I20250624 07:58:32.909085 21703 consensus_queue.cc:237] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15871, Committed index: 15871, Last appended: 2.15871, Last appended by leader: 15871, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:32.914839 17300 catalog_manager.cc:5582] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 reported cstate change: term changed from 2 to 3, leader changed from 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69) to 6cbd668c726a4b1298e789270a0031e2 (127.16.221.73). New cstate: current_term: 3 leader_uuid: "6cbd668c726a4b1298e789270a0031e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } health_report { overall_health: HEALTHY } } }
I20250624 07:58:33.006711 19723 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Refusing update from remote peer 6cbd668c726a4b1298e789270a0031e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 15871. Preceding OpId from leader: term: 3 index: 15872. (index mismatch)
I20250624 07:58:33.007638 21703 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15872, Last known committed idx: 15871, Time since last communication: 0.000s
#4 0x00000000004518ac in kudu::itest::MaintenanceModeRF3ITest_TestMaintenanceModeDoesntObstructMove_Test::TestBody()::$_2::operator()() const (this=<optimized out>) at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:353
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Thread 12 (LWP 20095):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a14aa320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007ff5a14aa310 in ?? ()
#4 0x00007ff5a14aa520 in ?? ()
#5 0x00007ff5a14aa360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007ff5a14aa3e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fab99f70c918000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000d6000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fff3428d47c in ?? ()
#17 0x00007ff5a14aa3c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a14aa390 in ?? ()
#20 0x3fab99f70c918000 in ?? ()
#21 0x00000000a14aa3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fab99f70c918000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20094):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a1cb3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007ff5a1cb3310 in ?? ()
#4 0x00007ff5a1cb3520 in ?? ()
#5 0x00007ff5a1cb3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007ff5a1cb33e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb982c1b35b6000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000018000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fff3438e67c in ?? ()
#17 0x00007ff5a1cb33c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a1cb3390 in ?? ()
#20 0x3fb982c1b35b6000 in ?? ()
#21 0x00000000a1cb33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb982c1b35b6000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20093):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a24b4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007ff5a24b4310 in ?? ()
#4 0x00007ff5a24b4520 in ?? ()
#5 0x00007ff5a24b4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007ff5a24b43e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb0bc0deb7e8000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000f0000000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a3 in ?? ()
#16 0x00000fff3448e87c in ?? ()
#17 0x00007ff5a24b43c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x00007ff5a24b4390 in ?? ()
#20 0x3fb0bc0deb7e8000 in ?? ()
#21 0x00000000a24b43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb0bc0deb7e8000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20092):
#0 0x00007ff5ad68ca47 in ?? ()
#1 0x00007ff5a44ad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007ff5a44ad310 in ?? ()
#4 0x00007ff5a44ad520 in ?? ()
#5 0x00007ff5a44ad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007ff5a44ad3e0 in ?? ()
#8 0x00007ff5afcff25d in ?? ()
#9 0x3fb968fc768c2000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000001d800000 in ?? ()
#14 0x00007ff500000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fff3488da7c in ?? ()
#17 0x00007ff5a44ad3c0 in ?? ()
#18 0x00007ff5afd03ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20089):
#0 0x00007ff5ad67fcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007ff5a5eb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20088):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a56b60c0 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007ff5a56b60b0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20087):
#0 0x00007ff5b495d9e2 in ?? ()
#1 0x00007ff5a4eb3ba0 in ?? ()
#2 0x00007ff5a4eb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007ff5a4eb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007ff5a4eb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007ff5a4eb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007ff5ba270edc in ?? ()
#12 0x00007ff5ba2608d9 in ?? ()
#13 0x00007ff5ba264ec6 in ?? ()
#14 0x00007ff5ba269900 in ?? ()
#15 0x00007ff5ba269729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007ff5b15354fe in ?? ()
#18 0x00007ff5b49536db in ?? ()
#19 0x00000fff349ce684 in ?? ()
#20 0x00007ff5a4eb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20081):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x00007ff5a6eb8c80 in ?? ()
#2 0x00000000000002f2 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007ff5a6eb8c70 in ?? ()
#6 0x00000000000005e4 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20080):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20079):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20078):
#0 0x00007ff5b4959fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007ff5a86bc450 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20075):
#0 0x00007ff5b495dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
I20250624 07:58:33.041600 19723 raft_consensus.cc:1238] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Rejecting Update request from peer 4f219d2e8b954461973a8ca8680a6383 for earlier term 2. Current term is 3. Ops: []
I20250624 07:58:33.042953 21695 consensus_queue.cc:1046] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 }, Status: INVALID_TERM, Last received: 2.15871, Next index: 15872, Last known committed idx: 15872, Time since last communication: 0.000s
************************* END STACKS ***************************
I20250624 07:58:33.060803 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 5 with UUID bede5abcb6174794bd2283843157bcef and pid 18718
I20250624 07:58:33.072947 21714 raft_consensus.cc:3053] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 LEADER]: Stepping down as leader of term 2
I20250624 07:58:33.073102 21714 raft_consensus.cc:738] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4f219d2e8b954461973a8ca8680a6383, State: Running, Role: LEADER
I20250624 07:58:33.073462 21714 consensus_queue.cc:260] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15871, Committed index: 15871, Last appended: 2.15871, Last appended by leader: 15871, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:33.073765 21714 logging.cc:424] LogThrottler TrackedPeer Lag: suppressed but not reported on 4 messages since previous log ~11 seconds ago
I20250624 07:58:33.074223 21714 raft_consensus.cc:3058] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 2 FOLLOWER]: Advancing to term 3
I20250624 07:58:33.075183 20162 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:33.075238 20163 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "6cbd668c726a4b1298e789270a0031e2" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383"
I20250624 07:58:33.077229 21043 raft_consensus.cc:1238] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [term 3 LEADER]: Rejecting Update request from peer 4f219d2e8b954461973a8ca8680a6383 for earlier term 2. Current term is 3. Ops: []
I20250624 07:58:33.085366 20162 raft_consensus.cc:2466] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cbd668c726a4b1298e789270a0031e2 in term 3.
I20250624 07:58:33.086609 20163 raft_consensus.cc:1273] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 4f219d2e8b954461973a8ca8680a6383 [term 3 FOLLOWER]: Refusing update from remote peer 6cbd668c726a4b1298e789270a0031e2: Log matching property violated. Preceding OpId in replica: term: 2 index: 15871. Preceding OpId from leader: term: 3 index: 15872. (index mismatch)
I20250624 07:58:33.087695 21704 consensus_queue.cc:1035] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15872, Last known committed idx: 15871, Time since last communication: 0.000s
I20250624 07:58:33.085345 20161 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5ca5ea09d51f4eb390c4433fec8ed9fa" candidate_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" candidate_term: 3 candidate_status { last_received { term: 2 index: 15871 } } ignore_live_leader: false dest_uuid: "4f219d2e8b954461973a8ca8680a6383" is_pre_election: true
I20250624 07:58:33.103713 19658 leader_election.cc:304] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 439cf2177bdb4c19aa8449b6cbd440a7; no voters: 4f219d2e8b954461973a8ca8680a6383, 6cbd668c726a4b1298e789270a0031e2
I20250624 07:58:33.104398 21699 raft_consensus.cc:2747] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
************************ BEGIN STACKS **************************
[New LWP 18721]
[New LWP 18722]
[New LWP 18723]
[New LWP 18724]
[New LWP 18730]
[New LWP 18731]
[New LWP 18732]
[New LWP 18735]
[New LWP 18736]
[New LWP 18737]
[New LWP 18738]
[New LWP 18739]
[New LWP 18740]
[New LWP 18741]
[New LWP 18742]
[New LWP 18743]
[New LWP 18744]
[New LWP 18745]
[New LWP 18746]
[New LWP 18747]
[New LWP 18748]
[New LWP 18749]
[New LWP 18750]
[New LWP 18751]
[New LWP 18752]
[New LWP 18753]
[New LWP 18754]
[New LWP 18755]
[New LWP 18756]
[New LWP 18757]
[New LWP 18758]
[New LWP 18759]
[New LWP 18760]
[New LWP 18761]
[New LWP 18762]
[New LWP 18763]
[New LWP 18764]
[New LWP 18765]
[New LWP 18766]
[New LWP 18767]
[New LWP 18768]
[New LWP 18769]
[New LWP 18770]
[New LWP 18771]
[New LWP 18772]
[New LWP 18773]
[New LWP 18774]
[New LWP 18775]
[New LWP 18776]
[New LWP 18777]
[New LWP 18778]
[New LWP 18779]
[New LWP 18780]
[New LWP 18781]
[New LWP 18782]
[New LWP 18783]
[New LWP 18784]
[New LWP 18785]
[New LWP 18786]
[New LWP 18787]
[New LWP 18788]
[New LWP 18789]
[New LWP 18790]
[New LWP 18791]
[New LWP 18792]
[New LWP 18793]
[New LWP 18794]
[New LWP 18795]
[New LWP 18796]
[New LWP 18797]
[New LWP 18798]
[New LWP 18799]
[New LWP 18800]
[New LWP 18801]
[New LWP 18802]
[New LWP 18803]
[New LWP 18804]
[New LWP 18805]
[New LWP 18806]
[New LWP 18807]
[New LWP 18808]
[New LWP 18809]
[New LWP 18810]
[New LWP 18812]
[New LWP 18813]
[New LWP 18815]
[New LWP 18816]
[New LWP 18817]
[New LWP 18818]
[New LWP 18819]
[New LWP 18820]
[New LWP 18821]
[New LWP 18822]
[New LWP 18823]
[New LWP 18824]
[New LWP 18825]
[New LWP 18826]
[New LWP 18827]
[New LWP 18828]
[New LWP 18829]
[New LWP 18830]
[New LWP 18831]
[New LWP 18832]
[New LWP 18833]
[New LWP 18834]
[New LWP 18835]
[New LWP 18836]
[New LWP 18837]
[New LWP 18838]
[New LWP 18839]
[New LWP 18840]
[New LWP 18841]
[New LWP 18842]
[New LWP 18843]
[New LWP 18844]
[New LWP 18845]
[New LWP 18846]
[New LWP 18847]
[New LWP 18848]
[New LWP 18849]
[New LWP 18850]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f7575afed50 in ?? ()
Id Target Id Frame
* 1 LWP 18718 "kudu" 0x00007f7575afed50 in ?? ()
2 LWP 18721 "kudu" 0x00007f7575afafb9 in ?? ()
3 LWP 18722 "kudu" 0x00007f7575afafb9 in ?? ()
4 LWP 18723 "kudu" 0x00007f7575afafb9 in ?? ()
5 LWP 18724 "kernel-watcher-" 0x00007f7575afafb9 in ?? ()
6 LWP 18730 "ntp client-1873" 0x00007f7575afe9e2 in ?? ()
7 LWP 18731 "file cache-evic" 0x00007f7575afafb9 in ?? ()
8 LWP 18732 "sq_acceptor" 0x00007f756e820cb9 in ?? ()
9 LWP 18735 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
10 LWP 18736 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
11 LWP 18737 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
12 LWP 18738 "rpc reactor-187" 0x00007f756e82da47 in ?? ()
13 LWP 18739 "MaintenanceMgr " 0x00007f7575afaad3 in ?? ()
14 LWP 18740 "txn-status-mana" 0x00007f7575afafb9 in ?? ()
15 LWP 18741 "collect_and_rem" 0x00007f7575afafb9 in ?? ()
16 LWP 18742 "tc-session-exp-" 0x00007f7575afafb9 in ?? ()
17 LWP 18743 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
18 LWP 18744 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
19 LWP 18745 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
20 LWP 18746 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
21 LWP 18747 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
22 LWP 18748 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
23 LWP 18749 "rpc worker-1874" 0x00007f7575afaad3 in ?? ()
24 LWP 18750 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
25 LWP 18751 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
26 LWP 18752 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
27 LWP 18753 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
28 LWP 18754 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
29 LWP 18755 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
30 LWP 18756 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
31 LWP 18757 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
32 LWP 18758 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
33 LWP 18759 "rpc worker-1875" 0x00007f7575afaad3 in ?? ()
34 LWP 18760 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
35 LWP 18761 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
36 LWP 18762 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
37 LWP 18763 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
38 LWP 18764 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
39 LWP 18765 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
40 LWP 18766 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
41 LWP 18767 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
42 LWP 18768 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
43 LWP 18769 "rpc worker-1876" 0x00007f7575afaad3 in ?? ()
44 LWP 18770 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
45 LWP 18771 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
46 LWP 18772 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
47 LWP 18773 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
48 LWP 18774 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
49 LWP 18775 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
50 LWP 18776 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
51 LWP 18777 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
52 LWP 18778 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
53 LWP 18779 "rpc worker-1877" 0x00007f7575afaad3 in ?? ()
54 LWP 18780 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
55 LWP 18781 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
56 LWP 18782 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
57 LWP 18783 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
58 LWP 18784 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
59 LWP 18785 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
60 LWP 18786 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
61 LWP 18787 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
62 LWP 18788 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
63 LWP 18789 "rpc worker-1878" 0x00007f7575afaad3 in ?? ()
64 LWP 18790 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
65 LWP 18791 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
66 LWP 18792 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
67 LWP 18793 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
68 LWP 18794 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
69 LWP 18795 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
70 LWP 18796 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
71 LWP 18797 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
72 LWP 18798 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
73 LWP 18799 "rpc worker-1879" 0x00007f7575afaad3 in ?? ()
74 LWP 18800 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
75 LWP 18801 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
76 LWP 18802 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
77 LWP 18803 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
78 LWP 18804 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
79 LWP 18805 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
80 LWP 18806 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
81 LWP 18807 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
82 LWP 18808 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
83 LWP 18809 "rpc worker-1880" 0x00007f7575afaad3 in ?? ()
84 LWP 18810 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
85 LWP 18812 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
86 LWP 18813 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
87 LWP 18815 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
88 LWP 18816 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
89 LWP 18817 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
90 LWP 18818 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
91 LWP 18819 "rpc worker-1881" 0x00007f7575afaad3 in ?? ()
92 LWP 18820 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
93 LWP 18821 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
94 LWP 18822 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
95 LWP 18823 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
96 LWP 18824 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
97 LWP 18825 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
98 LWP 18826 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
99 LWP 18827 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
100 LWP 18828 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
101 LWP 18829 "rpc worker-1882" 0x00007f7575afaad3 in ?? ()
102 LWP 18830 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
103 LWP 18831 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
104 LWP 18832 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
105 LWP 18833 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
106 LWP 18834 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
107 LWP 18835 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
108 LWP 18836 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
109 LWP 18837 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
110 LWP 18838 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
111 LWP 18839 "rpc worker-1883" 0x00007f7575afaad3 in ?? ()
112 LWP 18840 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
113 LWP 18841 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
114 LWP 18842 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
115 LWP 18843 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
116 LWP 18844 "rpc worker-1884" 0x00007f7575afaad3 in ?? ()
117 LWP 18845 "diag-logger-188" 0x00007f7575afafb9 in ?? ()
118 LWP 18846 "result-tracker-" 0x00007f7575afafb9 in ?? ()
119 LWP 18847 "excess-log-dele" 0x00007f7575afafb9 in ?? ()
120 LWP 18848 "acceptor-18848" 0x00007f756e82f0c7 in ?? ()
121 LWP 18849 "heartbeat-18849" 0x00007f7575afafb9 in ?? ()
122 LWP 18850 "maintenance_sch" 0x00007f7575afafb9 in ?? ()
Thread 122 (LWP 18850):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752b3a26e0 in ?? ()
#2 0x0000000000000338 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f752b3a2730 in ?? ()
#6 0x0000000000000670 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18849):
#0 0x00007f7575afafb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000cd in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f752bbba5f0 in ?? ()
#6 0x000000000000019a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18848):
#0 0x00007f756e82f0c7 in ?? ()
#1 0x0000614000020ca8 in ?? ()
#2 0x00007f752c3e1d90 in ?? ()
#3 0x00007f752c3e1dc0 in ?? ()
#4 0x00007f752c3e1ec0 in ?? ()
#5 0x00007f752c3e1db0 in ?? ()
#6 0x00007f752c3e1e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f75726b5980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18847):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752cbfbf40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffee7883d50 in ?? ()
#5 0x00007f752cbfbf30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18846):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f752d414100 in ?? ()
#2 0x00000000000000ce in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f752d4140f0 in ?? ()
#6 0x000000000000019c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18845):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18844):
#0 0x00007f7575afaad3 in ?? ()
#1 0x000000000000002c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f752e443e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f752e443eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 18843):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000018 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f752ec5be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f752ec5beb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 18842):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18841):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18840):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18839):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18838):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18837):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18836):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18835):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18834):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18833):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18832):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18831):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18830):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18829):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18828):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18827):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18826):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18825):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18824):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18823):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18822):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18821):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18820):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18819):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18818):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18817):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18816):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18815):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18813):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18812):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18810):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18809):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18808):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18807):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18806):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18805):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18804):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18803):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18802):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f7542814e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f7542814eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f7542814eb0 in ?? ()
#11 0x00000feea85029d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18801):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 18800):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18799):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18798):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18797):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18796):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18795):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18794):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18793):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18792):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18791):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18790):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18789):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18788):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18787):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18786):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18785):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18784):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18783):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18782):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f754c1dde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f754c1ddeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f754c1ddeb0 in ?? ()
#11 0x00000feea983bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18781):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18780):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18779):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18778):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18777):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18776):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18775):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18774):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18773):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18772):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18771):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18770):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18769):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18768):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18767):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18766):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18765):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18764):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18763):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18762):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f75563aee90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f75563aeeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f7575afa770 in ?? ()
#10 0x00007f75563aeeb0 in ?? ()
#11 0x00000feeaac75dd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18761):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18760):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18759):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18758):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18757):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18756):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18755):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18754):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18753):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18752):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18751):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18750):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18749):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18748):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18747):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18746):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18745):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18744):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18743):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18742):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75603c2cc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f75603c2cb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18741):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18740):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75613c4240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f75613c4230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18739):
#0 0x00007f7575afaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18738):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f75623c7320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f75623c7310 in ?? ()
#4 0x00007f75623c7520 in ?? ()
#5 0x00007f75623c7360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f75623c73e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb98202d391a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x000000002b000000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a4 in ?? ()
#16 0x00000fef2c470e7c in ?? ()
#17 0x00007f75623c73c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x00007f75623c7390 in ?? ()
#20 0x3fb98202d391a000 in ?? ()
#21 0x00000000623c73e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb98202d391a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18737):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f7562bc8320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f7562bc8310 in ?? ()
#4 0x00007f7562bc8520 in ?? ()
#5 0x00007f7562bc8360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f7562bc83e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb973330202a000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 10 (LWP 18736):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f75633c9320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f75633c9310 in ?? ()
#4 0x00007f75633c9520 in ?? ()
#5 0x00007f75633c9360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f75633c93e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb97161122de000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000cb000000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fef2c67127c in ?? ()
#17 0x00007f75633c93c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x00007f75633c9390 in ?? ()
#20 0x3fb97161122de000 in ?? ()
#21 0x00000000633c93e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb97161122de000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18735):
#0 0x00007f756e82da47 in ?? ()
#1 0x00007f7564fb3320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f7564fb3310 in ?? ()
#4 0x00007f7564fb3520 in ?? ()
#5 0x00007f7564fb3360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f7564fb33e0 in ?? ()
#8 0x00007f7570ea025d in ?? ()
#9 0x3fb987139ac58000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000029c00000 in ?? ()
#14 0x00007f7500000000 in ?? ()
#15 0x41da1696258f83a9 in ?? ()
#16 0x00000fef2c9ee67c in ?? ()
#17 0x00007f7564fb33c0 in ?? ()
#18 0x00007f7570ea4ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18732):
#0 0x00007f756e820cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f7566fb77c8 in ?? ()
#3 0x000060200001ccd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18731):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f75667b60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f75667b60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18730):
#0 0x00007f7575afe9e2 in ?? ()
#1 0x00007f75657b2ba0 in ?? ()
#2 0x00007f75657b2c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f75657b2bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f75657b2c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f75657b2c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f757b411edc in ?? ()
#12 0x00007f757b4018d9 in ?? ()
#13 0x00007f757b405ec6 in ?? ()
#14 0x00007f757b40a900 in ?? ()
#15 0x00007f757b40a729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f75726d64fe in ?? ()
#18 0x00007f7575af46db in ?? ()
#19 0x00000fef2caee484 in ?? ()
#20 0x00007f75657b2440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18724):
#0 0x00007f7575afafb9 in ?? ()
#1 0x00007f7567fb8c80 in ?? ()
#2 0x000000000000040a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f7567fb8c70 in ?? ()
#6 0x0000000000000814 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18723):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18722):
#0 0x00007f7575afafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18721):
#0 0x00007f7575afafb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f75697bc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18718):
#0 0x00007f7575afed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:33.994892 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 6 with UUID 1fe60cc8e38948b694506041d7734713 and pid 19000
************************ BEGIN STACKS **************************
[New LWP 19002]
[New LWP 19003]
[New LWP 19004]
[New LWP 19005]
[New LWP 19011]
[New LWP 19012]
[New LWP 19013]
[New LWP 19016]
[New LWP 19017]
[New LWP 19018]
[New LWP 19019]
[New LWP 19020]
[New LWP 19021]
[New LWP 19022]
[New LWP 19023]
[New LWP 19024]
[New LWP 19025]
[New LWP 19026]
[New LWP 19027]
[New LWP 19028]
[New LWP 19029]
[New LWP 19030]
[New LWP 19031]
[New LWP 19032]
[New LWP 19033]
[New LWP 19034]
[New LWP 19035]
[New LWP 19036]
[New LWP 19037]
[New LWP 19038]
[New LWP 19039]
[New LWP 19040]
[New LWP 19041]
[New LWP 19042]
[New LWP 19043]
[New LWP 19044]
[New LWP 19045]
[New LWP 19046]
[New LWP 19047]
[New LWP 19048]
[New LWP 19049]
[New LWP 19050]
[New LWP 19051]
[New LWP 19052]
[New LWP 19053]
[New LWP 19054]
[New LWP 19055]
[New LWP 19056]
[New LWP 19057]
[New LWP 19058]
[New LWP 19059]
[New LWP 19060]
[New LWP 19061]
[New LWP 19062]
[New LWP 19063]
[New LWP 19064]
[New LWP 19065]
[New LWP 19066]
[New LWP 19067]
[New LWP 19068]
[New LWP 19069]
[New LWP 19070]
[New LWP 19071]
[New LWP 19072]
[New LWP 19073]
[New LWP 19074]
[New LWP 19075]
[New LWP 19076]
[New LWP 19077]
[New LWP 19078]
[New LWP 19079]
[New LWP 19080]
[New LWP 19081]
[New LWP 19082]
[New LWP 19083]
[New LWP 19084]
[New LWP 19085]
[New LWP 19086]
[New LWP 19087]
[New LWP 19088]
[New LWP 19089]
[New LWP 19090]
[New LWP 19091]
[New LWP 19092]
[New LWP 19093]
[New LWP 19094]
[New LWP 19095]
[New LWP 19096]
[New LWP 19097]
[New LWP 19098]
[New LWP 19099]
[New LWP 19100]
[New LWP 19101]
[New LWP 19102]
[New LWP 19103]
[New LWP 19104]
[New LWP 19106]
[New LWP 19107]
[New LWP 19108]
[New LWP 19109]
[New LWP 19110]
[New LWP 19111]
[New LWP 19112]
[New LWP 19113]
[New LWP 19114]
[New LWP 19115]
[New LWP 19116]
[New LWP 19117]
[New LWP 19118]
[New LWP 19119]
[New LWP 19120]
[New LWP 19121]
[New LWP 19122]
[New LWP 19123]
[New LWP 19124]
[New LWP 19125]
[New LWP 19126]
[New LWP 19127]
[New LWP 19128]
[New LWP 19129]
[New LWP 19132]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f564721dd50 in ?? ()
Id Target Id Frame
* 1 LWP 19000 "kudu" 0x00007f564721dd50 in ?? ()
2 LWP 19002 "kudu" 0x00007f5647219fb9 in ?? ()
3 LWP 19003 "kudu" 0x00007f5647219fb9 in ?? ()
4 LWP 19004 "kudu" 0x00007f5647219fb9 in ?? ()
5 LWP 19005 "kernel-watcher-" 0x00007f5647219fb9 in ?? ()
6 LWP 19011 "ntp client-1901" 0x00007f564721d9e2 in ?? ()
7 LWP 19012 "file cache-evic" 0x00007f5647219fb9 in ?? ()
8 LWP 19013 "sq_acceptor" 0x00007f563ff3fcb9 in ?? ()
9 LWP 19016 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
10 LWP 19017 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
11 LWP 19018 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
12 LWP 19019 "rpc reactor-190" 0x00007f563ff4ca47 in ?? ()
13 LWP 19020 "MaintenanceMgr " 0x00007f5647219ad3 in ?? ()
14 LWP 19021 "txn-status-mana" 0x00007f5647219fb9 in ?? ()
15 LWP 19022 "collect_and_rem" 0x00007f5647219fb9 in ?? ()
16 LWP 19023 "tc-session-exp-" 0x00007f5647219fb9 in ?? ()
17 LWP 19024 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
18 LWP 19025 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
19 LWP 19026 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
20 LWP 19027 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
21 LWP 19028 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
22 LWP 19029 "rpc worker-1902" 0x00007f5647219ad3 in ?? ()
23 LWP 19030 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
24 LWP 19031 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
25 LWP 19032 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
26 LWP 19033 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
27 LWP 19034 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
28 LWP 19035 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
29 LWP 19036 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
30 LWP 19037 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
31 LWP 19038 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
32 LWP 19039 "rpc worker-1903" 0x00007f5647219ad3 in ?? ()
33 LWP 19040 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
34 LWP 19041 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
35 LWP 19042 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
36 LWP 19043 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
37 LWP 19044 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
38 LWP 19045 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
39 LWP 19046 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
40 LWP 19047 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
41 LWP 19048 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
42 LWP 19049 "rpc worker-1904" 0x00007f5647219ad3 in ?? ()
43 LWP 19050 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
44 LWP 19051 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
45 LWP 19052 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
46 LWP 19053 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
47 LWP 19054 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
48 LWP 19055 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
49 LWP 19056 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
50 LWP 19057 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
51 LWP 19058 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
52 LWP 19059 "rpc worker-1905" 0x00007f5647219ad3 in ?? ()
53 LWP 19060 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
54 LWP 19061 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
55 LWP 19062 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
56 LWP 19063 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
57 LWP 19064 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
58 LWP 19065 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
59 LWP 19066 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
60 LWP 19067 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
61 LWP 19068 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
62 LWP 19069 "rpc worker-1906" 0x00007f5647219ad3 in ?? ()
63 LWP 19070 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
64 LWP 19071 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
65 LWP 19072 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
66 LWP 19073 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
67 LWP 19074 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
68 LWP 19075 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
69 LWP 19076 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
70 LWP 19077 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
71 LWP 19078 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
72 LWP 19079 "rpc worker-1907" 0x00007f5647219ad3 in ?? ()
73 LWP 19080 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
74 LWP 19081 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
75 LWP 19082 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
76 LWP 19083 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
77 LWP 19084 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
78 LWP 19085 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
79 LWP 19086 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
80 LWP 19087 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
81 LWP 19088 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
82 LWP 19089 "rpc worker-1908" 0x00007f5647219ad3 in ?? ()
83 LWP 19090 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
84 LWP 19091 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
85 LWP 19092 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
86 LWP 19093 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
87 LWP 19094 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
88 LWP 19095 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
89 LWP 19096 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
90 LWP 19097 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
91 LWP 19098 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
92 LWP 19099 "rpc worker-1909" 0x00007f5647219ad3 in ?? ()
93 LWP 19100 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
94 LWP 19101 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
95 LWP 19102 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
96 LWP 19103 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
97 LWP 19104 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
98 LWP 19106 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
99 LWP 19107 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
100 LWP 19108 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
101 LWP 19109 "rpc worker-1910" 0x00007f5647219ad3 in ?? ()
102 LWP 19110 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
103 LWP 19111 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
104 LWP 19112 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
105 LWP 19113 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
106 LWP 19114 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
107 LWP 19115 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
108 LWP 19116 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
109 LWP 19117 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
110 LWP 19118 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
111 LWP 19119 "rpc worker-1911" 0x00007f5647219ad3 in ?? ()
112 LWP 19120 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
113 LWP 19121 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
114 LWP 19122 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
115 LWP 19123 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
116 LWP 19124 "rpc worker-1912" 0x00007f5647219ad3 in ?? ()
117 LWP 19125 "diag-logger-191" 0x00007f5647219fb9 in ?? ()
118 LWP 19126 "result-tracker-" 0x00007f5647219fb9 in ?? ()
119 LWP 19127 "excess-log-dele" 0x00007f5647219fb9 in ?? ()
120 LWP 19128 "acceptor-19128" 0x00007f563ff4e0c7 in ?? ()
121 LWP 19129 "heartbeat-19129" 0x00007f5647219fb9 in ?? ()
122 LWP 19132 "maintenance_sch" 0x00007f5647219fb9 in ?? ()
Thread 122 (LWP 19132):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55fc2b26e0 in ?? ()
#2 0x000000000000032c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f55fc2b2730 in ?? ()
#6 0x0000000000000658 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19129):
#0 0x00007f5647219fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x00000000000000ca in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f55fd2d95f0 in ?? ()
#6 0x0000000000000194 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19128):
#0 0x00007f563ff4e0c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007f55fdb00d90 in ?? ()
#3 0x00007f55fdb00dc0 in ?? ()
#4 0x00007f55fdb00ec0 in ?? ()
#5 0x00007f55fdb00db0 in ?? ()
#6 0x00007f55fdb00e20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x00007f5643dd4980 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 19127):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55fe31af40 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc5442a1d0 in ?? ()
#5 0x00007f55fe31af30 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19126):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f55feb33100 in ?? ()
#2 0x00000000000000cb in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f55feb330f0 in ?? ()
#6 0x0000000000000196 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19125):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19124):
#0 0x00007f5647219ad3 in ?? ()
#1 0x000000000000001b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f5638fb7e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5638fb7eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f5638fb7eb0 in ?? ()
#11 0x00000feac71f6fd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 19123):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000029 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f55ffb62e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f55ffb62eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f55ffb62eb0 in ?? ()
#11 0x00000feabff6c5d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 19122):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19121):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19120):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19119):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19118):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19117):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19116):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19115):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19114):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19113):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19112):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19111):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19110):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19109):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19108):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19107):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19106):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19104):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19103):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19102):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19101):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19100):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19099):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19098):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19097):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19096):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19095):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19094):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19093):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19092):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19091):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19090):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19089):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19088):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19087):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19086):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19085):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19084):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19083):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f561371be90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f561371beb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f561371beb0 in ?? ()
#11 0x00000feac26e37d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 19082):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19081):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19080):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19079):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19078):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19077):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19076):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19075):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19074):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19073):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19072):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19071):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19070):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19069):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19068):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19067):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19066):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19065):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19064):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19063):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f561d8fce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f561d8fceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f561d8fceb0 in ?? ()
#11 0x00000feac3b1f9d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 19062):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 19061):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 19060):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 19059):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 19058):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 19057):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 19056):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 19055):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 19054):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 19053):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 19052):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 19051):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 19050):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 19049):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 19048):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 19047):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 19046):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 19045):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 19044):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 19043):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f5627adde90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f5627addeb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f5647219770 in ?? ()
#10 0x00007f5627addeb0 in ?? ()
#11 0x00000feac4f5bbd8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 19042):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 19041):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 19040):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 19039):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 19038):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 19037):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 19036):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 19035):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 19034):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 19033):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 19032):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 19031):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 19030):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 19029):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 19028):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 19027):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 19026):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 19025):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 19024):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 19023):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5631cdccc0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f5631cdccb0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 19022):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 19021):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5632d1a240 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f5632d1a230 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 19020):
#0 0x00007f5647219ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 19019):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5633d69320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f5633d69310 in ?? ()
#4 0x00007f5633d69520 in ?? ()
#5 0x00007f5633d69360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f5633d693e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb96cb25277a000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000de000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f839f in ?? ()
#16 0x00000feb467a527c in ?? ()
#17 0x00007f5633d693c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5633d69390 in ?? ()
#20 0x3fb96cb25277a000 in ?? ()
#21 0x0000000033d693e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb96cb25277a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 19018):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5634580320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f5634580310 in ?? ()
#4 0x00007f5634580520 in ?? ()
#5 0x00007f5634580360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f56345803e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb96f7416db4000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000f8000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000feb468a807c in ?? ()
#17 0x00007f56345803c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5634580390 in ?? ()
#20 0x3fb96f7416db4000 in ?? ()
#21 0x00000000345803e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb96f7416db4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 19017):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5634d97320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f5634d97310 in ?? ()
#4 0x00007f5634d97520 in ?? ()
#5 0x00007f5634d97360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f5634d973e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb965dc1b2f0000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000f2000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000feb469aae7c in ?? ()
#17 0x00007f5634d973c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x00007f5634d97390 in ?? ()
#20 0x3fb965dc1b2f0000 in ?? ()
#21 0x0000000034d973e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb965dc1b2f0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 19016):
#0 0x00007f563ff4ca47 in ?? ()
#1 0x00007f5636dad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f5636dad310 in ?? ()
#4 0x00007f5636dad520 in ?? ()
#5 0x00007f5636dad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f5636dad3e0 in ?? ()
#8 0x00007f56425bf25d in ?? ()
#9 0x3fb98c28d86e6000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000001f000000 in ?? ()
#14 0x00007f5600000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000feb46dada7c in ?? ()
#17 0x00007f5636dad3c0 in ?? ()
#18 0x00007f56425c3ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 19013):
#0 0x00007f563ff3fcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f56387b77c8 in ?? ()
#3 0x000060200001e510 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 19012):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f5637fb60c0 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f5637fb60b0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 19011):
#0 0x00007f564721d9e2 in ?? ()
#1 0x00007f56377b3ba0 in ?? ()
#2 0x00007f56377b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f56377b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f56377b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f56377b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f564cb30edc in ?? ()
#12 0x00007f564cb208d9 in ?? ()
#13 0x00007f564cb24ec6 in ?? ()
#14 0x00007f564cb29900 in ?? ()
#15 0x00007f564cb29729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f5643df54fe in ?? ()
#18 0x00007f56472136db in ?? ()
#19 0x00000feb46eee684 in ?? ()
#20 0x00007f56377b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 19005):
#0 0x00007f5647219fb9 in ?? ()
#1 0x00007f56397b8c80 in ?? ()
#2 0x00000000000003f9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f56397b8c70 in ?? ()
#6 0x00000000000007f2 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 19004):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 19003):
#0 0x00007f5647219fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 19002):
#0 0x00007f5647219fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f563afbc450 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 19000):
#0 0x00007f564721dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:35.045310 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 7 with UUID 8150d3eeb1c84234a13484098d17ce8f and pid 20806
************************ BEGIN STACKS **************************
[New LWP 20807]
[New LWP 20808]
[New LWP 20809]
[New LWP 20811]
[New LWP 20818]
[New LWP 20819]
[New LWP 20820]
[New LWP 20825]
[New LWP 20827]
[New LWP 20828]
[New LWP 20829]
[New LWP 20830]
[New LWP 20831]
[New LWP 20832]
[New LWP 20833]
[New LWP 20834]
[New LWP 20835]
[New LWP 20836]
[New LWP 20837]
[New LWP 20838]
[New LWP 20839]
[New LWP 20840]
[New LWP 20841]
[New LWP 20842]
[New LWP 20843]
[New LWP 20844]
[New LWP 20845]
[New LWP 20846]
[New LWP 20847]
[New LWP 20848]
[New LWP 20849]
[New LWP 20850]
[New LWP 20851]
[New LWP 20852]
[New LWP 20853]
[New LWP 20854]
[New LWP 20855]
[New LWP 20856]
[New LWP 20857]
[New LWP 20858]
[New LWP 20859]
[New LWP 20860]
[New LWP 20861]
[New LWP 20863]
[New LWP 20864]
[New LWP 20865]
[New LWP 20866]
[New LWP 20867]
[New LWP 20868]
[New LWP 20869]
[New LWP 20870]
[New LWP 20871]
[New LWP 20872]
[New LWP 20873]
[New LWP 20874]
[New LWP 20875]
[New LWP 20876]
[New LWP 20877]
[New LWP 20878]
[New LWP 20879]
[New LWP 20880]
[New LWP 20881]
[New LWP 20882]
[New LWP 20883]
[New LWP 20884]
[New LWP 20885]
[New LWP 20886]
[New LWP 20887]
[New LWP 20888]
[New LWP 20889]
[New LWP 20890]
[New LWP 20891]
[New LWP 20892]
[New LWP 20893]
[New LWP 20894]
[New LWP 20895]
[New LWP 20896]
[New LWP 20897]
[New LWP 20898]
[New LWP 20899]
[New LWP 20900]
[New LWP 20901]
[New LWP 20902]
[New LWP 20903]
[New LWP 20904]
[New LWP 20905]
[New LWP 20906]
[New LWP 20907]
[New LWP 20908]
[New LWP 20909]
[New LWP 20910]
[New LWP 20911]
[New LWP 20912]
[New LWP 20913]
[New LWP 20914]
[New LWP 20915]
[New LWP 20916]
[New LWP 20917]
[New LWP 20918]
[New LWP 20919]
[New LWP 20920]
[New LWP 20921]
[New LWP 20922]
[New LWP 20923]
[New LWP 20924]
[New LWP 20925]
[New LWP 20926]
[New LWP 20927]
[New LWP 20928]
[New LWP 20929]
[New LWP 20930]
[New LWP 20931]
[New LWP 20932]
[New LWP 20933]
[New LWP 20934]
[New LWP 20935]
[New LWP 20936]
[New LWP 20937]
[New LWP 20938]
[New LWP 20939]
[New LWP 20944]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007f0f8125bd50 in ?? ()
Id Target Id Frame
* 1 LWP 20806 "kudu" 0x00007f0f8125bd50 in ?? ()
2 LWP 20807 "kudu" 0x00007f0f81257fb9 in ?? ()
3 LWP 20808 "kudu" 0x00007f0f81257fb9 in ?? ()
4 LWP 20809 "kudu" 0x00007f0f81257fb9 in ?? ()
5 LWP 20811 "kernel-watcher-" 0x00007f0f81257fb9 in ?? ()
6 LWP 20818 "ntp client-2081" 0x00007f0f8125b9e2 in ?? ()
7 LWP 20819 "file cache-evic" 0x00007f0f81257fb9 in ?? ()
8 LWP 20820 "sq_acceptor" 0x00007f0f79f7dcb9 in ?? ()
9 LWP 20825 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
10 LWP 20827 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
11 LWP 20828 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
12 LWP 20829 "rpc reactor-208" 0x00007f0f79f8aa47 in ?? ()
13 LWP 20830 "MaintenanceMgr " 0x00007f0f81257ad3 in ?? ()
14 LWP 20831 "txn-status-mana" 0x00007f0f81257fb9 in ?? ()
15 LWP 20832 "collect_and_rem" 0x00007f0f81257fb9 in ?? ()
16 LWP 20833 "tc-session-exp-" 0x00007f0f81257fb9 in ?? ()
17 LWP 20834 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
18 LWP 20835 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
19 LWP 20836 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
20 LWP 20837 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
21 LWP 20838 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
22 LWP 20839 "rpc worker-2083" 0x00007f0f81257ad3 in ?? ()
23 LWP 20840 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
24 LWP 20841 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
25 LWP 20842 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
26 LWP 20843 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
27 LWP 20844 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
28 LWP 20845 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
29 LWP 20846 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
30 LWP 20847 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
31 LWP 20848 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
32 LWP 20849 "rpc worker-2084" 0x00007f0f81257ad3 in ?? ()
33 LWP 20850 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
34 LWP 20851 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
35 LWP 20852 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
36 LWP 20853 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
37 LWP 20854 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
38 LWP 20855 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
39 LWP 20856 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
40 LWP 20857 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
41 LWP 20858 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
42 LWP 20859 "rpc worker-2085" 0x00007f0f81257ad3 in ?? ()
43 LWP 20860 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
44 LWP 20861 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
45 LWP 20863 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
46 LWP 20864 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
47 LWP 20865 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
48 LWP 20866 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
49 LWP 20867 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
50 LWP 20868 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
51 LWP 20869 "rpc worker-2086" 0x00007f0f81257ad3 in ?? ()
52 LWP 20870 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
53 LWP 20871 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
54 LWP 20872 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
55 LWP 20873 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
56 LWP 20874 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
57 LWP 20875 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
58 LWP 20876 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
59 LWP 20877 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
60 LWP 20878 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
61 LWP 20879 "rpc worker-2087" 0x00007f0f81257ad3 in ?? ()
62 LWP 20880 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
63 LWP 20881 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
64 LWP 20882 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
65 LWP 20883 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
66 LWP 20884 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
67 LWP 20885 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
68 LWP 20886 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
69 LWP 20887 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
70 LWP 20888 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
71 LWP 20889 "rpc worker-2088" 0x00007f0f81257ad3 in ?? ()
72 LWP 20890 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
73 LWP 20891 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
74 LWP 20892 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
75 LWP 20893 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
76 LWP 20894 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
77 LWP 20895 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
78 LWP 20896 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
79 LWP 20897 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
80 LWP 20898 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
81 LWP 20899 "rpc worker-2089" 0x00007f0f81257ad3 in ?? ()
82 LWP 20900 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
83 LWP 20901 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
84 LWP 20902 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
85 LWP 20903 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
86 LWP 20904 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
87 LWP 20905 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
88 LWP 20906 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
89 LWP 20907 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
90 LWP 20908 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
91 LWP 20909 "rpc worker-2090" 0x00007f0f81257ad3 in ?? ()
92 LWP 20910 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
93 LWP 20911 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
94 LWP 20912 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
95 LWP 20913 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
96 LWP 20914 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
97 LWP 20915 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
98 LWP 20916 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
99 LWP 20917 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
100 LWP 20918 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
101 LWP 20919 "rpc worker-2091" 0x00007f0f81257ad3 in ?? ()
102 LWP 20920 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
103 LWP 20921 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
104 LWP 20922 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
105 LWP 20923 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
106 LWP 20924 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
107 LWP 20925 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
108 LWP 20926 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
109 LWP 20927 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
110 LWP 20928 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
111 LWP 20929 "rpc worker-2092" 0x00007f0f81257ad3 in ?? ()
112 LWP 20930 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
113 LWP 20931 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
114 LWP 20932 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
115 LWP 20933 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
116 LWP 20934 "rpc worker-2093" 0x00007f0f81257ad3 in ?? ()
117 LWP 20935 "diag-logger-209" 0x00007f0f81257fb9 in ?? ()
118 LWP 20936 "result-tracker-" 0x00007f0f81257fb9 in ?? ()
119 LWP 20937 "excess-log-dele" 0x00007f0f81257fb9 in ?? ()
120 LWP 20938 "acceptor-20938" 0x00007f0f79f8c0c7 in ?? ()
121 LWP 20939 "heartbeat-20939" 0x00007f0f81257fb9 in ?? ()
122 LWP 20944 "maintenance_sch" 0x00007f0f81257fb9 in ?? ()
Thread 122 (LWP 20944):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f362f06e0 in ?? ()
#2 0x000000000000015a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007f0f362f0730 in ?? ()
#6 0x00000000000002b4 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 20939):
#0 0x00007f0f81257fb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007f0f373175f0 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 20938):
#0 0x00007f0f79f8c0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007f0f37b3ed90 in ?? ()
#3 0x00007f0f37b3edc0 in ?? ()
#4 0x00007f0f37b3eec0 in ?? ()
#5 0x00007f0f37b3edb0 in ?? ()
#6 0x00007f0f37b3ee20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 20937):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f38358f40 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffde53678d0 in ?? ()
#5 0x00007f0f38358f30 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 20936):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f38b71100 in ?? ()
#2 0x0000000000000056 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007f0f38b710f0 in ?? ()
#6 0x00000000000000ac in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 20935):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 20934):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x000000000000000f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f0f39ba0e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f39ba0eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f0f81257770 in ?? ()
#10 0x00007f0f39ba0eb0 in ?? ()
#11 0x00000fe1e73741d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 20933):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f0f3a3b8e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f3a3b8eb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f0f81257770 in ?? ()
#10 0x00007f0f3a3b8eb0 in ?? ()
#11 0x00000fe1e74771d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 20932):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 20931):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 20930):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 20929):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 20928):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 20927):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 20926):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 20925):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 20924):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 20923):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 20922):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 20921):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 20920):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 20919):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 20918):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 20917):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 20916):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 20915):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 20914):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 20913):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 20912):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 20911):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 20910):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 20909):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 20908):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 20907):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 20906):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 20905):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 20904):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 20903):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 20902):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 20901):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 20900):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 20899):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 20898):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 20897):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 20896):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 20895):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 20894):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f0f4df71e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f4df71eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 20893):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 20892):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 20891):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 20890):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 20889):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 20888):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 20887):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 20886):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 20885):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 20884):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 20883):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 20882):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 20881):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 20880):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 20879):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 20878):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 20877):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 20876):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 20875):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 20874):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f0f58152e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f58152eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 20873):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 20872):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 20871):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 20870):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 20869):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 20868):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 20867):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 20866):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 20865):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 20864):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 20863):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 20861):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 20860):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 20859):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 20858):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 20857):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 20856):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 20855):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 20854):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 20853):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f0f62333e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0f62333eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 20852):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 20851):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 20850):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20849):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20848):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20847):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20846):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20845):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20844):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20843):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20842):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20841):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20840):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20839):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20838):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20837):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20836):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20835):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20834):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20833):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f6bd1dcc0 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019ca0 in ?? ()
#5 0x00007f0f6bd1dcb0 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20832):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20831):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f6cd5b240 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007f0f6cd5b230 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20830):
#0 0x00007f0f81257ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 20829):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6ddaa320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007f0f6ddaa310 in ?? ()
#4 0x00007f0f6ddaa520 in ?? ()
#5 0x00007f0f6ddaa360 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f0f6ddaa3e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb97ec9bd418000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x00000000c1000000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a2 in ?? ()
#16 0x00000fe26dbad47c in ?? ()
#17 0x00007f0f6ddaa3c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6ddaa390 in ?? ()
#20 0x3fb97ec9bd418000 in ?? ()
#21 0x000000006ddaa3e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb97ec9bd418000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20828):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6e5b3320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f0f6e5b3310 in ?? ()
#4 0x00007f0f6e5b3520 in ?? ()
#5 0x00007f0f6e5b3360 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f0f6e5b33e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb97e76d7d2a000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000080c00000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fe26dcae67c in ?? ()
#17 0x00007f0f6e5b33c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6e5b3390 in ?? ()
#20 0x3fb97e76d7d2a000 in ?? ()
#21 0x000000006e5b33e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021098 in ?? ()
#24 0x3fb97e76d7d2a000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20827):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f6edb4320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f0f6edb4310 in ?? ()
#4 0x00007f0f6edb4520 in ?? ()
#5 0x00007f0f6edb4360 in ?? ()
#6 0x0000614000020e98 in ?? ()
#7 0x00007f0f6edb43e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb96e659b47e000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000fa400000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a1 in ?? ()
#16 0x00000fe26ddae87c in ?? ()
#17 0x00007f0f6edb43c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x00007f0f6edb4390 in ?? ()
#20 0x3fb96e659b47e000 in ?? ()
#21 0x000000006edb43e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000020e98 in ?? ()
#24 0x3fb96e659b47e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20825):
#0 0x00007f0f79f8aa47 in ?? ()
#1 0x00007f0f70dad320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f0f70dad310 in ?? ()
#4 0x00007f0f70dad520 in ?? ()
#5 0x00007f0f70dad360 in ?? ()
#6 0x0000614000020c98 in ?? ()
#7 0x00007f0f70dad3e0 in ?? ()
#8 0x00007f0f7c5fd25d in ?? ()
#9 0x3fb9827036a3c000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000d9000000 in ?? ()
#14 0x00007f0f00000000 in ?? ()
#15 0x41da1696258f83a3 in ?? ()
#16 0x00000fe26e1ada7c in ?? ()
#17 0x00007f0f70dad3c0 in ?? ()
#18 0x00007f0f7c601ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20820):
#0 0x00007f0f79f7dcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f0f727b77c8 in ?? ()
#3 0x000060200001e9d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20819):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f71fb60c0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007f0f71fb60b0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20818):
#0 0x00007f0f8125b9e2 in ?? ()
#1 0x00007f0f717b3ba0 in ?? ()
#2 0x00007f0f717b3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f0f717b3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007f0f717b3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007f0f717b3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007f0f86b6eedc in ?? ()
#12 0x00007f0f86b5e8d9 in ?? ()
#13 0x00007f0f86b62ec6 in ?? ()
#14 0x00007f0f86b67900 in ?? ()
#15 0x00007f0f86b67729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007f0f7de334fe in ?? ()
#18 0x00007f0f812516db in ?? ()
#19 0x00000fe26e2ee684 in ?? ()
#20 0x00007f0f717b3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20811):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x00007f0f737b8c80 in ?? ()
#2 0x00000000000001bb in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007f0f737b8c70 in ?? ()
#6 0x0000000000000376 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20809):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20808):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20807):
#0 0x00007f0f81257fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f0f74fbc450 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20806):
#0 0x00007f0f8125bd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:35.951174 17269 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 8 with UUID 6cbd668c726a4b1298e789270a0031e2 and pid 20953
************************ BEGIN STACKS **************************
[New LWP 20957]
[New LWP 20958]
[New LWP 20959]
[New LWP 20961]
[New LWP 20967]
[New LWP 20968]
[New LWP 20969]
[New LWP 20974]
[New LWP 20975]
[New LWP 20976]
[New LWP 20977]
[New LWP 20978]
[New LWP 20979]
[New LWP 20981]
[New LWP 20982]
[New LWP 20983]
[New LWP 20984]
[New LWP 20985]
[New LWP 20986]
[New LWP 20987]
[New LWP 20988]
[New LWP 20989]
[New LWP 20990]
[New LWP 20991]
[New LWP 20992]
[New LWP 20993]
[New LWP 20994]
[New LWP 20995]
[New LWP 20996]
[New LWP 20997]
[New LWP 20998]
[New LWP 20999]
[New LWP 21000]
[New LWP 21001]
[New LWP 21002]
[New LWP 21003]
[New LWP 21004]
[New LWP 21005]
[New LWP 21006]
[New LWP 21007]
[New LWP 21008]
[New LWP 21009]
[New LWP 21010]
[New LWP 21011]
[New LWP 21012]
[New LWP 21013]
[New LWP 21014]
[New LWP 21015]
[New LWP 21016]
[New LWP 21017]
[New LWP 21018]
[New LWP 21019]
[New LWP 21020]
[New LWP 21021]
[New LWP 21022]
[New LWP 21023]
[New LWP 21024]
[New LWP 21025]
[New LWP 21026]
[New LWP 21028]
[New LWP 21029]
[New LWP 21030]
[New LWP 21031]
[New LWP 21032]
[New LWP 21033]
[New LWP 21034]
[New LWP 21035]
[New LWP 21036]
[New LWP 21037]
[New LWP 21038]
[New LWP 21039]
[New LWP 21040]
[New LWP 21041]
[New LWP 21042]
[New LWP 21043]
[New LWP 21044]
[New LWP 21045]
[New LWP 21046]
[New LWP 21047]
[New LWP 21048]
[New LWP 21049]
[New LWP 21050]
[New LWP 21051]
[New LWP 21052]
[New LWP 21053]
[New LWP 21054]
[New LWP 21059]
[New LWP 21061]
[New LWP 21062]
[New LWP 21063]
[New LWP 21064]
[New LWP 21065]
[New LWP 21066]
[New LWP 21067]
[New LWP 21068]
[New LWP 21069]
[New LWP 21070]
[New LWP 21071]
[New LWP 21072]
[New LWP 21073]
[New LWP 21074]
[New LWP 21075]
[New LWP 21076]
[New LWP 21077]
[New LWP 21078]
[New LWP 21079]
[New LWP 21080]
[New LWP 21081]
[New LWP 21082]
[New LWP 21083]
[New LWP 21084]
[New LWP 21085]
[New LWP 21086]
[New LWP 21087]
[New LWP 21088]
[New LWP 21089]
[New LWP 21090]
[New LWP 21091]
[New LWP 21092]
[New LWP 21094]
[New LWP 21095]
[New LWP 21740]
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e71
Cannot access memory at address 0x6563656950676e69
0x00007ffa6e71ed50 in ?? ()
Id Target Id Frame
* 1 LWP 20953 "kudu" 0x00007ffa6e71ed50 in ?? ()
2 LWP 20957 "kudu" 0x00007ffa6e71afb9 in ?? ()
3 LWP 20958 "kudu" 0x00007ffa6e71afb9 in ?? ()
4 LWP 20959 "kudu" 0x00007ffa6e71afb9 in ?? ()
5 LWP 20961 "kernel-watcher-" 0x00007ffa6e71afb9 in ?? ()
6 LWP 20967 "ntp client-2096" 0x00007ffa6e71e9e2 in ?? ()
7 LWP 20968 "file cache-evic" 0x00007ffa6e71afb9 in ?? ()
8 LWP 20969 "sq_acceptor" 0x00007ffa67440cb9 in ?? ()
9 LWP 20974 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
10 LWP 20975 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
11 LWP 20976 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
12 LWP 20977 "rpc reactor-209" 0x00007ffa6744da47 in ?? ()
13 LWP 20978 "MaintenanceMgr " 0x00007ffa6e71aad3 in ?? ()
14 LWP 20979 "txn-status-mana" 0x00007ffa6e71afb9 in ?? ()
15 LWP 20981 "collect_and_rem" 0x00007ffa6e71afb9 in ?? ()
16 LWP 20982 "tc-session-exp-" 0x00007ffa6e71afb9 in ?? ()
17 LWP 20983 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
18 LWP 20984 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
19 LWP 20985 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
20 LWP 20986 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
21 LWP 20987 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
22 LWP 20988 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
23 LWP 20989 "rpc worker-2098" 0x00007ffa6e71aad3 in ?? ()
24 LWP 20990 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
25 LWP 20991 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
26 LWP 20992 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
27 LWP 20993 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
28 LWP 20994 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
29 LWP 20995 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
30 LWP 20996 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
31 LWP 20997 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
32 LWP 20998 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
33 LWP 20999 "rpc worker-2099" 0x00007ffa6e71aad3 in ?? ()
34 LWP 21000 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
35 LWP 21001 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
36 LWP 21002 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
37 LWP 21003 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
38 LWP 21004 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
39 LWP 21005 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
40 LWP 21006 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
41 LWP 21007 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
42 LWP 21008 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
43 LWP 21009 "rpc worker-2100" 0x00007ffa6e71aad3 in ?? ()
44 LWP 21010 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
45 LWP 21011 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
46 LWP 21012 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
47 LWP 21013 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
48 LWP 21014 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
49 LWP 21015 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
50 LWP 21016 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
51 LWP 21017 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
52 LWP 21018 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
53 LWP 21019 "rpc worker-2101" 0x00007ffa6e71aad3 in ?? ()
54 LWP 21020 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
55 LWP 21021 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
56 LWP 21022 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
57 LWP 21023 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
58 LWP 21024 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
59 LWP 21025 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
60 LWP 21026 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
61 LWP 21028 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
62 LWP 21029 "rpc worker-2102" 0x00007ffa6e71aad3 in ?? ()
63 LWP 21030 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
64 LWP 21031 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
65 LWP 21032 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
66 LWP 21033 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
67 LWP 21034 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
68 LWP 21035 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
69 LWP 21036 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
70 LWP 21037 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
71 LWP 21038 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
72 LWP 21039 "rpc worker-2103" 0x00007ffa6e71aad3 in ?? ()
73 LWP 21040 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
74 LWP 21041 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
75 LWP 21042 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
76 LWP 21043 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
77 LWP 21044 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
78 LWP 21045 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
79 LWP 21046 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
80 LWP 21047 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
81 LWP 21048 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
82 LWP 21049 "rpc worker-2104" 0x00007ffa6e71aad3 in ?? ()
83 LWP 21050 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
84 LWP 21051 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
85 LWP 21052 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
86 LWP 21053 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
87 LWP 21054 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
88 LWP 21059 "rpc worker-2105" 0x00007ffa6e71aad3 in ?? ()
89 LWP 21061 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
90 LWP 21062 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
91 LWP 21063 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
92 LWP 21064 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
93 LWP 21065 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
94 LWP 21066 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
95 LWP 21067 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
96 LWP 21068 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
97 LWP 21069 "rpc worker-2106" 0x00007ffa6e71aad3 in ?? ()
98 LWP 21070 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
99 LWP 21071 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
100 LWP 21072 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
101 LWP 21073 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
102 LWP 21074 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
103 LWP 21075 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
104 LWP 21076 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
105 LWP 21077 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
106 LWP 21078 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
107 LWP 21079 "rpc worker-2107" 0x00007ffa6e71aad3 in ?? ()
108 LWP 21080 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
109 LWP 21081 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
110 LWP 21082 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
111 LWP 21083 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
112 LWP 21084 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
113 LWP 21085 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
114 LWP 21086 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
115 LWP 21087 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
116 LWP 21088 "rpc worker-2108" 0x00007ffa6e71aad3 in ?? ()
117 LWP 21089 "diag-logger-210" 0x00007ffa6e71afb9 in ?? ()
118 LWP 21090 "result-tracker-" 0x00007ffa6e71afb9 in ?? ()
119 LWP 21091 "excess-log-dele" 0x00007ffa6e71afb9 in ?? ()
120 LWP 21092 "acceptor-21092" 0x00007ffa6744f0c7 in ?? ()
121 LWP 21094 "heartbeat-21094" 0x00007ffa6e71afb9 in ?? ()
122 LWP 21095 "maintenance_sch" 0x00007ffa6e71afb9 in ?? ()
123 LWP 21740 "raft [worker]-2" 0x00007ffa6e71afb9 in ?? ()
Thread 123 (LWP 21740):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000002f in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007ffa198beba4 in ?? ()
#5 0x00007ffa198be830 in ?? ()
#6 0x000000000000005f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ffa198be850 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 21095):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa225d06e0 in ?? ()
#2 0x000000000000014e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006160000157a0 in ?? ()
#5 0x00007ffa225d0730 in ?? ()
#6 0x000000000000029c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 21094):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0xd9fe2f690aca2e06 in ?? ()
#2 0x0000000000000057 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007ffa22df65f0 in ?? ()
#6 0x00000000000000af in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007ffa22df6610 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007ffa22df6690 in ?? ()
#12 0x00000fff445becd4 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 21092):
#0 0x00007ffa6744f0c7 in ?? ()
#1 0x0000614000040ea8 in ?? ()
#2 0x00007ffa23e1ed90 in ?? ()
#3 0x00007ffa23e1edc0 in ?? ()
#4 0x00007ffa23e1eec0 in ?? ()
#5 0x00007ffa23e1edb0 in ?? ()
#6 0x00007ffa23e1ee20 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008b0dab in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 21091):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa24637f40 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffef798a370 in ?? ()
#5 0x00007ffa24637f30 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 21090):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa24e50100 in ?? ()
#2 0x0000000000000054 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000898f0 in ?? ()
#5 0x00007ffa24e500f0 in ?? ()
#6 0x00000000000000a8 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 21089):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 21088):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000018 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000350078 in ?? ()
#4 0x00007ffa25e7fe90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa25e7feb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 21087):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00034d088 in ?? ()
#4 0x00007ffa266d9e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa266d9eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 21086):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 21085):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 21084):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 21083):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 21082):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 21081):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 21080):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 21079):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 21078):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 21077):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 21076):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 21075):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 21074):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 21073):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 21072):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 21071):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 21070):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 21069):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 21068):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 21067):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 21066):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 21065):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 21064):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 21063):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 21062):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 21061):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 21059):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 21054):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 21053):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 21052):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 21051):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 21050):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 21049):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 21048):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 21047):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 21046):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 21045):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 21044):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 21043):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x00000000000000ad in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007ffa3abcce90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa3abcceb0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007ffa6e71a770 in ?? ()
#10 0x00007ffa3abcceb0 in ?? ()
#11 0x00000fff475799d8 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 21042):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000206 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007ffa3b3e4e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa3b3e4eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 21041):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 21040):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 21039):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 21038):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 21037):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 21036):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 21035):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 21034):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 21033):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 21032):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 21031):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 21030):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 21029):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 21028):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 21026):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 21025):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 21024):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 21023):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 21022):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007ffa44dade90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa44dadeb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 21021):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 21020):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 21019):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 21018):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 21017):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 21016):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 21015):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 21014):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 21013):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 21012):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 21011):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 21010):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 21009):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 21008):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 21007):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 21006):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 21005):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 21004):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 21003):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 21002):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x000000000000000e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007ffa4d746e90 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007ffa4d746eb0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 21001):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 21000):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 20999):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 20998):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 20997):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 20996):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 20995):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 20994):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 20993):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 20992):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 20991):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 20990):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 20989):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 20988):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 20987):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 20986):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 20985):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 20984):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 20983):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 20982):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa5794bcc0 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a3a0 in ?? ()
#5 0x00007ffa5794bcb0 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 20981):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 20979):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa591e3240 in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000d1a8 in ?? ()
#5 0x00007ffa591e3230 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 20978):
#0 0x00007ffa6e71aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 20977):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5a232320 in ?? ()
#2 0x000061a00000cc80 in ?? ()
#3 0x00007ffa5a232310 in ?? ()
#4 0x00007ffa5a232520 in ?? ()
#5 0x00007ffa5a232360 in ?? ()
#6 0x0000614000041098 in ?? ()
#7 0x00007ffa5a2323e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3facce345eb50000 in ?? ()
#10 0x000061a00000cc80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000cc80 in ?? ()
#13 0x0000000076c00000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fffcb43e47c in ?? ()
#17 0x00007ffa5a2323c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5a232390 in ?? ()
#20 0x3facce345eb50000 in ?? ()
#21 0x000000005a2323e0 in ?? ()
#22 0x000061a00000cc80 in ?? ()
#23 0x0000614000041098 in ?? ()
#24 0x3facce345eb50000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 20976):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5aa49320 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007ffa5aa49310 in ?? ()
#4 0x00007ffa5aa49520 in ?? ()
#5 0x00007ffa5aa49360 in ?? ()
#6 0x0000614000040e98 in ?? ()
#7 0x00007ffa5aa493e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fb952ce8cb38000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x000000008d000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a0 in ?? ()
#16 0x00000fffcb54127c in ?? ()
#17 0x00007ffa5aa493c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5aa49390 in ?? ()
#20 0x3fb952ce8cb38000 in ?? ()
#21 0x000000005aa493e0 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000040e98 in ?? ()
#24 0x3fb952ce8cb38000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 20975):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5b260320 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007ffa5b260310 in ?? ()
#4 0x00007ffa5b260520 in ?? ()
#5 0x00007ffa5b260360 in ?? ()
#6 0x0000614000040c98 in ?? ()
#7 0x00007ffa5b2603e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fadbf5754b6c000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000007e000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f83a5 in ?? ()
#16 0x00000fffcb64407c in ?? ()
#17 0x00007ffa5b2603c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x00007ffa5b260390 in ?? ()
#20 0x3fadbf5754b6c000 in ?? ()
#21 0x000000005b2603e0 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000040c98 in ?? ()
#24 0x3fadbf5754b6c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 20974):
#0 0x00007ffa6744da47 in ?? ()
#1 0x00007ffa5d27f320 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007ffa5d27f310 in ?? ()
#4 0x00007ffa5d27f520 in ?? ()
#5 0x00007ffa5d27f360 in ?? ()
#6 0x0000614000040a98 in ?? ()
#7 0x00007ffa5d27f3e0 in ?? ()
#8 0x00007ffa69ac025d in ?? ()
#9 0x3fa7800ebf46c000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000042000000 in ?? ()
#14 0x00007ffa00000000 in ?? ()
#15 0x41da1696258f839e in ?? ()
#16 0x00000fffcba47e7c in ?? ()
#17 0x00007ffa5d27f3c0 in ?? ()
#18 0x00007ffa69ac4ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 20969):
#0 0x00007ffa67440cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007ffa5fcb77c8 in ?? ()
#3 0x000060200001ecd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a89f1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 20968):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa5f4b60c0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001d840 in ?? ()
#5 0x00007ffa5f4b60b0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 20967):
#0 0x00007ffa6e71e9e2 in ?? ()
#1 0x00007ffa5ecb3ba0 in ?? ()
#2 0x00007ffa5ecb3c40 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007ffa5ecb3bf0 in ?? ()
#6 0x000000000177ece0 in ?? ()
#7 0x00007ffa5ecb3c50 in ?? ()
#8 0x000061100008a890 in ?? ()
#9 0x00007ffa5ecb3c40 in ?? ()
#10 0x00000000008a2ee7 in __sanitizer::theDepot ()
#11 0x00007ffa74031edc in ?? ()
#12 0x00007ffa740218d9 in ?? ()
#13 0x00007ffa74025ec6 in ?? ()
#14 0x00007ffa7402a900 in ?? ()
#15 0x00007ffa7402a729 in ?? ()
#16 0x0000000000a7573d in __sanitizer::theDepot ()
#17 0x00007ffa6b2f64fe in ?? ()
#18 0x00007ffa6e7146db in ?? ()
#19 0x00000fffcbd8e684 in ?? ()
#20 0x00007ffa5ecb3440 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 20961):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x00007ffa60cb8c80 in ?? ()
#2 0x00000000000001ad in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001eea8 in ?? ()
#5 0x00007ffa60cb8c70 in ?? ()
#6 0x000000000000035a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 20959):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 20958):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 20957):
#0 0x00007ffa6e71afb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007ffa624bc450 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 20953):
#0 0x00007ffa6e71ed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250624 07:58:36.885171 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19496
I20250624 07:58:36.895292 19621 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.049970 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19496
W20250624 07:58:37.079545 17294 connection.cc:537] server connection from 127.16.221.65:41515 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250624 07:58:37.079855 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19354
I20250624 07:58:37.087955 19480 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.241357 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19354
I20250624 07:58:37.270782 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19638
I20250624 07:58:37.279215 19765 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.743018 21760 raft_consensus.cc:491] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6cbd668c726a4b1298e789270a0031e2)
I20250624 07:58:37.743366 21760 raft_consensus.cc:513] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 439cf2177bdb4c19aa8449b6cbd440a7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "439cf2177bdb4c19aa8449b6cbd440a7" member_type: VOTER last_known_addr { host: "127.16.221.67" port: 41189 } } peers { permanent_uuid: "4f219d2e8b954461973a8ca8680a6383" member_type: VOTER last_known_addr { host: "127.16.221.69" port: 35369 } } peers { permanent_uuid: "6cbd668c726a4b1298e789270a0031e2" member_type: VOTER last_known_addr { host: "127.16.221.73" port: 38265 } }
I20250624 07:58:37.743672 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19638
W20250624 07:58:37.823227 20974 connection.cc:537] client connection to 127.16.221.67:41189 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250624 07:58:37.823588 20974 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250624 07:58:37.824283 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18856
W20250624 07:58:37.827579 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250624 07:58:37.835234 18986 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:37.983995 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18856
I20250624 07:58:38.026615 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20075
I20250624 07:58:38.036019 20203 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.328924 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250624 07:58:38.545120 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20075
I20250624 07:58:38.617695 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 18718
I20250624 07:58:38.626900 18844 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.627946 20977 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250624 07:58:38.774062 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 18718
I20250624 07:58:38.804785 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 19000
I20250624 07:58:38.813060 19123 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:38.858184 20974 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 439cf2177bdb4c19aa8449b6cbd440a7 (127.16.221.67:41189): Couldn't send request to peer 439cf2177bdb4c19aa8449b6cbd440a7. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.67:41189: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250624 07:58:38.972692 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 19000
I20250624 07:58:39.002192 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20806
I20250624 07:58:39.012413 20933 generic_service.cc:196] Checking for leaks (request via RPC)
W20250624 07:58:39.117089 20977 consensus_peers.cc:487] T 5ca5ea09d51f4eb390c4433fec8ed9fa P 6cbd668c726a4b1298e789270a0031e2 -> Peer 4f219d2e8b954461973a8ca8680a6383 (127.16.221.69:35369): Couldn't send request to peer 4f219d2e8b954461973a8ca8680a6383. Status: Network error: Client connection negotiation failed: client connection to 127.16.221.69:35369: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250624 07:58:39.126052 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20806
I20250624 07:58:39.149242 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 20953
I20250624 07:58:39.158293 21088 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:39.600015 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 20953
I20250624 07:58:39.671206 17269 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu pid 17277
I20250624 07:58:39.679673 17337 generic_service.cc:196] Checking for leaks (request via RPC)
I20250624 07:58:39.920516 17269 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskaI4uEx/build/asan/bin/kudu with pid 17277
2025-06-24T07:58:39Z chronyd exiting
I20250624 07:58:39.987267 17269 test_util.cc:183] -----------------------------------------------
I20250624 07:58:39.987383 17269 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskaI4uEx/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1750751692493053-17269-0
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3, where GetParam() = 800-byte object <01-00 00-00 01-61 73-61 09-00 00-00 BE-BE BE-BE A0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-1F 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 BE-BE BE-BE BE-BE 03-00 00-00 03-00 00-00 00-BE BE-BE BE-BE BE-BE> (227437 ms)
[----------] 1 test from RollingRestartArgs/RollingRestartITest (227437 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (227437 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3, where GetParam() = 800-byte object <01-00 00-00 01-61 73-61 09-00 00-00 BE-BE BE-BE A0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-1F 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 BE-BE BE-BE BE-BE 03-00 00-00 03-00 00-00 00-BE BE-BE BE-BE BE-BE>
1 FAILED TEST