RollingRestartArgs/RollingRestartITest.TestWorkloads/3: /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:753: Failure
Failed
Bad status: Timed out: cluster didn't become healthy: Running 'cluster ksck 127.14.226.62:37841' did not succeed in 60 seconds: Runtime error: error running 'cluster ksck 127.14.226.62:37841': stdout: Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
6701e95595e54e48a427659232b86234 | 127.14.226.62:37841 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.14.226.20:36611 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server States
Server | State
----------------------------------+------------------
4c1123e925164e978604ac78771cb3ad | MAINTENANCE_MODE
cc4301cbd2b84d22ab621b2c399022dc | MAINTENANCE_MODE
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
09be37535c0a473a956abea447348515 | 127.14.226.1:38037 | HEALTHY | /L0 | 0 | 0
18452380c023464488c18995b91fa513 | 127.14.226.5:41645 | HEALTHY | /L1 | 0 | 0
49a14c4ba19447f8851a9a40990a4b91 | 127.14.226.7:37147 | HEALTHY | /L1 | 0 | 0
4c1123e925164e978604ac78771cb3ad | 127.14.226.9:39547 | HEALTHY | /L2 | 0 | 0
4e66492bc0e449a981045ff82ba305e8 | 127.14.226.4:33171 | HEALTHY | /L1 | 0 | 0
cc4301cbd2b84d22ab621b2c399022dc | 127.14.226.8:41133 | HEALTHY | /L2 | 0 | 0
dc04179134174d03bd03d3db91b245f8 | 127.14.226.3:45529 | HEALTHY | /L0 | 1 | 0
eb9ae591b5d24eb18f58490f46e7dfaf | 127.14.226.2:41707 | HEALTHY | /L0 | 0 | 0
edf2bd29095149dd9e4592ae2870d4ee | 127.14.226.6:42445 | HEALTHY | /L1 | 0 | 3
Tablet Server Location Summary
Location | Count
----------+---------
/L2 | 2
/L0 | 3
/L1 | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.14.226.1 | experimental | 127.14.226.1:38037
local_ip_for_outbound_sockets | 127.14.226.2 | experimental | 127.14.226.2:41707
local_ip_for_outbound_sockets | 127.14.226.3 | experimental | 127.14.226.3:45529
local_ip_for_outbound_sockets | 127.14.226.4 | experimental | 127.14.226.4:33171
local_ip_for_outbound_sockets | 127.14.226.5 | experimental | 127.14.226.5:41645
local_ip_for_outbound_sockets | 127.14.226.6 | experimental | 127.14.226.6:42445
local_ip_for_outbound_sockets | 127.14.226.7 | experimental | 127.14.226.7:37147
local_ip_for_outbound_sockets | 127.14.226.8 | experimental | 127.14.226.8:41133
local_ip_for_outbound_sockets | 127.14.226.9 | experimental | 127.14.226.9:39547
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb | hidden | 127.14.226.1:38037
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb | hidden | 127.14.226.2:41707
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb | hidden | 127.14.226.3:45529
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb | hidden | 127.14.226.4:33171
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb | hidden | 127.14.226.5:41645
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb | hidden | 127.14.226.6:42445
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb | hidden | 127.14.226.7:37147
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb | hidden | 127.14.226.8:41133
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb | hidden | 127.14.226.9:39547
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.14.226.20:36611 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
Tablet 54a6628492fa41d593f0c59af04ad973 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): RUNNING
dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): RUNNING [LEADER]
cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap replaying log segment 3/3 (579.2K/1.77M this segment, stats: ops{read=9742 overwritten=0 applied=9741 ignored=0} inserts{seen=486950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
All reported replicas are:
A = edf2bd29095149dd9e4592ae2870d4ee
B = dc04179134174d03bd03d3db91b245f8
C = cc4301cbd2b84d22ab621b2c399022dc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
, stderr: W20250111 21:01:54.147111 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 21:01:54.147701 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 21:01:54.161664 19505 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Runtime error: ksck discovered errors
: /tmp/dist-test-taskBASTb8/build/asan/bin/kudu: process exited with non-zero status 1
W20250111 21:01:56.234683 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 551: this message will repeat every 5th retry.
I20250111 21:01:56.712366 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 3/3 (1.72M/1.77M this segment, stats: ops{read=10423 overwritten=0 applied=10422 ignored=0} inserts{seen=521000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:56.724895 19557 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 4725 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:01:56.744405 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 556: this message will repeat every 5th retry.
I20250111 21:01:56.829862 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replayed 3/3 log segments. Stats: ops{read=10452 overwritten=0 applied=10452 ignored=0} inserts{seen=522500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 21:01:56.830878 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap complete.
I20250111 21:01:56.837132 18810 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent bootstrapping tablet: real 61.326s user 42.496s sys 1.360s
I20250111 21:01:56.843802 18810 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:56.845100 18810 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4301cbd2b84d22ab621b2c399022dc, State: Initialized, Role: FOLLOWER
I20250111 21:01:56.846081 18810 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10452, Last appended: 2.10452, Last appended by leader: 10452, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:56.860318 18810 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent starting tablet: real 0.023s user 0.005s sys 0.016s
I20250111 21:01:57.463286 19561 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:57.463577 19561 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:57.465703 19561 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:57.498972 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 11302 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:57.516265 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 11302 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:57.517169 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:57.759892 19561 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:58.590710 19573 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:58.590938 19573 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:58.591871 19573 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:58.592552 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13002 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:58.594054 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13002 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:58.594775 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:58.595048 19573 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.236052 19573 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:59.236269 19573 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:59.237141 19573 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:59.237602 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13852 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:59.237715 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13852 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:59.238401 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:59.238818 19573 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.861016 19575 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:59.861251 19575 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:59.862351 19575 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:59.863061 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 14702 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:59.863251 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 14702 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:59.864004 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:59.864497 19575 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.871767 19574 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 475 ops behind the committed index [suppressed 11 similar messages]
I20250111 21:02:00.216840 19575 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:00.217058 19575 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:00.217801 19575 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:00.218720 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15127 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:02:00.220028 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15127 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:02:00.221004 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:02:00.221526 19575 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:02:01.796489 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: FlushMRSOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 35.984s user 23.499s sys 0.728s Metrics: {"bytes_written":18669649,"cfile_init":1,"compiler_manager_pool.queue_time_us":718,"compiler_manager_pool.run_cpu_time_us":705010,"compiler_manager_pool.run_wall_time_us":1338006,"dirs.queue_time_us":6719,"dirs.run_cpu_time_us":288,"dirs.run_wall_time_us":6333,"drs_written":1,"lbm_read_time_us":151,"lbm_reads_lt_1ms":4,"lbm_write_time_us":142541,"lbm_writes_1-10_ms":8,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":806,"peak_mem_usage":0,"rows_written":672750,"spinlock_wait_cycles":653696,"thread_start_us":12492,"threads_started":3,"wal-append.queue_time_us":2587}
I20250111 21:02:01.798480 17614 maintenance_manager.cc:418] P dc04179134174d03bd03d3db91b245f8: Scheduling LogGCOp(54a6628492fa41d593f0c59af04ad973): free 16781455 bytes of WAL
I20250111 21:02:01.798970 17502 log_reader.cc:385] T 54a6628492fa41d593f0c59af04ad973: removed 2 log segments from log reader
I20250111 21:02:01.799135 17502 log.cc:1078] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Deleting log segment in path: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/wals/54a6628492fa41d593f0c59af04ad973/wal-000000001 (ops 1-4908)
I20250111 21:02:01.799324 17502 log.cc:1078] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Deleting log segment in path: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/wals/54a6628492fa41d593f0c59af04ad973/wal-000000002 (ops 4909-9529)
I20250111 21:02:01.804133 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: LogGCOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 0.005s user 0.000s sys 0.003s Metrics: {}
I20250111 21:02:01.804726 17614 maintenance_manager.cc:418] P dc04179134174d03bd03d3db91b245f8: Scheduling UndoDeltaBlockGCOp(54a6628492fa41d593f0c59af04ad973): 7913644 bytes on disk
I20250111 21:02:01.805950 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: UndoDeltaBlockGCOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20250111 21:02:02.717296 15240 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250111 21:02:02.717495 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 09be37535c0a473a956abea447348515 and pid 17622
************************ BEGIN STACKS **************************
[New LWP 17623]
[New LWP 17624]
[New LWP 17625]
[New LWP 17627]
[New LWP 17633]
[New LWP 17634]
[New LWP 17635]
[New LWP 17638]
[New LWP 17639]
[New LWP 17640]
[New LWP 17641]
[New LWP 17643]
[New LWP 17646]
[New LWP 17647]
[New LWP 17648]
[New LWP 17650]
[New LWP 17651]
[New LWP 17652]
[New LWP 17653]
[New LWP 17654]
[New LWP 17655]
[New LWP 17656]
[New LWP 17657]
[New LWP 17658]
[New LWP 17659]
[New LWP 17660]
[New LWP 17661]
[New LWP 17662]
[New LWP 17663]
[New LWP 17664]
[New LWP 17665]
[New LWP 17666]
[New LWP 17667]
[New LWP 17668]
[New LWP 17669]
[New LWP 17670]
[New LWP 17671]
[New LWP 17672]
[New LWP 17673]
[New LWP 17674]
[New LWP 17675]
[New LWP 17676]
[New LWP 17677]
[New LWP 17678]
[New LWP 17679]
[New LWP 17680]
[New LWP 17681]
[New LWP 17682]
[New LWP 17683]
[New LWP 17684]
[New LWP 17685]
[New LWP 17686]
[New LWP 17687]
[New LWP 17688]
[New LWP 17689]
[New LWP 17690]
[New LWP 17691]
[New LWP 17692]
[New LWP 17693]
[New LWP 17694]
[New LWP 17695]
[New LWP 17696]
[New LWP 17697]
[New LWP 17698]
[New LWP 17699]
[New LWP 17700]
[New LWP 17701]
[New LWP 17702]
[New LWP 17704]
[New LWP 17705]
[New LWP 17706]
[New LWP 17707]
[New LWP 17708]
[New LWP 17709]
[New LWP 17710]
[New LWP 17711]
[New LWP 17712]
[New LWP 17713]
[New LWP 17714]
[New LWP 17715]
[New LWP 17716]
[New LWP 17717]
[New LWP 17718]
[New LWP 17719]
[New LWP 17720]
[New LWP 17721]
[New LWP 17722]
[New LWP 17723]
[New LWP 17724]
[New LWP 17725]
[New LWP 17726]
[New LWP 17727]
[New LWP 17728]
[New LWP 17729]
[New LWP 17730]
[New LWP 17731]
[New LWP 17732]
[New LWP 17733]
[New LWP 17734]
[New LWP 17735]
[New LWP 17736]
[New LWP 17737]
[New LWP 17738]
[New LWP 17739]
[New LWP 17740]
[New LWP 17741]
[New LWP 17742]
[New LWP 17743]
[New LWP 17744]
[New LWP 17745]
[New LWP 17746]
[New LWP 17747]
[New LWP 17748]
[New LWP 17749]
[New LWP 17750]
[New LWP 17751]
[New LWP 17752]
[New LWP 17753]
[New LWP 17754]
[New LWP 17755]
[New LWP 17758]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fc62e46ad50 in ?? ()
Id Target Id Frame
* 1 LWP 17622 "kudu" 0x00007fc62e46ad50 in ?? ()
2 LWP 17623 "kudu" 0x00007fc62e466fb9 in ?? ()
3 LWP 17624 "kudu" 0x00007fc62e466fb9 in ?? ()
4 LWP 17625 "kudu" 0x00007fc62e466fb9 in ?? ()
5 LWP 17627 "kernel-watcher-" 0x00007fc62e466fb9 in ?? ()
6 LWP 17633 "ntp client-1763" 0x00007fc62e46a9e2 in ?? ()
7 LWP 17634 "file cache-evic" 0x00007fc62e466fb9 in ?? ()
8 LWP 17635 "sq_acceptor" 0x00007fc6271c3cb9 in ?? ()
9 LWP 17638 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
10 LWP 17639 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
11 LWP 17640 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
12 LWP 17641 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
13 LWP 17643 "MaintenanceMgr " 0x00007fc62e466ad3 in ?? ()
14 LWP 17646 "txn-status-mana" 0x00007fc62e466fb9 in ?? ()
15 LWP 17647 "collect_and_rem" 0x00007fc62e466fb9 in ?? ()
16 LWP 17648 "tc-session-exp-" 0x00007fc62e466fb9 in ?? ()
17 LWP 17650 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
18 LWP 17651 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
19 LWP 17652 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
20 LWP 17653 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
21 LWP 17654 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
22 LWP 17655 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
23 LWP 17656 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
24 LWP 17657 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
25 LWP 17658 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
26 LWP 17659 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
27 LWP 17660 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
28 LWP 17661 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
29 LWP 17662 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
30 LWP 17663 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
31 LWP 17664 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
32 LWP 17665 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
33 LWP 17666 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
34 LWP 17667 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
35 LWP 17668 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
36 LWP 17669 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
37 LWP 17670 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
38 LWP 17671 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
39 LWP 17672 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
40 LWP 17673 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
41 LWP 17674 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
42 LWP 17675 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
43 LWP 17676 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
44 LWP 17677 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
45 LWP 17678 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
46 LWP 17679 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
47 LWP 17680 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
48 LWP 17681 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
49 LWP 17682 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
50 LWP 17683 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
51 LWP 17684 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
52 LWP 17685 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
53 LWP 17686 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
54 LWP 17687 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
55 LWP 17688 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
56 LWP 17689 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
57 LWP 17690 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
58 LWP 17691 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
59 LWP 17692 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
60 LWP 17693 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
61 LWP 17694 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
62 LWP 17695 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
63 LWP 17696 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
64 LWP 17697 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
65 LWP 17698 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
66 LWP 17699 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
67 LWP 17700 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
68 LWP 17701 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
69 LWP 17702 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
70 LWP 17704 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
71 LWP 17705 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
72 LWP 17706 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
73 LWP 17707 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
74 LWP 17708 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
75 LWP 17709 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
76 LWP 17710 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
77 LWP 17711 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
78 LWP 17712 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
79 LWP 17713 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
80 LWP 17714 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
81 LWP 17715 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
82 LWP 17716 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
83 LWP 17717 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
84 LWP 17718 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
85 LWP 17719 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
86 LWP 17720 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
87 LWP 17721 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
88 LWP 17722 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
89 LWP 17723 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
90 LWP 17724 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
91 LWP 17725 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
92 LWP 17726 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
93 LWP 17727 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
94 LWP 17728 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
95 LWP 17729 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
96 LWP 17730 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
97 LWP 17731 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
98 LWP 17732 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
99 LWP 17733 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
100 LWP 17734 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
101 LWP 17735 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
102 LWP 17736 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
103 LWP 17737 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
104 LWP 17738 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
105 LWP 17739 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
106 LWP 17740 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
107 LWP 17741 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
108 LWP 17742 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
109 LWP 17743 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
110 LWP 17744 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
111 LWP 17745 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
112 LWP 17746 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
113 LWP 17747 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
114 LWP 17748 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
115 LWP 17749 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
116 LWP 17750 "rpc worker-1775" 0x00007fc62e466ad3 in ?? ()
117 LWP 17751 "diag-logger-177" 0x00007fc62e466fb9 in ?? ()
118 LWP 17752 "result-tracker-" 0x00007fc62e466fb9 in ?? ()
119 LWP 17753 "excess-log-dele" 0x00007fc62e466fb9 in ?? ()
120 LWP 17754 "acceptor-17754" 0x00007fc6271d20c7 in ?? ()
121 LWP 17755 "heartbeat-17755" 0x00007fc62e466fb9 in ?? ()
122 LWP 17758 "maintenance_sch" 0x00007fc62e466fb9 in ?? ()
Thread 122 (LWP 17758):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002a4 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fc5e353b550 in ?? ()
#6 0x0000000000000548 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17755):
#0 0x00007fc62e466fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000a8 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fc5e45625b0 in ?? ()
#6 0x0000000000000150 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17754):
#0 0x00007fc6271d20c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007fc5e4d89d50 in ?? ()
#3 0x00007fc5e4d89d80 in ?? ()
#4 0x00007fc5e4d89e80 in ?? ()
#5 0x00007fc5e4d89d70 in ?? ()
#6 0x00007fc5e4d89de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007fc62b05c620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17753):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc5e55a3f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe8a18b9d0 in ?? ()
#5 0x00007fc5e55a3ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17752):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc5e5dbc0c0 in ?? ()
#2 0x00000000000000a9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fc5e5dbc0b0 in ?? ()
#6 0x0000000000000152 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17751):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17750):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x000000000000001c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007fc5e6debe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5e6debe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 17749):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000024 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007fc5e7603e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5e7603e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 17748):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17747):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17746):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17745):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17744):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17743):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17742):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17741):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17740):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17739):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17738):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17737):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17736):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17735):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17734):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17733):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17732):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17731):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17730):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17729):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17728):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17727):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17726):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17725):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17724):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17723):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17722):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17721):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17720):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17719):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17718):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17717):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17716):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17715):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17714):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17713):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17712):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17711):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17710):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007fc5fb1bce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5fb1bce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17709):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17708):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17707):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17706):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17705):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17704):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17702):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17701):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17700):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17699):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17698):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17697):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17696):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17695):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17694):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17693):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17692):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17691):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17690):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17689):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fc60539de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc60539de70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17688):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17687):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17686):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17685):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17684):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17683):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17682):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17681):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17680):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17679):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17678):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17677):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17676):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17675):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17674):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17673):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17672):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17671):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17670):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17669):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007fc6201b7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc6201b7e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17668):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17667):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17666):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17665):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17664):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17663):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17662):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17661):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17660):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17659):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17658):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17657):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17656):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17655):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17654):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17653):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17652):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17651):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17650):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17648):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc618f61c80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fc618f61c70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17647):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17646):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc619f9f1c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fc619f9f1b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17643):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 17641):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61afb22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fc61afb22d0 in ?? ()
#4 0x00007fc61afb24e0 in ?? ()
#5 0x00007fc61afb2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fc61afb23a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb96c9c0220c000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000da000000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000ff9435ee474 in ?? ()
#17 0x00007fc61afb2380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61afb2350 in ?? ()
#20 0x3fb96c9c0220c000 in ?? ()
#21 0x000000001afb23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb96c9c0220c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17640):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61b7b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fc61b7b32d0 in ?? ()
#4 0x00007fc61b7b34e0 in ?? ()
#5 0x00007fc61b7b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fc61b7b33a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb96b18eca48000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000005e800000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff9436ee674 in ?? ()
#17 0x00007fc61b7b3380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61b7b3350 in ?? ()
#20 0x3fb96b18eca48000 in ?? ()
#21 0x000000001b7b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96b18eca48000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17639):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61bfb42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fc61bfb42d0 in ?? ()
#4 0x00007fc61bfb44e0 in ?? ()
#5 0x00007fc61bfb4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fc61bfb43a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb95de1b7034000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000051000000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff9437ee874 in ?? ()
#17 0x00007fc61bfb4380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61bfb4350 in ?? ()
#20 0x3fb95de1b7034000 in ?? ()
#21 0x000000001bfb43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb95de1b7034000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 17638):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61dfad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fc61dfad2d0 in ?? ()
#4 0x00007fc61dfad4e0 in ?? ()
#5 0x00007fc61dfad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fc61dfad3a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb988e27896a000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000f7800000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000ff943beda74 in ?? ()
#17 0x00007fc61dfad380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17635):
#0 0x00007fc6271c3cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fc61f9b7788 in ?? ()
#3 0x00006020000167d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17634):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc61f1b6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fc61f1b6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17633):
#0 0x00007fc62e46a9e2 in ?? ()
#1 0x00007fc61e9b3b60 in ?? ()
#2 0x00007fc61e9b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fc61e9b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fc61e9b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fc61e9b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fc633d8ef0c in ?? ()
#12 0x00007fc633d7e7b9 in ?? ()
#13 0x00007fc633d82da6 in ?? ()
#14 0x00007fc633d877e0 in ?? ()
#15 0x00007fc633d87609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fc62b07d0be in ?? ()
#18 0x00007fc62e4606db in ?? ()
#19 0x00000ff943d2e67c in ?? ()
#20 0x00007fc61e9b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17627):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc6209b8c40 in ?? ()
#2 0x0000000000000353 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fc6209b8c30 in ?? ()
#6 0x00000000000006a6 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17625):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17624):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17623):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fc6221bc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17622):
#0 0x00007fc62e46ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:04.004901 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID eb9ae591b5d24eb18f58490f46e7dfaf and pid 17338
************************ BEGIN STACKS **************************
[New LWP 17340]
[New LWP 17341]
[New LWP 17342]
[New LWP 17343]
[New LWP 17350]
[New LWP 17351]
[New LWP 17352]
[New LWP 17355]
[New LWP 17356]
[New LWP 17357]
[New LWP 17358]
[New LWP 17359]
[New LWP 17360]
[New LWP 17361]
[New LWP 17362]
[New LWP 17363]
[New LWP 17364]
[New LWP 17365]
[New LWP 17366]
[New LWP 17367]
[New LWP 17368]
[New LWP 17369]
[New LWP 17370]
[New LWP 17371]
[New LWP 17372]
[New LWP 17373]
[New LWP 17374]
[New LWP 17375]
[New LWP 17376]
[New LWP 17377]
[New LWP 17378]
[New LWP 17379]
[New LWP 17380]
[New LWP 17381]
[New LWP 17382]
[New LWP 17383]
[New LWP 17384]
[New LWP 17385]
[New LWP 17386]
[New LWP 17387]
[New LWP 17388]
[New LWP 17389]
[New LWP 17390]
[New LWP 17391]
[New LWP 17392]
[New LWP 17393]
[New LWP 17394]
[New LWP 17395]
[New LWP 17396]
[New LWP 17397]
[New LWP 17398]
[New LWP 17399]
[New LWP 17400]
[New LWP 17401]
[New LWP 17402]
[New LWP 17403]
[New LWP 17404]
[New LWP 17405]
[New LWP 17406]
[New LWP 17407]
[New LWP 17408]
[New LWP 17409]
[New LWP 17410]
[New LWP 17411]
[New LWP 17412]
[New LWP 17413]
[New LWP 17414]
[New LWP 17415]
[New LWP 17416]
[New LWP 17417]
[New LWP 17418]
[New LWP 17419]
[New LWP 17420]
[New LWP 17421]
[New LWP 17423]
[New LWP 17424]
[New LWP 17425]
[New LWP 17426]
[New LWP 17427]
[New LWP 17428]
[New LWP 17429]
[New LWP 17430]
[New LWP 17432]
[New LWP 17433]
[New LWP 17434]
[New LWP 17435]
[New LWP 17436]
[New LWP 17437]
[New LWP 17438]
[New LWP 17439]
[New LWP 17440]
[New LWP 17441]
[New LWP 17442]
[New LWP 17443]
[New LWP 17444]
[New LWP 17445]
[New LWP 17446]
[New LWP 17447]
[New LWP 17448]
[New LWP 17449]
[New LWP 17450]
[New LWP 17451]
[New LWP 17452]
[New LWP 17453]
[New LWP 17454]
[New LWP 17455]
[New LWP 17456]
[New LWP 17457]
[New LWP 17458]
[New LWP 17459]
[New LWP 17460]
[New LWP 17461]
[New LWP 17462]
[New LWP 17463]
[New LWP 17464]
[New LWP 17465]
[New LWP 17466]
[New LWP 17467]
[New LWP 17468]
[New LWP 17469]
[New LWP 17472]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f921f414d50 in ?? ()
Id Target Id Frame
* 1 LWP 17338 "kudu" 0x00007f921f414d50 in ?? ()
2 LWP 17340 "kudu" 0x00007f921f410fb9 in ?? ()
3 LWP 17341 "kudu" 0x00007f921f410fb9 in ?? ()
4 LWP 17342 "kudu" 0x00007f921f410fb9 in ?? ()
5 LWP 17343 "kernel-watcher-" 0x00007f921f410fb9 in ?? ()
6 LWP 17350 "ntp client-1735" 0x00007f921f4149e2 in ?? ()
7 LWP 17351 "file cache-evic" 0x00007f921f410fb9 in ?? ()
8 LWP 17352 "sq_acceptor" 0x00007f921816dcb9 in ?? ()
9 LWP 17355 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
10 LWP 17356 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
11 LWP 17357 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
12 LWP 17358 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
13 LWP 17359 "MaintenanceMgr " 0x00007f921f410ad3 in ?? ()
14 LWP 17360 "txn-status-mana" 0x00007f921f410fb9 in ?? ()
15 LWP 17361 "collect_and_rem" 0x00007f921f410fb9 in ?? ()
16 LWP 17362 "tc-session-exp-" 0x00007f921f410fb9 in ?? ()
17 LWP 17363 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
18 LWP 17364 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
19 LWP 17365 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
20 LWP 17366 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
21 LWP 17367 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
22 LWP 17368 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
23 LWP 17369 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
24 LWP 17370 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
25 LWP 17371 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
26 LWP 17372 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
27 LWP 17373 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
28 LWP 17374 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
29 LWP 17375 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
30 LWP 17376 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
31 LWP 17377 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
32 LWP 17378 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
33 LWP 17379 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
34 LWP 17380 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
35 LWP 17381 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
36 LWP 17382 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
37 LWP 17383 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
38 LWP 17384 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
39 LWP 17385 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
40 LWP 17386 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
41 LWP 17387 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
42 LWP 17388 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
43 LWP 17389 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
44 LWP 17390 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
45 LWP 17391 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
46 LWP 17392 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
47 LWP 17393 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
48 LWP 17394 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
49 LWP 17395 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
50 LWP 17396 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
51 LWP 17397 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
52 LWP 17398 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
53 LWP 17399 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
54 LWP 17400 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
55 LWP 17401 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
56 LWP 17402 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
57 LWP 17403 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
58 LWP 17404 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
59 LWP 17405 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
60 LWP 17406 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
61 LWP 17407 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
62 LWP 17408 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
63 LWP 17409 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
64 LWP 17410 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
65 LWP 17411 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
66 LWP 17412 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
67 LWP 17413 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
68 LWP 17414 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
69 LWP 17415 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
70 LWP 17416 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
71 LWP 17417 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
72 LWP 17418 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
73 LWP 17419 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
74 LWP 17420 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
75 LWP 17421 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
76 LWP 17423 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
77 LWP 17424 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
78 LWP 17425 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
79 LWP 17426 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
80 LWP 17427 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
81 LWP 17428 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
82 LWP 17429 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
83 LWP 17430 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
84 LWP 17432 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
85 LWP 17433 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
86 LWP 17434 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
87 LWP 17435 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
88 LWP 17436 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
89 LWP 17437 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
90 LWP 17438 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
91 LWP 17439 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
92 LWP 17440 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
93 LWP 17441 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
94 LWP 17442 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
95 LWP 17443 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
96 LWP 17444 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
97 LWP 17445 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
98 LWP 17446 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
99 LWP 17447 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
100 LWP 17448 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
101 LWP 17449 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
102 LWP 17450 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
103 LWP 17451 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
104 LWP 17452 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
105 LWP 17453 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
106 LWP 17454 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
107 LWP 17455 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
108 LWP 17456 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
109 LWP 17457 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
110 LWP 17458 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
111 LWP 17459 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
112 LWP 17460 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
113 LWP 17461 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
114 LWP 17462 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
115 LWP 17463 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
116 LWP 17464 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
117 LWP 17465 "diag-logger-174" 0x00007f921f410fb9 in ?? ()
118 LWP 17466 "result-tracker-" 0x00007f921f410fb9 in ?? ()
119 LWP 17467 "excess-log-dele" 0x00007f921f410fb9 in ?? ()
120 LWP 17468 "acceptor-17468" 0x00007f921817c0c7 in ?? ()
121 LWP 17469 "heartbeat-17469" 0x00007f921f410fb9 in ?? ()
122 LWP 17472 "maintenance_sch" 0x00007f921f410fb9 in ?? ()
Thread 122 (LWP 17472):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002bd in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f91d4cf4550 in ?? ()
#6 0x000000000000057a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17469):
#0 0x00007f921f410fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000af in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f91d550c5b0 in ?? ()
#6 0x000000000000015e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17468):
#0 0x00007f921817c0c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f91d5d33d50 in ?? ()
#3 0x00007f91d5d33d80 in ?? ()
#4 0x00007f91d5d33e80 in ?? ()
#5 0x00007f91d5d33d70 in ?? ()
#6 0x00007f91d5d33de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f921c006620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17467):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f91d654df00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffefea1d730 in ?? ()
#5 0x00007f91d654def0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17466):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f91d6d660c0 in ?? ()
#2 0x00000000000000af in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f91d6d660b0 in ?? ()
#6 0x000000000000015e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17465):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17464):
#0 0x00007f921f410ad3 in ?? ()
#1 0x000000000000002c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f91d7d95e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91d7d95e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 17463):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f91d85ade50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91d85ade70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 17462):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17461):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17460):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17459):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17458):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17457):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17456):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17455):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17454):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17453):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17452):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17451):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17450):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17449):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17448):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17447):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17446):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17445):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17444):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17443):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17442):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17441):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17440):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17439):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17438):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17437):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17436):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17435):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17434):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17433):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17432):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17430):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17429):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17428):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17427):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17426):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17425):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17424):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17423):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f91ec166e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91ec166e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17421):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17420):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17419):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17418):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17417):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17416):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17415):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17414):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17413):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17412):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17411):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17410):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17409):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17408):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17407):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17406):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17405):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17404):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17403):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17402):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f91f6347e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91f6347e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17401):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17400):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17399):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17398):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17397):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17396):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17395):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17394):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17393):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17392):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17391):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17390):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17389):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17388):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17387):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17386):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17385):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17384):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17383):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17382):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007f91ffd10e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91ffd10e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17381):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17380):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17379):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17378):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17377):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17376):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17375):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17374):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17373):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17372):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17371):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17370):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17369):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17368):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17367):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17366):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17365):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17364):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17363):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17362):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f9209f0fc80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f9209f0fc70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17361):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17360):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f920af4d1c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f920af4d1b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17359):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 17358):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920bf9c2e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f920bf9c2d0 in ?? ()
#4 0x00007f920bf9c4e0 in ?? ()
#5 0x00007f920bf9c320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f920bf9c3a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb96ea77db42000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000ef800000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ff2c17eb874 in ?? ()
#17 0x00007f920bf9c380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920bf9c350 in ?? ()
#20 0x3fb96ea77db42000 in ?? ()
#21 0x000000000bf9c3a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb96ea77db42000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17357):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920c7b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f920c7b32d0 in ?? ()
#4 0x00007f920c7b34e0 in ?? ()
#5 0x00007f920c7b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f920c7b33a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb96cf145286000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000064000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ff2c18ee674 in ?? ()
#17 0x00007f920c7b3380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920c7b3350 in ?? ()
#20 0x3fb96cf145286000 in ?? ()
#21 0x000000000c7b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96cf145286000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17356):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920cfb42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f920cfb42d0 in ?? ()
#4 0x00007f920cfb44e0 in ?? ()
#5 0x00007f920cfb4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f920cfb43a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb952fad3356000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000005b000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2c19ee874 in ?? ()
#17 0x00007f920cfb4380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920cfb4350 in ?? ()
#20 0x3fb952fad3356000 in ?? ()
#21 0x000000000cfb43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb952fad3356000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 17355):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920efad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f920efad2d0 in ?? ()
#4 0x00007f920efad4e0 in ?? ()
#5 0x00007f920efad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f920efad3a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb98ca5f3f9c000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000077000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff2c1deda74 in ?? ()
#17 0x00007f920efad380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17352):
#0 0x00007f921816dcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f92109b7788 in ?? ()
#3 0x0000602000015cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17351):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f92101b6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f92101b6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17350):
#0 0x00007f921f4149e2 in ?? ()
#1 0x00007f920f9b3b60 in ?? ()
#2 0x00007f920f9b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f920f9b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f920f9b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f920f9b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f9224d38f0c in ?? ()
#12 0x00007f9224d287b9 in ?? ()
#13 0x00007f9224d2cda6 in ?? ()
#14 0x00007f9224d317e0 in ?? ()
#15 0x00007f9224d31609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f921c0270be in ?? ()
#18 0x00007f921f40a6db in ?? ()
#19 0x00000ff2c1f2e67c in ?? ()
#20 0x00007f920f9b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17343):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f92119b8c40 in ?? ()
#2 0x000000000000036f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f92119b8c30 in ?? ()
#6 0x00000000000006de in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17342):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17341):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17340):
#0 0x00007f921f410fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f92131bc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17338):
#0 0x00007f921f414d50 in ?? ()
#1 0x0000000000000000 in ?? ()
W20250111 21:02:05.228107 17465 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
************************* END STACKS ***************************
I20250111 21:02:05.244814 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID dc04179134174d03bd03d3db91b245f8 and pid 17479
************************ BEGIN STACKS **************************
[New LWP 17481]
[New LWP 17482]
[New LWP 17483]
[New LWP 17484]
[New LWP 17490]
[New LWP 17491]
[New LWP 17492]
[New LWP 17498]
[New LWP 17499]
[New LWP 17500]
[New LWP 17501]
[New LWP 17502]
[New LWP 17503]
[New LWP 17505]
[New LWP 17506]
[New LWP 17507]
[New LWP 17508]
[New LWP 17509]
[New LWP 17510]
[New LWP 17511]
[New LWP 17512]
[New LWP 17513]
[New LWP 17514]
[New LWP 17515]
[New LWP 17516]
[New LWP 17517]
[New LWP 17518]
[New LWP 17519]
[New LWP 17520]
[New LWP 17521]
[New LWP 17522]
[New LWP 17523]
[New LWP 17524]
[New LWP 17525]
[New LWP 17526]
[New LWP 17527]
[New LWP 17528]
[New LWP 17529]
[New LWP 17530]
[New LWP 17531]
[New LWP 17532]
[New LWP 17533]
[New LWP 17534]
[New LWP 17535]
[New LWP 17536]
[New LWP 17537]
[New LWP 17538]
[New LWP 17539]
[New LWP 17540]
[New LWP 17541]
[New LWP 17542]
[New LWP 17543]
[New LWP 17544]
[New LWP 17545]
[New LWP 17546]
[New LWP 17547]
[New LWP 17548]
[New LWP 17549]
[New LWP 17550]
[New LWP 17551]
[New LWP 17552]
[New LWP 17553]
[New LWP 17554]
[New LWP 17555]
[New LWP 17556]
[New LWP 17557]
[New LWP 17558]
[New LWP 17559]
[New LWP 17560]
[New LWP 17561]
[New LWP 17562]
[New LWP 17563]
[New LWP 17564]
[New LWP 17565]
[New LWP 17566]
[New LWP 17567]
[New LWP 17568]
[New LWP 17569]
[New LWP 17570]
[New LWP 17571]
[New LWP 17572]
[New LWP 17573]
[New LWP 17574]
[New LWP 17575]
[New LWP 17576]
[New LWP 17577]
[New LWP 17578]
[New LWP 17579]
[New LWP 17580]
[New LWP 17581]
[New LWP 17582]
[New LWP 17583]
[New LWP 17584]
[New LWP 17585]
[New LWP 17586]
[New LWP 17587]
[New LWP 17588]
[New LWP 17589]
[New LWP 17590]
[New LWP 17591]
[New LWP 17592]
[New LWP 17593]
[New LWP 17594]
[New LWP 17595]
[New LWP 17596]
[New LWP 17597]
[New LWP 17598]
[New LWP 17599]
[New LWP 17600]
[New LWP 17601]
[New LWP 17602]
[New LWP 17603]
[New LWP 17604]
[New LWP 17605]
[New LWP 17606]
[New LWP 17607]
[New LWP 17608]
[New LWP 17609]
[New LWP 17610]
[New LWP 17611]
[New LWP 17614]
[New LWP 19593]
[New LWP 19602]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f87e30dbd50 in ?? ()
Id Target Id Frame
* 1 LWP 17479 "kudu" 0x00007f87e30dbd50 in ?? ()
2 LWP 17481 "kudu" 0x00007f87e30d7fb9 in ?? ()
3 LWP 17482 "kudu" 0x00007f87e30d7fb9 in ?? ()
4 LWP 17483 "kudu" 0x00007f87e30d7fb9 in ?? ()
5 LWP 17484 "kernel-watcher-" 0x00007f87e30d7fb9 in ?? ()
6 LWP 17490 "ntp client-1749" 0x00007f87e30db9e2 in ?? ()
7 LWP 17491 "file cache-evic" 0x00007f87e30d7fb9 in ?? ()
8 LWP 17492 "sq_acceptor" 0x00007f87dbe34cb9 in ?? ()
9 LWP 17498 "rpc reactor-174" 0x00007f87dbe41a47 in ?? ()
10 LWP 17499 "rpc reactor-174" 0x0000000000926cb7 in __sanitizer::theDepot ()
11 LWP 17500 "rpc reactor-175" 0x00007f87dbe41a47 in ?? ()
12 LWP 17501 "rpc reactor-175" 0x00007f87dbe41a47 in ?? ()
13 LWP 17502 "MaintenanceMgr " 0x00007f87e30d7ad3 in ?? ()
14 LWP 17503 "txn-status-mana" 0x00007f87e30d7fb9 in ?? ()
15 LWP 17505 "collect_and_rem" 0x00007f87e30d7fb9 in ?? ()
16 LWP 17506 "tc-session-exp-" 0x00007f87e30d7fb9 in ?? ()
17 LWP 17507 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
18 LWP 17508 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
19 LWP 17509 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
20 LWP 17510 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
21 LWP 17511 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
22 LWP 17512 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
23 LWP 17513 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
24 LWP 17514 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
25 LWP 17515 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
26 LWP 17516 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
27 LWP 17517 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
28 LWP 17518 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
29 LWP 17519 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
30 LWP 17520 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
31 LWP 17521 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
32 LWP 17522 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
33 LWP 17523 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
34 LWP 17524 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
35 LWP 17525 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
36 LWP 17526 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
37 LWP 17527 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
38 LWP 17528 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
39 LWP 17529 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
40 LWP 17530 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
41 LWP 17531 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
42 LWP 17532 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
43 LWP 17533 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
44 LWP 17534 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
45 LWP 17535 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
46 LWP 17536 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
47 LWP 17537 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
48 LWP 17538 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
49 LWP 17539 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
50 LWP 17540 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
51 LWP 17541 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
52 LWP 17542 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
53 LWP 17543 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
54 LWP 17544 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
55 LWP 17545 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
56 LWP 17546 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
57 LWP 17547 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
58 LWP 17548 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
59 LWP 17549 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
60 LWP 17550 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
61 LWP 17551 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
62 LWP 17552 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
63 LWP 17553 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
64 LWP 17554 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
65 LWP 17555 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
66 LWP 17556 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
67 LWP 17557 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
68 LWP 17558 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
69 LWP 17559 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
70 LWP 17560 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
71 LWP 17561 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
72 LWP 17562 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
73 LWP 17563 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
74 LWP 17564 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
75 LWP 17565 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
76 LWP 17566 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
77 LWP 17567 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
78 LWP 17568 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
79 LWP 17569 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
80 LWP 17570 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
81 LWP 17571 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
82 LWP 17572 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
83 LWP 17573 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
84 LWP 17574 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
85 LWP 17575 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
86 LWP 17576 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
87 LWP 17577 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
88 LWP 17578 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
89 LWP 17579 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
90 LWP 17580 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
91 LWP 17581 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
92 LWP 17582 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
93 LWP 17583 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
94 LWP 17584 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
95 LWP 17585 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
96 LWP 17586 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
97 LWP 17587 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
98 LWP 17588 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
99 LWP 17589 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
100 LWP 17590 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
101 LWP 17591 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
102 LWP 17592 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
103 LWP 17593 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
104 LWP 17594 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
105 LWP 17595 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
106 LWP 17596 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
107 LWP 17597 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
108 LWP 17598 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
109 LWP 17599 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
110 LWP 17600 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
111 LWP 17601 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
112 LWP 17602 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
113 LWP 17603 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
114 LWP 17604 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
115 LWP 17605 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
116 LWP 17606 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
117 LWP 17607 "diag-logger-176" 0x00007f87e30d7fb9 in ?? ()
118 LWP 17608 "result-tracker-" 0x00007f87e30d7fb9 in ?? ()
119 LWP 17609 "excess-log-dele" 0x00007f87e30d7fb9 in ?? ()
120 LWP 17610 "acceptor-17610" 0x00007f87dbe430c7 in ?? ()
121 LWP 17611 "heartbeat-17611" 0x00007f87e30d7fb9 in ?? ()
122 LWP 17614 "maintenance_sch" 0x00007f87e30d7fb9 in ?? ()
123 LWP 19593 "raft [worker]-1" 0x00007f87e30d7fb9 in ?? ()
124 LWP 19602 "rpc reactor-174" 0x00007f87dbe41711 in ?? ()
Thread 124 (LWP 19602):
#0 0x00007f87dbe41711 in ?? ()
#1 0x00007f87e30d1600 in ?? ()
#2 0x00007f878b0fa700 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 123 (LWP 19593):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000045 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f878cbf4b24 in ?? ()
#5 0x00007f878cbf47b0 in ?? ()
#6 0x000000000000008b in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f878cbf47d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 17614):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002ba in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000616000017584 in ?? ()
#5 0x00007f8798161550 in ?? ()
#6 0x0000000000000575 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f8798161570 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0xffffffff0000006e in ?? ()
#12 0x00007f8798162030 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 121 (LWP 17611):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000b2 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061300001aac4 in ?? ()
#5 0x00007f87989795b0 in ?? ()
#6 0x0000000000000165 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f87989795d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f8798979650 in ?? ()
#12 0x00000ff0f312f2cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 17610):
#0 0x00007f87dbe430c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f87991a0d50 in ?? ()
#3 0x00007f87991a0d80 in ?? ()
#4 0x00007f87991a0e80 in ?? ()
#5 0x00007f87991a0d70 in ?? ()
#6 0x00007f87991a0de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f87dfccd620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17609):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87999b9f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc6591bef0 in ?? ()
#5 0x00007f87999b9ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17608):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f879a1d30c0 in ?? ()
#2 0x00000000000000ae in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f879a1d30b0 in ?? ()
#6 0x000000000000015c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17607):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17606):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000027 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002d682c in ?? ()
#4 0x00007f879b202e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f879b202e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002d67e0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f879b202e70 in ?? ()
#11 0x00000ff0f36405d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 17605):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000019 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002d002c in ?? ()
#4 0x00007f879ba1ae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f879ba1ae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002cffe0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f879ba1ae70 in ?? ()
#11 0x00000ff0f37435d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 17604):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17603):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17602):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17601):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17600):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17599):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17598):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17597):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17596):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17595):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17594):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17593):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17592):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17591):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17590):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17589):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17588):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17587):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17586):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17585):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17584):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17583):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17582):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17581):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17580):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17579):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17578):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17577):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17576):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17575):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17574):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17573):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17572):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17571):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17570):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17569):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17568):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17567):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17566):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000330 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001c0068 in ?? ()
#4 0x00007f87af5d3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87af5d3e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17565):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000304 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001b6878 in ?? ()
#4 0x00007f87afdebe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87afdebe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 17564):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001b007c in ?? ()
#4 0x00007f87b0603e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87b0603e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001b0030 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87b0603e70 in ?? ()
#11 0x00000ff0f60c07d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 17563):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17562):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17561):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17560):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17559):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17558):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17557):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17556):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17555):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17554):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17553):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17552):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17551):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17550):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17549):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17548):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17547):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17546):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000136828 in ?? ()
#4 0x00007f87b8f9ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87b8f9ce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17545):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17544):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17543):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17542):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17541):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17540):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17539):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17538):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17537):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17536):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17535):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17534):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17533):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17532):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17531):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17530):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17529):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17528):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17527):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17526):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000001016 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007f87c317de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c317de70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17525):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000dfc in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007f87c3995e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c3995e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 17524):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000bcf in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f87c41ade50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c41ade70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87c41ade70 in ?? ()
#11 0x00000ff0f8835bd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 17523):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17522):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17521):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17520):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17519):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17518):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17517):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17516):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17515):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17514):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17513):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17512):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17511):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17510):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17509):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17508):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17507):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17506):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87cd384c80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f87cd384c70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17505):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17503):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87cec191c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f87cec191b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17502):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x00007f87cf440b24 in ?? ()
#4 0x00007f87cf440830 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87cf440850 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00006160000177c0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87cf440850 in ?? ()
#11 0x00007f87cf440850 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 12 (LWP 17501):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87cfc682e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f87cfc682d0 in ?? ()
#4 0x00007f87cfc684e0 in ?? ()
#5 0x00007f87cfc68320 in ?? ()
#6 0x0000614000021a98 in ?? ()
#7 0x00007f87cfc683a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3f95ec6d96790000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000ca000000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff179f85074 in ?? ()
#17 0x00007f87cfc68380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x00007f87cfc68350 in ?? ()
#20 0x3f95ec6d96790000 in ?? ()
#21 0x00000000cfc683a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021a98 in ?? ()
#24 0x3f95ec6d96790000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17500):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87d047f2e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f87d047f2d0 in ?? ()
#4 0x00007f87d047f4e0 in ?? ()
#5 0x00007f87d047f320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f87d047f3a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3fb97224309e0000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000009d000000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff17a087e74 in ?? ()
#17 0x00007f87d047f380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x00007f87d047f350 in ?? ()
#20 0x3fb97224309e0000 in ?? ()
#21 0x00000000d047f3a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb97224309e0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17499):
#0 0x0000000000926cb7 in __sanitizer::theDepot ()
#1 0x000000000088c36d in __sanitizer::theDepot ()
#2 0x00007f87f313a2ee in ?? ()
#3 0x00000000e69ae515 in ?? ()
#4 0x00007f87d7a88000 in ?? ()
#5 0x0000000000000000 in ?? ()
Thread 9 (LWP 17498):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87d2cad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f87d2cad2d0 in ?? ()
#4 0x00007f87d2cad4e0 in ?? ()
#5 0x00007f87d2cad320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f87d2cad3a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3f930eb6fdb18000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000068f00000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ff17a58da74 in ?? ()
#17 0x00007f87d2cad380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17492):
#0 0x00007f87dbe34cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f87d46b7788 in ?? ()
#3 0x0000602000015a10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17491):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87d3eb6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f87d3eb6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17490):
#0 0x00007f87e30db9e2 in ?? ()
#1 0x00007f87d36b3b60 in ?? ()
#2 0x00007f87d36b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f87d36b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f87d36b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f87d36b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f87e89fff0c in ?? ()
#12 0x00007f87e89ef7b9 in ?? ()
#13 0x00007f87e89f3da6 in ?? ()
#14 0x00007f87e89f87e0 in ?? ()
#15 0x00007f87e89f8609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f87dfcee0be in ?? ()
#18 0x00007f87e30d16db in ?? ()
#19 0x00000ff17a6ce67c in ?? ()
#20 0x00007f87d36b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17484):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87d56b8c40 in ?? ()
#2 0x000000000000036c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f87d56b8c30 in ?? ()
#6 0x00000000000006d8 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17483):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17482):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17481):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f87d6ebc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17479):
#0 0x00007f87e30dbd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:06.637135 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 3 with UUID 4e66492bc0e449a981045ff82ba305e8 and pid 16832
************************ BEGIN STACKS **************************
[New LWP 16834]
[New LWP 16835]
[New LWP 16836]
[New LWP 16837]
[New LWP 16843]
[New LWP 16844]
[New LWP 16845]
[New LWP 16848]
[New LWP 16849]
[New LWP 16850]
[New LWP 16851]
[New LWP 16852]
[New LWP 16853]
[New LWP 16854]
[New LWP 16855]
[New LWP 16856]
[New LWP 16857]
[New LWP 16858]
[New LWP 16859]
[New LWP 16860]
[New LWP 16861]
[New LWP 16862]
[New LWP 16863]
[New LWP 16864]
[New LWP 16865]
[New LWP 16866]
[New LWP 16867]
[New LWP 16869]
[New LWP 16870]
[New LWP 16871]
[New LWP 16872]
[New LWP 16873]
[New LWP 16874]
[New LWP 16876]
[New LWP 16877]
[New LWP 16878]
[New LWP 16879]
[New LWP 16880]
[New LWP 16881]
[New LWP 16882]
[New LWP 16883]
[New LWP 16884]
[New LWP 16885]
[New LWP 16886]
[New LWP 16887]
[New LWP 16888]
[New LWP 16889]
[New LWP 16890]
[New LWP 16891]
[New LWP 16892]
[New LWP 16893]
[New LWP 16894]
[New LWP 16895]
[New LWP 16896]
[New LWP 16897]
[New LWP 16898]
[New LWP 16899]
[New LWP 16900]
[New LWP 16901]
[New LWP 16902]
[New LWP 16903]
[New LWP 16904]
[New LWP 16905]
[New LWP 16906]
[New LWP 16907]
[New LWP 16908]
[New LWP 16909]
[New LWP 16910]
[New LWP 16911]
[New LWP 16912]
[New LWP 16913]
[New LWP 16914]
[New LWP 16915]
[New LWP 16916]
[New LWP 16917]
[New LWP 16918]
[New LWP 16919]
[New LWP 16921]
[New LWP 16922]
[New LWP 16923]
[New LWP 16924]
[New LWP 16925]
[New LWP 16926]
[New LWP 16928]
[New LWP 16929]
[New LWP 16930]
[New LWP 16931]
[New LWP 16932]
[New LWP 16933]
[New LWP 16934]
[New LWP 16935]
[New LWP 16937]
[New LWP 16938]
[New LWP 16939]
[New LWP 16940]
[New LWP 16941]
[New LWP 16942]
[New LWP 16943]
[New LWP 16944]
[New LWP 16945]
[New LWP 16946]
[New LWP 16947]
[New LWP 16948]
[New LWP 16949]
[New LWP 16950]
[New LWP 16951]
[New LWP 16952]
[New LWP 16953]
[New LWP 16954]
[New LWP 16955]
[New LWP 16956]
[New LWP 16957]
[New LWP 16958]
[New LWP 16959]
[New LWP 16960]
[New LWP 16961]
[New LWP 16962]
[New LWP 16963]
[New LWP 16964]
[New LWP 16965]
[New LWP 16968]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f05c5d4ad50 in ?? ()
Id Target Id Frame
* 1 LWP 16832 "kudu" 0x00007f05c5d4ad50 in ?? ()
2 LWP 16834 "kudu" 0x00007f05c5d46fb9 in ?? ()
3 LWP 16835 "kudu" 0x00007f05c5d46fb9 in ?? ()
4 LWP 16836 "kudu" 0x00007f05c5d46fb9 in ?? ()
5 LWP 16837 "kernel-watcher-" 0x00007f05c5d46fb9 in ?? ()
6 LWP 16843 "ntp client-1684" 0x00007f05c5d4a9e2 in ?? ()
7 LWP 16844 "file cache-evic" 0x00007f05c5d46fb9 in ?? ()
8 LWP 16845 "sq_acceptor" 0x00007f05beaa3cb9 in ?? ()
9 LWP 16848 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
10 LWP 16849 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
11 LWP 16850 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
12 LWP 16851 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
13 LWP 16852 "MaintenanceMgr " 0x00007f05c5d46ad3 in ?? ()
14 LWP 16853 "txn-status-mana" 0x00007f05c5d46fb9 in ?? ()
15 LWP 16854 "collect_and_rem" 0x00007f05c5d46fb9 in ?? ()
16 LWP 16855 "tc-session-exp-" 0x00007f05c5d46fb9 in ?? ()
17 LWP 16856 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
18 LWP 16857 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
19 LWP 16858 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
20 LWP 16859 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
21 LWP 16860 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
22 LWP 16861 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
23 LWP 16862 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
24 LWP 16863 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
25 LWP 16864 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
26 LWP 16865 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
27 LWP 16866 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
28 LWP 16867 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
29 LWP 16869 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
30 LWP 16870 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
31 LWP 16871 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
32 LWP 16872 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
33 LWP 16873 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
34 LWP 16874 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
35 LWP 16876 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
36 LWP 16877 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
37 LWP 16878 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
38 LWP 16879 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
39 LWP 16880 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
40 LWP 16881 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
41 LWP 16882 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
42 LWP 16883 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
43 LWP 16884 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
44 LWP 16885 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
45 LWP 16886 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
46 LWP 16887 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
47 LWP 16888 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
48 LWP 16889 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
49 LWP 16890 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
50 LWP 16891 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
51 LWP 16892 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
52 LWP 16893 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
53 LWP 16894 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
54 LWP 16895 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
55 LWP 16896 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
56 LWP 16897 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
57 LWP 16898 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
58 LWP 16899 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
59 LWP 16900 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
60 LWP 16901 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
61 LWP 16902 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
62 LWP 16903 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
63 LWP 16904 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
64 LWP 16905 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
65 LWP 16906 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
66 LWP 16907 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
67 LWP 16908 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
68 LWP 16909 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
69 LWP 16910 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
70 LWP 16911 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
71 LWP 16912 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
72 LWP 16913 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
73 LWP 16914 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
74 LWP 16915 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
75 LWP 16916 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
76 LWP 16917 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
77 LWP 16918 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
78 LWP 16919 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
79 LWP 16921 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
80 LWP 16922 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
81 LWP 16923 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
82 LWP 16924 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
83 LWP 16925 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
84 LWP 16926 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
85 LWP 16928 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
86 LWP 16929 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
87 LWP 16930 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
88 LWP 16931 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
89 LWP 16932 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
90 LWP 16933 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
91 LWP 16934 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
92 LWP 16935 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
93 LWP 16937 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
94 LWP 16938 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
95 LWP 16939 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
96 LWP 16940 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
97 LWP 16941 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
98 LWP 16942 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
99 LWP 16943 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
100 LWP 16944 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
101 LWP 16945 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
102 LWP 16946 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
103 LWP 16947 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
104 LWP 16948 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
105 LWP 16949 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
106 LWP 16950 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
107 LWP 16951 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
108 LWP 16952 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
109 LWP 16953 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
110 LWP 16954 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
111 LWP 16955 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
112 LWP 16956 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
113 LWP 16957 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
114 LWP 16958 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
115 LWP 16959 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
116 LWP 16960 "rpc worker-1696" 0x00007f05c5d46ad3 in ?? ()
117 LWP 16961 "diag-logger-169" 0x00007f05c5d46fb9 in ?? ()
118 LWP 16962 "result-tracker-" 0x00007f05c5d46fb9 in ?? ()
119 LWP 16963 "excess-log-dele" 0x00007f05c5d46fb9 in ?? ()
120 LWP 16964 "acceptor-16964" 0x00007f05beab20c7 in ?? ()
121 LWP 16965 "heartbeat-16965" 0x00007f05c5d46fb9 in ?? ()
122 LWP 16968 "maintenance_sch" 0x00007f05c5d46fb9 in ?? ()
Thread 122 (LWP 16968):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000031b in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f057b62a550 in ?? ()
#6 0x0000000000000636 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 16965):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f057be425b0 in ?? ()
#6 0x000000000000018c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 16964):
#0 0x00007f05beab20c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007f057c669d50 in ?? ()
#3 0x00007f057c669d80 in ?? ()
#4 0x00007f057c669e80 in ?? ()
#5 0x00007f057c669d70 in ?? ()
#6 0x00007f057c669de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f05c293c620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 16963):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f057ce83f00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007fff7624fbb0 in ?? ()
#5 0x00007f057ce83ef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 16962):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f057d69c0c0 in ?? ()
#2 0x00000000000000c7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f057d69c0b0 in ?? ()
#6 0x000000000000018e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 16961):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 16960):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000027 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f057e6cbe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f057e6cbe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f057e6cbe70 in ?? ()
#11 0x00000fe0afcd97d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 16959):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x000000000000001d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f057eee3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f057eee3e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f057eee3e70 in ?? ()
#11 0x00000fe0afddc7d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 16958):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 16957):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 16956):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 16955):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 16954):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 16953):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 16952):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 16951):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 16950):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 16949):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 16948):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 16947):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 16946):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 16945):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 16944):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 16943):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 16942):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 16941):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 16940):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 16939):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 16938):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 16937):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 16935):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 16934):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 16933):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 16932):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 16931):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 16930):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 16929):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 16928):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 16926):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 16925):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 16924):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 16923):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 16922):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 16921):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 16919):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 16918):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 16917):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f0592a9ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0592a9ce70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f0592a9ce70 in ?? ()
#11 0x00000fe0b25539d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 16916):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 16915):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 16914):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 16913):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 16912):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 16911):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 16910):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 16909):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 16908):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 16907):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 16906):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 16905):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 16904):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 16903):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 16902):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 16901):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 16900):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 16899):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 16898):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 16897):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f059cc7de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f059cc7de70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f059cc7de70 in ?? ()
#11 0x00000fe0b398fbd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 16896):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 16895):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 16894):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 16893):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 16892):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 16891):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 16890):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 16889):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 16888):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 16887):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 16886):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 16885):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 16884):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 16883):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 16882):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 16881):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 16880):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 16879):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 16878):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 16877):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f05a6647e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f05a6647e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f05a6647e70 in ?? ()
#11 0x00000fe0b4cc8fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 16876):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 16874):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 16873):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 16872):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 16871):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 16870):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 16869):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 16867):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 16866):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 16865):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 16864):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 16863):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 16862):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 16861):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 16860):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 16859):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 16858):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 16857):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16856):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16855):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b084bc80 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f05b084bc70 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16854):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16853):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b18891c0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f05b18891b0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16852):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16851):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b28b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f05b28b22d0 in ?? ()
#4 0x00007f05b28b24e0 in ?? ()
#5 0x00007f05b28b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f05b28b23a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb970ceac376000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000068800000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000fe13650e474 in ?? ()
#17 0x00007f05b28b2380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b28b2350 in ?? ()
#20 0x3fb970ceac376000 in ?? ()
#21 0x00000000b28b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb970ceac376000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16850):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b30b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f05b30b32d0 in ?? ()
#4 0x00007f05b30b34e0 in ?? ()
#5 0x00007f05b30b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f05b30b33a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb97cc4d9b80000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000ce000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000fe13660e674 in ?? ()
#17 0x00007f05b30b3380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b30b3350 in ?? ()
#20 0x3fb97cc4d9b80000 in ?? ()
#21 0x00000000b30b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb97cc4d9b80000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16849):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b38b42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f05b38b42d0 in ?? ()
#4 0x00007f05b38b44e0 in ?? ()
#5 0x00007f05b38b4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f05b38b43a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb97682f1f22000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000022000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000fe13670e874 in ?? ()
#17 0x00007f05b38b4380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b38b4350 in ?? ()
#20 0x3fb97682f1f22000 in ?? ()
#21 0x00000000b38b43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb97682f1f22000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16848):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b58ad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f05b58ad2d0 in ?? ()
#4 0x00007f05b58ad4e0 in ?? ()
#5 0x00007f05b58ad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f05b58ad3a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb956bbb4e4e000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000f0000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000fe136b0da74 in ?? ()
#17 0x00007f05b58ad380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16845):
#0 0x00007f05beaa3cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f05b72b7788 in ?? ()
#3 0x0000602000015a10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16844):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b6ab6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f05b6ab6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16843):
#0 0x00007f05c5d4a9e2 in ?? ()
#1 0x00007f05b62b3b60 in ?? ()
#2 0x00007f05b62b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f05b62b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f05b62b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f05b62b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f05cb66ef0c in ?? ()
#12 0x00007f05cb65e7b9 in ?? ()
#13 0x00007f05cb662da6 in ?? ()
#14 0x00007f05cb6677e0 in ?? ()
#15 0x00007f05cb667609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f05c295d0be in ?? ()
#18 0x00007f05c5d406db in ?? ()
#19 0x00000fe136c4e67c in ?? ()
#20 0x00007f05b62b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16837):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b82b8c40 in ?? ()
#2 0x00000000000003e8 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f05b82b8c30 in ?? ()
#6 0x00000000000007d0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16836):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16835):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16834):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f05b9abc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16832):
#0 0x00007f05c5d4ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:07.464926 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 4 with UUID 18452380c023464488c18995b91fa513 and pid 16694
************************ BEGIN STACKS **************************
[New LWP 16695]
[New LWP 16696]
[New LWP 16697]
[New LWP 16698]
[New LWP 16704]
[New LWP 16705]
[New LWP 16706]
[New LWP 16709]
[New LWP 16710]
[New LWP 16711]
[New LWP 16712]
[New LWP 16713]
[New LWP 16714]
[New LWP 16715]
[New LWP 16716]
[New LWP 16717]
[New LWP 16718]
[New LWP 16719]
[New LWP 16720]
[New LWP 16721]
[New LWP 16722]
[New LWP 16723]
[New LWP 16724]
[New LWP 16725]
[New LWP 16726]
[New LWP 16727]
[New LWP 16728]
[New LWP 16729]
[New LWP 16730]
[New LWP 16731]
[New LWP 16732]
[New LWP 16733]
[New LWP 16734]
[New LWP 16735]
[New LWP 16736]
[New LWP 16737]
[New LWP 16738]
[New LWP 16739]
[New LWP 16740]
[New LWP 16741]
[New LWP 16742]
[New LWP 16743]
[New LWP 16744]
[New LWP 16745]
[New LWP 16746]
[New LWP 16747]
[New LWP 16748]
[New LWP 16749]
[New LWP 16750]
[New LWP 16751]
[New LWP 16752]
[New LWP 16753]
[New LWP 16754]
[New LWP 16755]
[New LWP 16756]
[New LWP 16757]
[New LWP 16758]
[New LWP 16759]
[New LWP 16760]
[New LWP 16761]
[New LWP 16762]
[New LWP 16763]
[New LWP 16764]
[New LWP 16765]
[New LWP 16766]
[New LWP 16767]
[New LWP 16768]
[New LWP 16769]
[New LWP 16770]
[New LWP 16771]
[New LWP 16772]
[New LWP 16773]
[New LWP 16774]
[New LWP 16775]
[New LWP 16776]
[New LWP 16777]
[New LWP 16778]
[New LWP 16779]
[New LWP 16780]
[New LWP 16781]
[New LWP 16782]
[New LWP 16784]
[New LWP 16785]
[New LWP 16786]
[New LWP 16787]
[New LWP 16788]
[New LWP 16789]
[New LWP 16790]
[New LWP 16791]
[New LWP 16792]
[New LWP 16793]
[New LWP 16794]
[New LWP 16795]
[New LWP 16796]
[New LWP 16797]
[New LWP 16799]
[New LWP 16801]
[New LWP 16802]
[New LWP 16803]
[New LWP 16804]
[New LWP 16805]
[New LWP 16806]
[New LWP 16807]
[New LWP 16808]
[New LWP 16809]
[New LWP 16810]
[New LWP 16811]
[New LWP 16812]
[New LWP 16813]
[New LWP 16814]
[New LWP 16815]
[New LWP 16816]
[New LWP 16817]
[New LWP 16818]
[New LWP 16819]
[New LWP 16820]
[New LWP 16821]
[New LWP 16822]
[New LWP 16823]
[New LWP 16824]
[New LWP 16825]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fe24f60ed50 in ?? ()
Id Target Id Frame
* 1 LWP 16694 "kudu" 0x00007fe24f60ed50 in ?? ()
2 LWP 16695 "kudu" 0x00007fe24f60afb9 in ?? ()
3 LWP 16696 "kudu" 0x00007fe24f60afb9 in ?? ()
4 LWP 16697 "kudu" 0x00007fe24f60afb9 in ?? ()
5 LWP 16698 "kernel-watcher-" 0x00007fe24f60afb9 in ?? ()
6 LWP 16704 "ntp client-1670" 0x00007fe24f60e9e2 in ?? ()
7 LWP 16705 "file cache-evic" 0x00007fe24f60afb9 in ?? ()
8 LWP 16706 "sq_acceptor" 0x00007fe248367cb9 in ?? ()
9 LWP 16709 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
10 LWP 16710 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
11 LWP 16711 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
12 LWP 16712 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
13 LWP 16713 "MaintenanceMgr " 0x00007fe24f60aad3 in ?? ()
14 LWP 16714 "txn-status-mana" 0x00007fe24f60afb9 in ?? ()
15 LWP 16715 "collect_and_rem" 0x00007fe24f60afb9 in ?? ()
16 LWP 16716 "tc-session-exp-" 0x00007fe24f60afb9 in ?? ()
17 LWP 16717 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
18 LWP 16718 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
19 LWP 16719 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
20 LWP 16720 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
21 LWP 16721 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
22 LWP 16722 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
23 LWP 16723 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
24 LWP 16724 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
25 LWP 16725 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
26 LWP 16726 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
27 LWP 16727 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
28 LWP 16728 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
29 LWP 16729 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
30 LWP 16730 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
31 LWP 16731 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
32 LWP 16732 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
33 LWP 16733 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
34 LWP 16734 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
35 LWP 16735 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
36 LWP 16736 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
37 LWP 16737 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
38 LWP 16738 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
39 LWP 16739 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
40 LWP 16740 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
41 LWP 16741 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
42 LWP 16742 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
43 LWP 16743 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
44 LWP 16744 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
45 LWP 16745 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
46 LWP 16746 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
47 LWP 16747 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
48 LWP 16748 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
49 LWP 16749 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
50 LWP 16750 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
51 LWP 16751 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
52 LWP 16752 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
53 LWP 16753 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
54 LWP 16754 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
55 LWP 16755 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
56 LWP 16756 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
57 LWP 16757 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
58 LWP 16758 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
59 LWP 16759 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
60 LWP 16760 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
61 LWP 16761 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
62 LWP 16762 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
63 LWP 16763 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
64 LWP 16764 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
65 LWP 16765 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
66 LWP 16766 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
67 LWP 16767 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
68 LWP 16768 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
69 LWP 16769 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
70 LWP 16770 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
71 LWP 16771 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
72 LWP 16772 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
73 LWP 16773 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
74 LWP 16774 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
75 LWP 16775 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
76 LWP 16776 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
77 LWP 16777 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
78 LWP 16778 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
79 LWP 16779 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
80 LWP 16780 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
81 LWP 16781 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
82 LWP 16782 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
83 LWP 16784 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
84 LWP 16785 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
85 LWP 16786 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
86 LWP 16787 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
87 LWP 16788 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
88 LWP 16789 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
89 LWP 16790 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
90 LWP 16791 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
91 LWP 16792 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
92 LWP 16793 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
93 LWP 16794 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
94 LWP 16795 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
95 LWP 16796 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
96 LWP 16797 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
97 LWP 16799 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
98 LWP 16801 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
99 LWP 16802 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
100 LWP 16803 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
101 LWP 16804 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
102 LWP 16805 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
103 LWP 16806 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
104 LWP 16807 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
105 LWP 16808 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
106 LWP 16809 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
107 LWP 16810 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
108 LWP 16811 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
109 LWP 16812 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
110 LWP 16813 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
111 LWP 16814 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
112 LWP 16815 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
113 LWP 16816 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
114 LWP 16817 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
115 LWP 16818 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
116 LWP 16819 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
117 LWP 16820 "diag-logger-168" 0x00007fe24f60afb9 in ?? ()
118 LWP 16821 "result-tracker-" 0x00007fe24f60afb9 in ?? ()
119 LWP 16822 "excess-log-dele" 0x00007fe24f60afb9 in ?? ()
120 LWP 16823 "acceptor-16823" 0x00007fe2483760c7 in ?? ()
121 LWP 16824 "heartbeat-16824" 0x00007fe24f60afb9 in ?? ()
122 LWP 16825 "maintenance_sch" 0x00007fe24f60afb9 in ?? ()
Thread 122 (LWP 16825):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000329 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fe204eee550 in ?? ()
#6 0x0000000000000652 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 16824):
#0 0x00007fe24f60afb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fe2057065b0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 16823):
#0 0x00007fe2483760c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007fe205f2dd50 in ?? ()
#3 0x00007fe205f2dd80 in ?? ()
#4 0x00007fe205f2de80 in ?? ()
#5 0x00007fe205f2dd70 in ?? ()
#6 0x00007fe205f2dde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007fe24c200620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 16822):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe206747f00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe9d1fdc90 in ?? ()
#5 0x00007fe206747ef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 16821):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe206f600c0 in ?? ()
#2 0x00000000000000ca in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fe206f600b0 in ?? ()
#6 0x0000000000000194 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 16820):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 16819):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x000000000000001f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007fe207f8fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe207f8fe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe207f8fe70 in ?? ()
#11 0x00000ffc40ff1fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 16818):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007fe2087a7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe2087a7e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe2087a7e70 in ?? ()
#11 0x00000ffc410f4fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 16817):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 16816):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 16815):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 16814):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 16813):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 16812):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 16811):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 16810):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 16809):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 16808):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 16807):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 16806):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 16805):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 16804):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 16803):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 16802):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 16801):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 16799):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 16797):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 16796):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 16795):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 16794):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 16793):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 16792):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 16791):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 16790):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 16789):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 16788):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 16787):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 16786):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 16785):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 16784):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 16782):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 16781):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 16780):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 16779):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 16778):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 16777):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 16776):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007fe21bb48e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe21bb48e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe21bb48e70 in ?? ()
#11 0x00000ffc437691d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 16775):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 16774):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 16773):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 16772):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 16771):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 16770):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 16769):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 16768):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 16767):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 16766):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 16765):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 16764):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 16763):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 16762):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 16761):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 16760):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 16759):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 16758):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 16757):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 16756):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007fe225d29e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe225d29e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe225d29e70 in ?? ()
#11 0x00000ffc44ba53d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 16755):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 16754):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 16753):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 16752):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 16751):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 16750):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 16749):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 16748):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 16747):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 16746):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 16745):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 16744):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 16743):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 16742):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 16741):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 16740):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 16739):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 16738):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 16737):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 16736):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007fe22ff0ae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe22ff0ae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe22ff0ae70 in ?? ()
#11 0x00000ffc45fe15d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 16735):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 16734):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 16733):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 16732):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 16731):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 16730):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 16729):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 16728):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 16727):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 16726):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 16725):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 16724):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 16723):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 16722):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 16721):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 16720):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 16719):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 16718):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16717):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16716):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe239fc3c80 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fe239fc3c70 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16715):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16714):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe23afc51c0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fe23afc51b0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16713):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16712):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23bfc82e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fe23bfc82d0 in ?? ()
#4 0x00007fe23bfc84e0 in ?? ()
#5 0x00007fe23bfc8320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fe23bfc83a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb9895fb68d0000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000080800000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ffcc77f1074 in ?? ()
#17 0x00007fe23bfc8380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23bfc8350 in ?? ()
#20 0x3fb9895fb68d0000 in ?? ()
#21 0x000000003bfc83a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb9895fb68d0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16711):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23c7c92e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fe23c7c92d0 in ?? ()
#4 0x00007fe23c7c94e0 in ?? ()
#5 0x00007fe23c7c9320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fe23c7c93a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb98a38391aa000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000038800000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffcc78f1274 in ?? ()
#17 0x00007fe23c7c9380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23c7c9350 in ?? ()
#20 0x3fb98a38391aa000 in ?? ()
#21 0x000000003c7c93a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb98a38391aa000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16710):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23cfca2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fe23cfca2d0 in ?? ()
#4 0x00007fe23cfca4e0 in ?? ()
#5 0x00007fe23cfca320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fe23cfca3a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb982d359b6c000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000003e000000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ffcc79f1474 in ?? ()
#17 0x00007fe23cfca380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23cfca350 in ?? ()
#20 0x3fb982d359b6c000 in ?? ()
#21 0x000000003cfca3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb982d359b6c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16709):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23ebb32e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fe23ebb32d0 in ?? ()
#4 0x00007fe23ebb34e0 in ?? ()
#5 0x00007fe23ebb3320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fe23ebb33a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb9646078a9e000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000041000000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffcc7d6e674 in ?? ()
#17 0x00007fe23ebb3380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16706):
#0 0x00007fe248367cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fe240bb7788 in ?? ()
#3 0x0000602000017cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16705):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe2403b6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fe2403b6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16704):
#0 0x00007fe24f60e9e2 in ?? ()
#1 0x00007fe23f3b2b60 in ?? ()
#2 0x00007fe23f3b2c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fe23f3b2bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fe23f3b2c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fe23f3b2c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fe254f32f0c in ?? ()
#12 0x00007fe254f227b9 in ?? ()
#13 0x00007fe254f26da6 in ?? ()
#14 0x00007fe254f2b7e0 in ?? ()
#15 0x00007fe254f2b609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fe24c2210be in ?? ()
#18 0x00007fe24f6046db in ?? ()
#19 0x00000ffcc7e6e47c in ?? ()
#20 0x00007fe23f3b2400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16698):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe241bb8c40 in ?? ()
#2 0x00000000000003f7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fe241bb8c30 in ?? ()
#6 0x00000000000007ee in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16697):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16696):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16695):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fe2433bc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16694):
#0 0x00007fe24f60ed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:08.285482 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 5 with UUID edf2bd29095149dd9e4592ae2870d4ee and pid 18094
************************ BEGIN STACKS **************************
[New LWP 18097]
[New LWP 18098]
[New LWP 18099]
[New LWP 18100]
[New LWP 18106]
[New LWP 18107]
[New LWP 18108]
[New LWP 18111]
[New LWP 18112]
[New LWP 18113]
[New LWP 18114]
[New LWP 18115]
[New LWP 18116]
[New LWP 18118]
[New LWP 18119]
[New LWP 18120]
[New LWP 18121]
[New LWP 18122]
[New LWP 18123]
[New LWP 18124]
[New LWP 18125]
[New LWP 18126]
[New LWP 18127]
[New LWP 18128]
[New LWP 18129]
[New LWP 18130]
[New LWP 18131]
[New LWP 18132]
[New LWP 18133]
[New LWP 18134]
[New LWP 18135]
[New LWP 18136]
[New LWP 18137]
[New LWP 18138]
[New LWP 18139]
[New LWP 18140]
[New LWP 18141]
[New LWP 18142]
[New LWP 18143]
[New LWP 18144]
[New LWP 18145]
[New LWP 18146]
[New LWP 18147]
[New LWP 18148]
[New LWP 18149]
[New LWP 18150]
[New LWP 18151]
[New LWP 18152]
[New LWP 18153]
[New LWP 18154]
[New LWP 18155]
[New LWP 18156]
[New LWP 18157]
[New LWP 18158]
[New LWP 18159]
[New LWP 18160]
[New LWP 18161]
[New LWP 18162]
[New LWP 18163]
[New LWP 18164]
[New LWP 18165]
[New LWP 18166]
[New LWP 18167]
[New LWP 18168]
[New LWP 18169]
[New LWP 18170]
[New LWP 18171]
[New LWP 18172]
[New LWP 18173]
[New LWP 18174]
[New LWP 18175]
[New LWP 18176]
[New LWP 18177]
[New LWP 18179]
[New LWP 18180]
[New LWP 18181]
[New LWP 18182]
[New LWP 18186]
[New LWP 18187]
[New LWP 18188]
[New LWP 18189]
[New LWP 18190]
[New LWP 18191]
[New LWP 18192]
[New LWP 18193]
[New LWP 18194]
[New LWP 18195]
[New LWP 18196]
[New LWP 18197]
[New LWP 18198]
[New LWP 18199]
[New LWP 18200]
[New LWP 18201]
[New LWP 18202]
[New LWP 18203]
[New LWP 18205]
[New LWP 18206]
[New LWP 18207]
[New LWP 18208]
[New LWP 18209]
[New LWP 18210]
[New LWP 18211]
[New LWP 18212]
[New LWP 18213]
[New LWP 18214]
[New LWP 18215]
[New LWP 18216]
[New LWP 18217]
[New LWP 18218]
[New LWP 18219]
[New LWP 18220]
[New LWP 18221]
[New LWP 18222]
[New LWP 18223]
[New LWP 18224]
[New LWP 18225]
[New LWP 18226]
[New LWP 18227]
[New LWP 18228]
[New LWP 18229]
[New LWP 18230]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f91bfd7dd50 in ?? ()
Id Target Id Frame
* 1 LWP 18094 "kudu" 0x00007f91bfd7dd50 in ?? ()
2 LWP 18097 "kudu" 0x00007f91bfd79fb9 in ?? ()
3 LWP 18098 "kudu" 0x00007f91bfd79fb9 in ?? ()
4 LWP 18099 "kudu" 0x00007f91bfd79fb9 in ?? ()
5 LWP 18100 "kernel-watcher-" 0x00007f91bfd79fb9 in ?? ()
6 LWP 18106 "ntp client-1810" 0x00007f91bfd7d9e2 in ?? ()
7 LWP 18107 "file cache-evic" 0x00007f91bfd79fb9 in ?? ()
8 LWP 18108 "sq_acceptor" 0x00007f91b8ad6cb9 in ?? ()
9 LWP 18111 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
10 LWP 18112 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
11 LWP 18113 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
12 LWP 18114 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
13 LWP 18115 "MaintenanceMgr " 0x00007f91bfd79ad3 in ?? ()
14 LWP 18116 "txn-status-mana" 0x00007f91bfd79fb9 in ?? ()
15 LWP 18118 "collect_and_rem" 0x00007f91bfd79fb9 in ?? ()
16 LWP 18119 "tc-session-exp-" 0x00007f91bfd79fb9 in ?? ()
17 LWP 18120 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
18 LWP 18121 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
19 LWP 18122 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
20 LWP 18123 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
21 LWP 18124 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
22 LWP 18125 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
23 LWP 18126 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
24 LWP 18127 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
25 LWP 18128 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
26 LWP 18129 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
27 LWP 18130 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
28 LWP 18131 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
29 LWP 18132 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
30 LWP 18133 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
31 LWP 18134 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
32 LWP 18135 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
33 LWP 18136 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
34 LWP 18137 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
35 LWP 18138 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
36 LWP 18139 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
37 LWP 18140 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
38 LWP 18141 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
39 LWP 18142 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
40 LWP 18143 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
41 LWP 18144 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
42 LWP 18145 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
43 LWP 18146 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
44 LWP 18147 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
45 LWP 18148 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
46 LWP 18149 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
47 LWP 18150 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
48 LWP 18151 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
49 LWP 18152 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
50 LWP 18153 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
51 LWP 18154 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
52 LWP 18155 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
53 LWP 18156 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
54 LWP 18157 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
55 LWP 18158 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
56 LWP 18159 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
57 LWP 18160 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
58 LWP 18161 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
59 LWP 18162 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
60 LWP 18163 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
61 LWP 18164 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
62 LWP 18165 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
63 LWP 18166 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
64 LWP 18167 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
65 LWP 18168 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
66 LWP 18169 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
67 LWP 18170 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
68 LWP 18171 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
69 LWP 18172 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
70 LWP 18173 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
71 LWP 18174 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
72 LWP 18175 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
73 LWP 18176 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
74 LWP 18177 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
75 LWP 18179 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
76 LWP 18180 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
77 LWP 18181 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
78 LWP 18182 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
79 LWP 18186 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
80 LWP 18187 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
81 LWP 18188 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
82 LWP 18189 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
83 LWP 18190 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
84 LWP 18191 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
85 LWP 18192 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
86 LWP 18193 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
87 LWP 18194 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
88 LWP 18195 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
89 LWP 18196 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
90 LWP 18197 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
91 LWP 18198 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
92 LWP 18199 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
93 LWP 18200 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
94 LWP 18201 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
95 LWP 18202 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
96 LWP 18203 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
97 LWP 18205 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
98 LWP 18206 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
99 LWP 18207 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
100 LWP 18208 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
101 LWP 18209 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
102 LWP 18210 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
103 LWP 18211 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
104 LWP 18212 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
105 LWP 18213 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
106 LWP 18214 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
107 LWP 18215 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
108 LWP 18216 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
109 LWP 18217 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
110 LWP 18218 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
111 LWP 18219 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
112 LWP 18220 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
113 LWP 18221 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
114 LWP 18222 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
115 LWP 18223 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
116 LWP 18224 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
117 LWP 18225 "diag-logger-182" 0x00007f91bfd79fb9 in ?? ()
118 LWP 18226 "result-tracker-" 0x00007f91bfd79fb9 in ?? ()
119 LWP 18227 "excess-log-dele" 0x00007f91bfd79fb9 in ?? ()
120 LWP 18228 "acceptor-18228" 0x00007f91b8ae50c7 in ?? ()
121 LWP 18229 "heartbeat-18229" 0x00007f91bfd79fb9 in ?? ()
122 LWP 18230 "maintenance_sch" 0x00007f91bfd79fb9 in ?? ()
Thread 122 (LWP 18230):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000228 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f91744cb550 in ?? ()
#6 0x0000000000000450 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18229):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x000000000000008a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f9174ce35b0 in ?? ()
#6 0x0000000000000114 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18228):
#0 0x00007f91b8ae50c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f917550ad50 in ?? ()
#3 0x00007f917550ad80 in ?? ()
#4 0x00007f917550ae80 in ?? ()
#5 0x00007f917550ad70 in ?? ()
#6 0x00007f917550ade0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f91bc96f620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18227):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f9175d23f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffde77dcef0 in ?? ()
#5 0x00007f9175d23ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18226):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f917653c0c0 in ?? ()
#2 0x000000000000008a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f917653c0b0 in ?? ()
#6 0x0000000000000114 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18225):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18224):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000020 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00032ffc8 in ?? ()
#4 0x00007f917756ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f917756ce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 18223):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00031ffd8 in ?? ()
#4 0x00007f9177d84e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9177d84e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 18222):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18221):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18220):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18219):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18218):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18217):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18216):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18215):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18214):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18213):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18212):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18211):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18210):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18209):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18208):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18207):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18206):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18205):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18203):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18202):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18201):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18200):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18199):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18198):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18197):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18196):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18195):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18194):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18193):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18192):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18191):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18190):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18189):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18188):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18187):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18186):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18182):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18181):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18180):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x00000000000011e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001c006c in ?? ()
#4 0x00007f918c277e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918c277e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001c0020 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f918c277e70 in ?? ()
#11 0x00000ff23184efd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18179):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001440 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001b6878 in ?? ()
#4 0x00007f918ca8fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918ca8fe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 18177):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000042f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001b007c in ?? ()
#4 0x00007f918d2b0e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918d2b0e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001b0030 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f918d2b0e70 in ?? ()
#11 0x00000ff231a561d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 18176):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18175):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18174):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18173):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18172):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18171):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18170):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18169):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18168):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18167):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18166):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18165):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18164):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18163):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18162):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18161):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18160):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18159):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000000d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00013682c in ?? ()
#4 0x00007f9196458e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9196458e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001367e0 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f9196458e70 in ?? ()
#11 0x00000ff232c8b1d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18158):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18157):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18156):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18155):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18154):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18153):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18152):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18151):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18150):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18149):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18148):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18147):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18146):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18145):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18144):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18143):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18142):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18141):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18140):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18139):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001b90 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000affd8 in ?? ()
#4 0x00007f919fe21e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f919fe21e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 18138):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001a3e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007f91a0639e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a0639e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 18137):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001594 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f91a0e51e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a0e51e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 33 (LWP 18136):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000114a in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000967f8 in ?? ()
#4 0x00007f91a1669e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a1669e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 18135):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001495 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008fffc in ?? ()
#4 0x00007f91a1e81e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a1e81e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008ffb0 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f91a1e81e70 in ?? ()
#11 0x00000ff2343d03d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 31 (LWP 18134):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000612 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008d008 in ?? ()
#4 0x00007f91a2699e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a2699e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 30 (LWP 18133):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18132):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18131):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18130):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18129):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18128):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18127):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18126):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18125):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18124):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18123):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18122):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18121):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18120):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18119):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91aa036c80 in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f91aa036c70 in ?? ()
#6 0x000000000000001c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18118):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18116):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91ab8af1c0 in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f91ab8af1b0 in ?? ()
#6 0x000000000000001c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18115):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18114):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ac8b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f91ac8b22d0 in ?? ()
#4 0x00007f91ac8b24e0 in ?? ()
#5 0x00007f91ac8b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f91ac8b23a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb95d6275a80000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000da800000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff2b590e474 in ?? ()
#17 0x00007f91ac8b2380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ac8b2350 in ?? ()
#20 0x3fb95d6275a80000 in ?? ()
#21 0x00000000ac8b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb95d6275a80000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18113):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ad0b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f91ad0b32d0 in ?? ()
#4 0x00007f91ad0b34e0 in ?? ()
#5 0x00007f91ad0b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f91ad0b33a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb96c4f2f718000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000006b000000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2b5a0e674 in ?? ()
#17 0x00007f91ad0b3380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ad0b3350 in ?? ()
#20 0x3fb96c4f2f718000 in ?? ()
#21 0x00000000ad0b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96c4f2f718000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18112):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ad8b42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f91ad8b42d0 in ?? ()
#4 0x00007f91ad8b44e0 in ?? ()
#5 0x00007f91ad8b4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f91ad8b43a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb96195ba67c000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x00000000a2000000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2b5b0e874 in ?? ()
#17 0x00007f91ad8b4380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ad8b4350 in ?? ()
#20 0x3fb96195ba67c000 in ?? ()
#21 0x00000000ad8b43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb96195ba67c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18111):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91af8ad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f91af8ad2d0 in ?? ()
#4 0x00007f91af8ad4e0 in ?? ()
#5 0x00007f91af8ad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f91af8ad3a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb95d6594850000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000029800000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318dbf in ?? ()
#16 0x00000ff2b5f0da74 in ?? ()
#17 0x00007f91af8ad380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18108):
#0 0x00007f91b8ad6cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f91b12b7788 in ?? ()
#3 0x0000602000017cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18107):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91b0ab6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f91b0ab6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18106):
#0 0x00007f91bfd7d9e2 in ?? ()
#1 0x00007f91b02b3b60 in ?? ()
#2 0x00007f91b02b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f91b02b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f91b02b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f91b02b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f91c56a1f0c in ?? ()
#12 0x00007f91c56917b9 in ?? ()
#13 0x00007f91c5695da6 in ?? ()
#14 0x00007f91c569a7e0 in ?? ()
#15 0x00007f91c569a609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f91bc9900be in ?? ()
#18 0x00007f91bfd736db in ?? ()
#19 0x00000ff2b604e67c in ?? ()
#20 0x00007f91b02b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18100):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91b22b8c40 in ?? ()
#2 0x00000000000002bc in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f91b22b8c30 in ?? ()
#6 0x0000000000000578 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18099):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18098):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18097):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000006 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f91b3abc410 in ?? ()
#6 0x000000000000000c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18094):
#0 0x00007f91bfd7dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:09.073047 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 6 with UUID 49a14c4ba19447f8851a9a40990a4b91 and pid 16972
************************ BEGIN STACKS **************************
[New LWP 16976]
[New LWP 16977]
[New LWP 16978]
[New LWP 16979]
[New LWP 16985]
[New LWP 16986]
[New LWP 16987]
[New LWP 16991]
[New LWP 16992]
[New LWP 16993]
[New LWP 16994]
[New LWP 16995]
[New LWP 16996]
[New LWP 16997]
[New LWP 16998]
[New LWP 16999]
[New LWP 17000]
[New LWP 17001]
[New LWP 17002]
[New LWP 17003]
[New LWP 17004]
[New LWP 17005]
[New LWP 17006]
[New LWP 17007]
[New LWP 17008]
[New LWP 17009]
[New LWP 17010]
[New LWP 17011]
[New LWP 17012]
[New LWP 17013]
[New LWP 17014]
[New LWP 17015]
[New LWP 17016]
[New LWP 17017]
[New LWP 17019]
[New LWP 17020]
[New LWP 17021]
[New LWP 17022]
[New LWP 17023]
[New LWP 17024]
[New LWP 17025]
[New LWP 17026]
[New LWP 17027]
[New LWP 17028]
[New LWP 17029]
[New LWP 17030]
[New LWP 17031]
[New LWP 17032]
[New LWP 17033]
[New LWP 17034]
[New LWP 17035]
[New LWP 17036]
[New LWP 17037]
[New LWP 17038]
[New LWP 17039]
[New LWP 17040]
[New LWP 17041]
[New LWP 17042]
[New LWP 17043]
[New LWP 17044]
[New LWP 17045]
[New LWP 17046]
[New LWP 17047]
[New LWP 17048]
[New LWP 17049]
[New LWP 17050]
[New LWP 17051]
[New LWP 17052]
[New LWP 17053]
[New LWP 17054]
[New LWP 17055]
[New LWP 17056]
[New LWP 17057]
[New LWP 17058]
[New LWP 17059]
[New LWP 17060]
[New LWP 17061]
[New LWP 17062]
[New LWP 17063]
[New LWP 17064]
[New LWP 17065]
[New LWP 17067]
[New LWP 17068]
[New LWP 17069]
[New LWP 17070]
[New LWP 17071]
[New LWP 17072]
[New LWP 17073]
[New LWP 17074]
[New LWP 17075]
[New LWP 17076]
[New LWP 17077]
[New LWP 17078]
[New LWP 17079]
[New LWP 17081]
[New LWP 17082]
[New LWP 17083]
[New LWP 17084]
[New LWP 17085]
[New LWP 17086]
[New LWP 17087]
[New LWP 17088]
[New LWP 17089]
[New LWP 17090]
[New LWP 17091]
[New LWP 17092]
[New LWP 17093]
[New LWP 17094]
[New LWP 17095]
[New LWP 17096]
[New LWP 17097]
[New LWP 17098]
[New LWP 17099]
[New LWP 17100]
[New LWP 17101]
[New LWP 17102]
[New LWP 17103]
[New LWP 17104]
[New LWP 17105]
[New LWP 17106]
[New LWP 17107]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f9dac861d50 in ?? ()
Id Target Id Frame
* 1 LWP 16972 "kudu" 0x00007f9dac861d50 in ?? ()
2 LWP 16976 "kudu" 0x00007f9dac85dfb9 in ?? ()
3 LWP 16977 "kudu" 0x00007f9dac85dfb9 in ?? ()
4 LWP 16978 "kudu" 0x00007f9dac85dfb9 in ?? ()
5 LWP 16979 "kernel-watcher-" 0x00007f9dac85dfb9 in ?? ()
6 LWP 16985 "ntp client-1698" 0x00007f9dac8619e2 in ?? ()
7 LWP 16986 "file cache-evic" 0x00007f9dac85dfb9 in ?? ()
8 LWP 16987 "sq_acceptor" 0x00007f9da55bacb9 in ?? ()
9 LWP 16991 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
10 LWP 16992 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
11 LWP 16993 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
12 LWP 16994 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
13 LWP 16995 "MaintenanceMgr " 0x00007f9dac85dad3 in ?? ()
14 LWP 16996 "txn-status-mana" 0x00007f9dac85dfb9 in ?? ()
15 LWP 16997 "collect_and_rem" 0x00007f9dac85dfb9 in ?? ()
16 LWP 16998 "tc-session-exp-" 0x00007f9dac85dfb9 in ?? ()
17 LWP 16999 "rpc worker-1699" 0x00007f9dac85dad3 in ?? ()
18 LWP 17000 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
19 LWP 17001 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
20 LWP 17002 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
21 LWP 17003 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
22 LWP 17004 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
23 LWP 17005 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
24 LWP 17006 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
25 LWP 17007 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
26 LWP 17008 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
27 LWP 17009 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
28 LWP 17010 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
29 LWP 17011 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
30 LWP 17012 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
31 LWP 17013 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
32 LWP 17014 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
33 LWP 17015 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
34 LWP 17016 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
35 LWP 17017 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
36 LWP 17019 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
37 LWP 17020 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
38 LWP 17021 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
39 LWP 17022 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
40 LWP 17023 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
41 LWP 17024 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
42 LWP 17025 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
43 LWP 17026 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
44 LWP 17027 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
45 LWP 17028 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
46 LWP 17029 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
47 LWP 17030 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
48 LWP 17031 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
49 LWP 17032 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
50 LWP 17033 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
51 LWP 17034 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
52 LWP 17035 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
53 LWP 17036 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
54 LWP 17037 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
55 LWP 17038 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
56 LWP 17039 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
57 LWP 17040 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
58 LWP 17041 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
59 LWP 17042 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
60 LWP 17043 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
61 LWP 17044 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
62 LWP 17045 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
63 LWP 17046 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
64 LWP 17047 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
65 LWP 17048 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
66 LWP 17049 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
67 LWP 17050 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
68 LWP 17051 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
69 LWP 17052 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
70 LWP 17053 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
71 LWP 17054 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
72 LWP 17055 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
73 LWP 17056 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
74 LWP 17057 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
75 LWP 17058 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
76 LWP 17059 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
77 LWP 17060 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
78 LWP 17061 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
79 LWP 17062 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
80 LWP 17063 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
81 LWP 17064 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
82 LWP 17065 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
83 LWP 17067 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
84 LWP 17068 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
85 LWP 17069 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
86 LWP 17070 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
87 LWP 17071 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
88 LWP 17072 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
89 LWP 17073 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
90 LWP 17074 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
91 LWP 17075 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
92 LWP 17076 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
93 LWP 17077 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
94 LWP 17078 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
95 LWP 17079 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
96 LWP 17081 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
97 LWP 17082 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
98 LWP 17083 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
99 LWP 17084 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
100 LWP 17085 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
101 LWP 17086 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
102 LWP 17087 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
103 LWP 17088 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
104 LWP 17089 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
105 LWP 17090 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
106 LWP 17091 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
107 LWP 17092 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
108 LWP 17093 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
109 LWP 17094 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
110 LWP 17095 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
111 LWP 17096 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
112 LWP 17097 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
113 LWP 17098 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
114 LWP 17099 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
115 LWP 17100 "rpc worker-1710" 0x00007f9dac85dad3 in ?? ()
116 LWP 17101 "rpc worker-1710" 0x00007f9dac85dad3 in ?? ()
117 LWP 17102 "diag-logger-171" 0x00007f9dac85dfb9 in ?? ()
118 LWP 17103 "result-tracker-" 0x00007f9dac85dfb9 in ?? ()
119 LWP 17104 "excess-log-dele" 0x00007f9dac85dfb9 in ?? ()
120 LWP 17105 "acceptor-17105" 0x00007f9da55c90c7 in ?? ()
121 LWP 17106 "heartbeat-17106" 0x00007f9dac85dfb9 in ?? ()
122 LWP 17107 "maintenance_sch" 0x00007f9dac85dfb9 in ?? ()
Thread 122 (LWP 17107):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000318 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f9d62141550 in ?? ()
#6 0x0000000000000630 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17106):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f9d629595b0 in ?? ()
#6 0x000000000000018a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17105):
#0 0x00007f9da55c90c7 in ?? ()
#1 0x00006140000214a8 in ?? ()
#2 0x00007f9d63180d50 in ?? ()
#3 0x00007f9d63180d80 in ?? ()
#4 0x00007f9d63180e80 in ?? ()
#5 0x00007f9d63180d70 in ?? ()
#6 0x00007f9d63180de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f9da9453620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17104):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d6399af00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffd8b8e6b70 in ?? ()
#5 0x00007f9d6399aef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17103):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d641b30c0 in ?? ()
#2 0x00000000000000c6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f9d641b30b0 in ?? ()
#6 0x000000000000018c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17102):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17101):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000023 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f9d651e2e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d651e2e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d651e2e70 in ?? ()
#11 0x00000ff3aca3c5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 17100):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000021 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f9d659fae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d659fae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d659fae70 in ?? ()
#11 0x00000ff3acb3f5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 17099):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17098):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17097):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17096):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17095):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17094):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17093):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17092):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17091):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17090):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17089):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17088):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17087):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17086):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17085):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17084):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17083):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17082):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17081):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17079):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17078):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17077):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17076):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17075):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17074):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17073):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17072):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17071):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17070):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17069):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17068):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17067):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17065):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17064):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17063):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17062):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17061):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17060):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17059):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f9d795b3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d795b3e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d795b3e70 in ?? ()
#11 0x00000ff3af2b67d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 17058):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17057):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17056):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17055):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17054):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17053):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17052):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17051):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17050):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17049):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17048):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17047):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17046):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17045):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17044):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17043):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17042):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17041):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17040):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17039):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f9d83794e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d83794e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d83794e70 in ?? ()
#11 0x00000ff3b06f29d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 17038):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17037):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17036):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17035):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17034):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17033):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17032):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17031):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17030):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17029):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17028):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17027):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17026):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17025):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17024):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17023):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17022):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17021):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17020):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17019):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f9d8d975e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d8d975e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d8d975e70 in ?? ()
#11 0x00000ff3b1b2ebd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 17017):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17016):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17015):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17014):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17013):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17012):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17011):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17010):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17009):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17008):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17007):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17006):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17005):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17004):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17003):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17002):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17001):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17000):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16999):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16998):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d971c3c80 in ?? ()
#2 0x0000000000000013 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f9d971c3c70 in ?? ()
#6 0x0000000000000026 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16997):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16996):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d981c51c0 in ?? ()
#2 0x0000000000000013 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f9d981c51b0 in ?? ()
#6 0x0000000000000026 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16995):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16994):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d991c82e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f9d991c82d0 in ?? ()
#4 0x00007f9d991c84e0 in ?? ()
#5 0x00007f9d991c8320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f9d991c83a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb9589f42710000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x000000008e000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ff433231074 in ?? ()
#17 0x00007f9d991c8380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d991c8350 in ?? ()
#20 0x3fb9589f42710000 in ?? ()
#21 0x00000000991c83a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb9589f42710000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16993):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d999c92e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f9d999c92d0 in ?? ()
#4 0x00007f9d999c94e0 in ?? ()
#5 0x00007f9d999c9320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f9d999c93a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb952282feac000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000024000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ff433331274 in ?? ()
#17 0x00007f9d999c9380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d999c9350 in ?? ()
#20 0x3fb952282feac000 in ?? ()
#21 0x00000000999c93a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb952282feac000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16992):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d9a1ca2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f9d9a1ca2d0 in ?? ()
#4 0x00007f9d9a1ca4e0 in ?? ()
#5 0x00007f9d9a1ca320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f9d9a1ca3a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb96d1093970000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000001a000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ff433431474 in ?? ()
#17 0x00007f9d9a1ca380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d9a1ca350 in ?? ()
#20 0x3fb96d1093970000 in ?? ()
#21 0x000000009a1ca3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb96d1093970000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16991):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d9bdb32e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f9d9bdb32d0 in ?? ()
#4 0x00007f9d9bdb34e0 in ?? ()
#5 0x00007f9d9bdb3320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f9d9bdb33a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb96a165200a000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x000000002d800000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff4337ae674 in ?? ()
#17 0x00007f9d9bdb3380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16987):
#0 0x00007f9da55bacb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f9d9ddb7788 in ?? ()
#3 0x0000602000016790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16986):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d9d5b6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f9d9d5b6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16985):
#0 0x00007f9dac8619e2 in ?? ()
#1 0x00007f9d9c5b2b60 in ?? ()
#2 0x00007f9d9c5b2c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f9d9c5b2bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f9d9c5b2c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f9d9c5b2c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f9db2185f0c in ?? ()
#12 0x00007f9db21757b9 in ?? ()
#13 0x00007f9db2179da6 in ?? ()
#14 0x00007f9db217e7e0 in ?? ()
#15 0x00007f9db217e609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f9da94740be in ?? ()
#18 0x00007f9dac8576db in ?? ()
#19 0x00000ff4338ae47c in ?? ()
#20 0x00007f9d9c5b2400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16979):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d9edb8c40 in ?? ()
#2 0x00000000000003e7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f9d9edb8c30 in ?? ()
#6 0x00000000000007ce in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16978):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16977):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16976):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f9da05bc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16972):
#0 0x00007f9dac861d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:09.879071 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 7 with UUID cc4301cbd2b84d22ab621b2c399022dc and pid 18787
************************ BEGIN STACKS **************************
[New LWP 18789]
[New LWP 18790]
[New LWP 18791]
[New LWP 18792]
[New LWP 18798]
[New LWP 18799]
[New LWP 18800]
[New LWP 18804]
[New LWP 18805]
[New LWP 18806]
[New LWP 18807]
[New LWP 18808]
[New LWP 18809]
[New LWP 18811]
[New LWP 18812]
[New LWP 18813]
[New LWP 18814]
[New LWP 18815]
[New LWP 18816]
[New LWP 18817]
[New LWP 18818]
[New LWP 18819]
[New LWP 18820]
[New LWP 18821]
[New LWP 18822]
[New LWP 18823]
[New LWP 18824]
[New LWP 18825]
[New LWP 18826]
[New LWP 18827]
[New LWP 18828]
[New LWP 18829]
[New LWP 18830]
[New LWP 18831]
[New LWP 18832]
[New LWP 18833]
[New LWP 18834]
[New LWP 18835]
[New LWP 18836]
[New LWP 18837]
[New LWP 18838]
[New LWP 18839]
[New LWP 18840]
[New LWP 18841]
[New LWP 18842]
[New LWP 18843]
[New LWP 18844]
[New LWP 18845]
[New LWP 18846]
[New LWP 18847]
[New LWP 18848]
[New LWP 18849]
[New LWP 18850]
[New LWP 18851]
[New LWP 18852]
[New LWP 18853]
[New LWP 18854]
[New LWP 18855]
[New LWP 18856]
[New LWP 18857]
[New LWP 18858]
[New LWP 18859]
[New LWP 18860]
[New LWP 18861]
[New LWP 18862]
[New LWP 18863]
[New LWP 18864]
[New LWP 18865]
[New LWP 18866]
[New LWP 18867]
[New LWP 18869]
[New LWP 18870]
[New LWP 18871]
[New LWP 18872]
[New LWP 18874]
[New LWP 18875]
[New LWP 18876]
[New LWP 18877]
[New LWP 18878]
[New LWP 18879]
[New LWP 18880]
[New LWP 18881]
[New LWP 18882]
[New LWP 18883]
[New LWP 18884]
[New LWP 18885]
[New LWP 18886]
[New LWP 18887]
[New LWP 18888]
[New LWP 18889]
[New LWP 18890]
[New LWP 18891]
[New LWP 18892]
[New LWP 18893]
[New LWP 18894]
[New LWP 18895]
[New LWP 18896]
[New LWP 18897]
[New LWP 18898]
[New LWP 18899]
[New LWP 18900]
[New LWP 18901]
[New LWP 18902]
[New LWP 18903]
[New LWP 18904]
[New LWP 18905]
[New LWP 18906]
[New LWP 18907]
[New LWP 18908]
[New LWP 18909]
[New LWP 18910]
[New LWP 18911]
[New LWP 18912]
[New LWP 18913]
[New LWP 18914]
[New LWP 18915]
[New LWP 18916]
[New LWP 18917]
[New LWP 18918]
[New LWP 18920]
[New LWP 18923]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f1bd4396d50 in ?? ()
Id Target Id Frame
* 1 LWP 18787 "kudu" 0x00007f1bd4396d50 in ?? ()
2 LWP 18789 "kudu" 0x00007f1bd4392fb9 in ?? ()
3 LWP 18790 "kudu" 0x00007f1bd4392fb9 in ?? ()
4 LWP 18791 "kudu" 0x00007f1bd4392fb9 in ?? ()
5 LWP 18792 "kernel-watcher-" 0x00007f1bd4392fb9 in ?? ()
6 LWP 18798 "ntp client-1879" 0x00007f1bd43969e2 in ?? ()
7 LWP 18799 "file cache-evic" 0x00007f1bd4392fb9 in ?? ()
8 LWP 18800 "sq_acceptor" 0x00007f1bcd0efcb9 in ?? ()
9 LWP 18804 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
10 LWP 18805 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
11 LWP 18806 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
12 LWP 18807 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
13 LWP 18808 "MaintenanceMgr " 0x00007f1bd4392ad3 in ?? ()
14 LWP 18809 "txn-status-mana" 0x00007f1bd4392fb9 in ?? ()
15 LWP 18811 "collect_and_rem" 0x00007f1bd4392fb9 in ?? ()
16 LWP 18812 "tc-session-exp-" 0x00007f1bd4392fb9 in ?? ()
17 LWP 18813 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
18 LWP 18814 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
19 LWP 18815 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
20 LWP 18816 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
21 LWP 18817 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
22 LWP 18818 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
23 LWP 18819 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
24 LWP 18820 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
25 LWP 18821 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
26 LWP 18822 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
27 LWP 18823 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
28 LWP 18824 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
29 LWP 18825 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
30 LWP 18826 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
31 LWP 18827 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
32 LWP 18828 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
33 LWP 18829 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
34 LWP 18830 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
35 LWP 18831 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
36 LWP 18832 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
37 LWP 18833 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
38 LWP 18834 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
39 LWP 18835 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
40 LWP 18836 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
41 LWP 18837 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
42 LWP 18838 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
43 LWP 18839 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
44 LWP 18840 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
45 LWP 18841 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
46 LWP 18842 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
47 LWP 18843 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
48 LWP 18844 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
49 LWP 18845 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
50 LWP 18846 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
51 LWP 18847 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
52 LWP 18848 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
53 LWP 18849 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
54 LWP 18850 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
55 LWP 18851 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
56 LWP 18852 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
57 LWP 18853 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
58 LWP 18854 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
59 LWP 18855 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
60 LWP 18856 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
61 LWP 18857 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
62 LWP 18858 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
63 LWP 18859 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
64 LWP 18860 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
65 LWP 18861 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
66 LWP 18862 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
67 LWP 18863 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
68 LWP 18864 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
69 LWP 18865 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
70 LWP 18866 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
71 LWP 18867 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
72 LWP 18869 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
73 LWP 18870 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
74 LWP 18871 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
75 LWP 18872 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
76 LWP 18874 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
77 LWP 18875 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
78 LWP 18876 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
79 LWP 18877 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
80 LWP 18878 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
81 LWP 18879 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
82 LWP 18880 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
83 LWP 18881 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
84 LWP 18882 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
85 LWP 18883 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
86 LWP 18884 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
87 LWP 18885 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
88 LWP 18886 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
89 LWP 18887 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
90 LWP 18888 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
91 LWP 18889 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
92 LWP 18890 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
93 LWP 18891 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
94 LWP 18892 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
95 LWP 18893 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
96 LWP 18894 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
97 LWP 18895 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
98 LWP 18896 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
99 LWP 18897 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
100 LWP 18898 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
101 LWP 18899 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
102 LWP 18900 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
103 LWP 18901 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
104 LWP 18902 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
105 LWP 18903 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
106 LWP 18904 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
107 LWP 18905 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
108 LWP 18906 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
109 LWP 18907 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
110 LWP 18908 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
111 LWP 18909 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
112 LWP 18910 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
113 LWP 18911 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
114 LWP 18912 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
115 LWP 18913 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
116 LWP 18914 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
117 LWP 18915 "diag-logger-189" 0x00007f1bd4392fb9 in ?? ()
118 LWP 18916 "result-tracker-" 0x00007f1bd4392fb9 in ?? ()
119 LWP 18917 "excess-log-dele" 0x00007f1bd4392fb9 in ?? ()
120 LWP 18918 "acceptor-18918" 0x00007f1bcd0fe0c7 in ?? ()
121 LWP 18920 "heartbeat-18920" 0x00007f1bd4392fb9 in ?? ()
122 LWP 18923 "maintenance_sch" 0x00007f1bd4392fb9 in ?? ()
Thread 122 (LWP 18923):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000126 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f1b88b0d550 in ?? ()
#6 0x000000000000024c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18920):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x000000000000004a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f1b894275b0 in ?? ()
#6 0x0000000000000094 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18918):
#0 0x00007f1bcd0fe0c7 in ?? ()
#1 0x00006140000410a8 in ?? ()
#2 0x00007f1b8a44dd50 in ?? ()
#3 0x00007f1b8a44dd80 in ?? ()
#4 0x00007f1b8a44de80 in ?? ()
#5 0x00007f1b8a44dd70 in ?? ()
#6 0x00007f1b8a44dde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 18917):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1b8ac66f00 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffca3bd0050 in ?? ()
#5 0x00007f1b8ac66ef0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18916):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1b8b48f0c0 in ?? ()
#2 0x0000000000000049 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f1b8b48f0b0 in ?? ()
#6 0x0000000000000092 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18915):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18914):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000017 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002e001c in ?? ()
#4 0x00007f1b8c4bfe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1b8c4bfe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002dffd0 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1b8c4bfe70 in ?? ()
#11 0x00000fe371897fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 18913):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000009 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002dd02c in ?? ()
#4 0x00007f1b8ccd7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1b8ccd7e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002dcfe0 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1b8ccd7e70 in ?? ()
#11 0x00000fe37199afd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 18912):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18911):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18910):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18909):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18908):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18907):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18906):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18905):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18904):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18903):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18902):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18901):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18900):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18899):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18898):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18897):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18896):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18895):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18894):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18893):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18892):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18891):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18890):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18889):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18888):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18887):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18886):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18885):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18884):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18883):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18882):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18881):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18880):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18879):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18878):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18877):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18876):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18875):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18874):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001d005c in ?? ()
#4 0x00007f1ba0880e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba0880e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001d0010 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1ba0880e70 in ?? ()
#11 0x00000fe3741101d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18872):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000285 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001cd06c in ?? ()
#4 0x00007f1ba1098e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba1098e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001cd020 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1ba1098e70 in ?? ()
#11 0x00000fe3742131d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 18871):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18870):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18869):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18867):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18866):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18865):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18864):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18863):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18862):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18861):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18860):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18859):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18858):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18857):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18856):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18855):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18854):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18853):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18852):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000146818 in ?? ()
#4 0x00007f1ba9a31e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba9a31e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 18851):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18850):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18849):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18848):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18847):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18846):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18845):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18844):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18843):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18842):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18841):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18840):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18839):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18838):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18837):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18836):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18835):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18834):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18833):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18832):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x000000000000000b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000bffcc in ?? ()
#4 0x00007f1bb3c12e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1bb3c12e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000bff80 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1bb3c12e70 in ?? ()
#11 0x00000fe3767825d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18831):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18830):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18829):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18828):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18827):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18826):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18825):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18824):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18823):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18822):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18821):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18820):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18819):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18818):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18817):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18816):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18815):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18814):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18813):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18812):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bbde17c80 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f1bbde17c70 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18811):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18809):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bbf6ae1c0 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f1bbf6ae1b0 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18808):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18807):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc06b12e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f1bc06b12d0 in ?? ()
#4 0x00007f1bc06b14e0 in ?? ()
#5 0x00007f1bc06b1320 in ?? ()
#6 0x0000614000041698 in ?? ()
#7 0x00007f1bc06b13a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb2178b43e2e000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x000000006b000000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000fe3f80ce274 in ?? ()
#17 0x00007f1bc06b1380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc06b1350 in ?? ()
#20 0x3fb2178b43e2e000 in ?? ()
#21 0x00000000c06b13a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000041698 in ?? ()
#24 0x3fb2178b43e2e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18806):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc0eb22e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f1bc0eb22d0 in ?? ()
#4 0x00007f1bc0eb24e0 in ?? ()
#5 0x00007f1bc0eb2320 in ?? ()
#6 0x0000614000041498 in ?? ()
#7 0x00007f1bc0eb23a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb986d252eca000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000035000000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d20 in ?? ()
#16 0x00000fe3f81ce474 in ?? ()
#17 0x00007f1bc0eb2380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc0eb2350 in ?? ()
#20 0x3fb986d252eca000 in ?? ()
#21 0x00000000c0eb23a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000041498 in ?? ()
#24 0x3fb986d252eca000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18805):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc16b32e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f1bc16b32d0 in ?? ()
#4 0x00007f1bc16b34e0 in ?? ()
#5 0x00007f1bc16b3320 in ?? ()
#6 0x0000614000041298 in ?? ()
#7 0x00007f1bc16b33a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb4a3fc49150000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000008e00000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000fe3f82ce674 in ?? ()
#17 0x00007f1bc16b3380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc16b3350 in ?? ()
#20 0x3fb4a3fc49150000 in ?? ()
#21 0x00000000c16b33a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000041298 in ?? ()
#24 0x3fb4a3fc49150000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18804):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc36ac2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f1bc36ac2d0 in ?? ()
#4 0x00007f1bc36ac4e0 in ?? ()
#5 0x00007f1bc36ac320 in ?? ()
#6 0x0000614000041098 in ?? ()
#7 0x00007f1bc36ac3a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb98ae576cf8000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000cb000000 in ?? ()
#14 0x0000000300000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x3eb0000000000000 in ?? ()
#17 0x00007f1bc36ac380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18800):
#0 0x00007f1bcd0efcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f1bc58b7788 in ?? ()
#3 0x00006020000179d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18799):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bc50b6080 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f1bc50b6070 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18798):
#0 0x00007f1bd43969e2 in ?? ()
#1 0x00007f1bc48b3b60 in ?? ()
#2 0x00007f1bc48b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f1bc48b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f1bc48b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f1bc48b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f1bd9cbaf0c in ?? ()
#12 0x00007f1bd9caa7b9 in ?? ()
#13 0x00007f1bd9caeda6 in ?? ()
#14 0x00007f1bd9cb37e0 in ?? ()
#15 0x00007f1bd9cb3609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f1bd0fa90be in ?? ()
#18 0x00007f1bd438c6db in ?? ()
#19 0x00000fe3f890e67c in ?? ()
#20 0x00007f1bc48b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18792):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bc68b8c40 in ?? ()
#2 0x0000000000000177 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f1bc68b8c30 in ?? ()
#6 0x00000000000002ee in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18791):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18790):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18789):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f1bc80bc410 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18787):
#0 0x00007f1bd4396d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:10.675465 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 8 with UUID 4c1123e925164e978604ac78771cb3ad and pid 18929
************************ BEGIN STACKS **************************
[New LWP 18933]
[New LWP 18934]
[New LWP 18935]
[New LWP 18936]
[New LWP 18942]
[New LWP 18943]
[New LWP 18944]
[New LWP 18947]
[New LWP 18948]
[New LWP 18949]
[New LWP 18950]
[New LWP 18951]
[New LWP 18952]
[New LWP 18953]
[New LWP 18954]
[New LWP 18955]
[New LWP 18956]
[New LWP 18957]
[New LWP 18958]
[New LWP 18959]
[New LWP 18960]
[New LWP 18961]
[New LWP 18962]
[New LWP 18963]
[New LWP 18964]
[New LWP 18965]
[New LWP 18966]
[New LWP 18967]
[New LWP 18968]
[New LWP 18974]
[New LWP 18975]
[New LWP 18976]
[New LWP 18977]
[New LWP 18978]
[New LWP 18979]
[New LWP 18980]
[New LWP 18981]
[New LWP 18982]
[New LWP 18983]
[New LWP 18984]
[New LWP 18985]
[New LWP 18987]
[New LWP 18988]
[New LWP 18989]
[New LWP 18990]
[New LWP 18991]
[New LWP 18992]
[New LWP 18993]
[New LWP 18994]
[New LWP 18995]
[New LWP 18996]
[New LWP 18997]
[New LWP 18998]
[New LWP 18999]
[New LWP 19000]
[New LWP 19001]
[New LWP 19002]
[New LWP 19003]
[New LWP 19004]
[New LWP 19005]
[New LWP 19006]
[New LWP 19007]
[New LWP 19008]
[New LWP 19009]
[New LWP 19010]
[New LWP 19011]
[New LWP 19012]
[New LWP 19013]
[New LWP 19014]
[New LWP 19015]
[New LWP 19016]
[New LWP 19017]
[New LWP 19018]
[New LWP 19019]
[New LWP 19020]
[New LWP 19021]
[New LWP 19022]
[New LWP 19023]
[New LWP 19024]
[New LWP 19025]
[New LWP 19026]
[New LWP 19027]
[New LWP 19028]
[New LWP 19029]
[New LWP 19030]
[New LWP 19031]
[New LWP 19032]
[New LWP 19033]
[New LWP 19034]
[New LWP 19035]
[New LWP 19036]
[New LWP 19037]
[New LWP 19038]
[New LWP 19039]
[New LWP 19040]
[New LWP 19041]
[New LWP 19042]
[New LWP 19043]
[New LWP 19044]
[New LWP 19045]
[New LWP 19046]
[New LWP 19047]
[New LWP 19048]
[New LWP 19049]
[New LWP 19050]
[New LWP 19051]
[New LWP 19052]
[New LWP 19053]
[New LWP 19054]
[New LWP 19055]
[New LWP 19056]
[New LWP 19057]
[New LWP 19058]
[New LWP 19059]
[New LWP 19060]
[New LWP 19061]
[New LWP 19062]
[New LWP 19063]
[New LWP 19064]
[New LWP 19065]
[New LWP 19068]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fe5e8a63d50 in ?? ()
Id Target Id Frame
* 1 LWP 18929 "kudu" 0x00007fe5e8a63d50 in ?? ()
2 LWP 18933 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
3 LWP 18934 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
4 LWP 18935 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
5 LWP 18936 "kernel-watcher-" 0x00007fe5e8a5ffb9 in ?? ()
6 LWP 18942 "ntp client-1894" 0x00007fe5e8a639e2 in ?? ()
7 LWP 18943 "file cache-evic" 0x00007fe5e8a5ffb9 in ?? ()
8 LWP 18944 "sq_acceptor" 0x00007fe5e17bccb9 in ?? ()
9 LWP 18947 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
10 LWP 18948 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
11 LWP 18949 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
12 LWP 18950 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
13 LWP 18951 "MaintenanceMgr " 0x00007fe5e8a5fad3 in ?? ()
14 LWP 18952 "txn-status-mana" 0x00007fe5e8a5ffb9 in ?? ()
15 LWP 18953 "collect_and_rem" 0x00007fe5e8a5ffb9 in ?? ()
16 LWP 18954 "tc-session-exp-" 0x00007fe5e8a5ffb9 in ?? ()
17 LWP 18955 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
18 LWP 18956 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
19 LWP 18957 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
20 LWP 18958 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
21 LWP 18959 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
22 LWP 18960 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
23 LWP 18961 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
24 LWP 18962 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
25 LWP 18963 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
26 LWP 18964 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
27 LWP 18965 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
28 LWP 18966 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
29 LWP 18967 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
30 LWP 18968 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
31 LWP 18974 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
32 LWP 18975 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
33 LWP 18976 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
34 LWP 18977 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
35 LWP 18978 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
36 LWP 18979 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
37 LWP 18980 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
38 LWP 18981 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
39 LWP 18982 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
40 LWP 18983 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
41 LWP 18984 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
42 LWP 18985 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
43 LWP 18987 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
44 LWP 18988 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
45 LWP 18989 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
46 LWP 18990 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
47 LWP 18991 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
48 LWP 18992 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
49 LWP 18993 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
50 LWP 18994 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
51 LWP 18995 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
52 LWP 18996 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
53 LWP 18997 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
54 LWP 18998 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
55 LWP 18999 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
56 LWP 19000 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
57 LWP 19001 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
58 LWP 19002 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
59 LWP 19003 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
60 LWP 19004 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
61 LWP 19005 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
62 LWP 19006 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
63 LWP 19007 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
64 LWP 19008 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
65 LWP 19009 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
66 LWP 19010 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
67 LWP 19011 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
68 LWP 19012 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
69 LWP 19013 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
70 LWP 19014 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
71 LWP 19015 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
72 LWP 19016 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
73 LWP 19017 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
74 LWP 19018 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
75 LWP 19019 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
76 LWP 19020 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
77 LWP 19021 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
78 LWP 19022 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
79 LWP 19023 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
80 LWP 19024 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
81 LWP 19025 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
82 LWP 19026 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
83 LWP 19027 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
84 LWP 19028 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
85 LWP 19029 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
86 LWP 19030 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
87 LWP 19031 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
88 LWP 19032 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
89 LWP 19033 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
90 LWP 19034 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
91 LWP 19035 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
92 LWP 19036 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
93 LWP 19037 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
94 LWP 19038 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
95 LWP 19039 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
96 LWP 19040 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
97 LWP 19041 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
98 LWP 19042 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
99 LWP 19043 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
100 LWP 19044 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
101 LWP 19045 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
102 LWP 19046 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
103 LWP 19047 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
104 LWP 19048 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
105 LWP 19049 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
106 LWP 19050 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
107 LWP 19051 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
108 LWP 19052 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
109 LWP 19053 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
110 LWP 19054 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
111 LWP 19055 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
112 LWP 19056 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
113 LWP 19057 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
114 LWP 19058 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
115 LWP 19059 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
116 LWP 19060 "rpc worker-1906" 0x00007fe5e8a5fad3 in ?? ()
117 LWP 19061 "diag-logger-190" 0x00007fe5e8a5ffb9 in ?? ()
118 LWP 19062 "result-tracker-" 0x00007fe5e8a5ffb9 in ?? ()
119 LWP 19063 "excess-log-dele" 0x00007fe5e8a5ffb9 in ?? ()
120 LWP 19064 "acceptor-19064" 0x00007fe5e17cb0c7 in ?? ()
121 LWP 19065 "heartbeat-19065" 0x00007fe5e8a5ffb9 in ?? ()
122 LWP 19068 "maintenance_sch" 0x00007fe5e8a5ffb9 in ?? ()
Thread 122 (LWP 19068):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000011f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fe59db34550 in ?? ()
#6 0x000000000000023e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19065):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x0000000000000047 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fe59eb5b5b0 in ?? ()
#6 0x000000000000008e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19064):
#0 0x00007fe5e17cb0c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007fe59f382d50 in ?? ()
#3 0x00007fe59f382d80 in ?? ()
#4 0x00007fe59f382e80 in ?? ()
#5 0x00007fe59f382d70 in ?? ()
#6 0x00007fe59f382de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 19063):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe59fb9cf00 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc06fa12f0 in ?? ()
#5 0x00007fe59fb9cef0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19062):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5a03b50c0 in ?? ()
#2 0x0000000000000047 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fe5a03b50b0 in ?? ()
#6 0x000000000000008e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19061):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19060):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007fe5a13e4e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5a13e4e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 19059):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x000000000000000c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007fe5a1bfce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5a1bfce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 19058):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19057):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19056):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19055):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19054):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19053):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19052):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19051):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19050):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19049):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19048):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19047):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19046):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19045):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19044):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19043):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19042):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19041):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19040):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19039):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19038):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19037):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19036):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19035):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19034):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19033):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19032):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19031):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19030):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19029):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19028):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19027):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19026):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19025):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19024):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19023):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19022):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19021):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19020):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007fe5b57b5e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5b57b5e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19019):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19018):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19017):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19016):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19015):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19014):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19013):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19012):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19011):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19010):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19009):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19008):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19007):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19006):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19005):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19004):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19003):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19002):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19001):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19000):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fe5bf996e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5bf996e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 18999):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18998):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18997):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18996):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18995):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18994):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18993):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18992):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18991):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18990):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18989):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18988):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18987):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18985):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18984):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18983):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18982):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18981):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18980):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18979):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007fe5c9b77e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5c9b77e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 18978):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18977):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18976):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18975):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18974):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18968):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18967):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18966):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18965):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18964):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18963):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18962):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18961):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18960):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18959):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18958):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18957):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18956):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18955):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18954):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d3561c80 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fe5d3561c70 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18953):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18952):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d459f1c0 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fe5d459f1b0 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18951):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18950):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d55b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fe5d55b22d0 in ?? ()
#4 0x00007fe5d55b24e0 in ?? ()
#5 0x00007fe5d55b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fe5d55b23a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb951f311ed4000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000038800000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ffd3aaae474 in ?? ()
#17 0x00007fe5d55b2380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d55b2350 in ?? ()
#20 0x3fb951f311ed4000 in ?? ()
#21 0x00000000d55b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb951f311ed4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18949):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d5db32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fe5d5db32d0 in ?? ()
#4 0x00007fe5d5db34e0 in ?? ()
#5 0x00007fe5d5db3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fe5d5db33a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb951a6b33c6000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000069000000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ffd3abae674 in ?? ()
#17 0x00007fe5d5db3380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d5db3350 in ?? ()
#20 0x3fb951a6b33c6000 in ?? ()
#21 0x00000000d5db33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb951a6b33c6000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18948):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d85ad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fe5d85ad2d0 in ?? ()
#4 0x00007fe5d85ad4e0 in ?? ()
#5 0x00007fe5d85ad320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fe5d85ad3a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb985a5a0394000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000086400000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffd3b0ada74 in ?? ()
#17 0x00007fe5d85ad380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d85ad350 in ?? ()
#20 0x3fb985a5a0394000 in ?? ()
#21 0x00000000d85ad3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb985a5a0394000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18947):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d65b42e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fe5d65b42d0 in ?? ()
#4 0x00007fe5d65b44e0 in ?? ()
#5 0x00007fe5d65b4320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fe5d65b43a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb957cb2da2c000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000fb800000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ffd3acae874 in ?? ()
#17 0x00007fe5d65b4380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18944):
#0 0x00007fe5e17bccb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fe5d9fb7788 in ?? ()
#3 0x0000602000016c90 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18943):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d97b6080 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fe5d97b6070 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18942):
#0 0x00007fe5e8a639e2 in ?? ()
#1 0x00007fe5d8fb3b60 in ?? ()
#2 0x00007fe5d8fb3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fe5d8fb3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fe5d8fb3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fe5d8fb3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fe5ee387f0c in ?? ()
#12 0x00007fe5ee3777b9 in ?? ()
#13 0x00007fe5ee37bda6 in ?? ()
#14 0x00007fe5ee3807e0 in ?? ()
#15 0x00007fe5ee380609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fe5e56760be in ?? ()
#18 0x00007fe5e8a596db in ?? ()
#19 0x00000ffd3b1ee67c in ?? ()
#20 0x00007fe5d8fb3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18936):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5dafb8c40 in ?? ()
#2 0x000000000000016e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fe5dafb8c30 in ?? ()
#6 0x00000000000002dc in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18935):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18934):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18933):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fe5dc7bc410 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18929):
#0 0x00007fe5e8a63d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:11.458729 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17622
I20250111 21:02:11.473263 17750 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:11.604350 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17622
I20250111 21:02:11.632748 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17338
I20250111 21:02:11.639816 17464 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:11.768431 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17338
I20250111 21:02:11.793288 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17479
I20250111 21:02:11.800853 17605 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.022619 19663 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.022848 19663 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.023700 19663 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:02:12.037497 19667 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.037674 19667 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.038491 19667 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:12.038937 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:02:12.039214 18177 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4301cbd2b84d22ab621b2c399022dc in term 2.
I20250111 21:02:12.039685 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 21:02:12.039994 19667 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250111 21:02:12.040087 19667 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting leader election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.040182 19667 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Advancing to term 3
I20250111 21:02:12.042973 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 21:02:12.045189 19667 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.045614 18872 raft_consensus.cc:2388] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edf2bd29095149dd9e4592ae2870d4ee in current term 3: Already voted for candidate cc4301cbd2b84d22ab621b2c399022dc in this term.
I20250111 21:02:12.045905 19667 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: Requested vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:12.046509 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee"
I20250111 21:02:12.046746 18177 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Advancing to term 3
I20250111 21:02:12.052289 18177 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4301cbd2b84d22ab621b2c399022dc in term 3.
I20250111 21:02:12.052911 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 21:02:12.053246 19667 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Leader election won for term 3
I20250111 21:02:12.053663 19667 raft_consensus.cc:695] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 LEADER]: Becoming Leader. State: Replica: cc4301cbd2b84d22ab621b2c399022dc, State: Running, Role: LEADER
I20250111 21:02:12.054176 19667 consensus_queue.cc:237] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15602, Committed index: 15602, Last appended: 2.15602, Last appended by leader: 15602, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.058341 15272 catalog_manager.cc:5526] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc reported cstate change: term changed from 2 to 3, leader changed from dc04179134174d03bd03d3db91b245f8 (127.14.226.3) to cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8). New cstate: current_term: 3 leader_uuid: "cc4301cbd2b84d22ab621b2c399022dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } health_report { overall_health: HEALTHY } } }
I20250111 21:02:12.153136 18177 raft_consensus.cc:1270] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Refusing update from remote peer cc4301cbd2b84d22ab621b2c399022dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 15602. Preceding OpId from leader: term: 3 index: 15603. (index mismatch)
I20250111 21:02:12.153901 19667 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15603, Last known committed idx: 15602, Time since last communication: 0.000s
I20250111 21:02:12.176502 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:02:12.176983 18872 raft_consensus.cc:1235] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 LEADER]: Rejecting Update request from peer dc04179134174d03bd03d3db91b245f8 for earlier term 2. Current term is 3. Ops: []
I20250111 21:02:12.177083 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17479
W20250111 21:02:12.237635 18805 connection.cc:537] client connection to 127.14.226.3:45529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.237951 18805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250111 21:02:12.238274 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16832
W20250111 21:02:12.238996 19665 negotiation.cc:337] Failed RPC negotiation. Trace:
0111 21:02:12.023988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.226.3:45529 (local address 127.14.226.6:55827)
0111 21:02:12.035483 (+ 11495us) negotiation.cc:107] Waiting for socket to connect
0111 21:02:12.035512 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0111 21:02:12.035702 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0111 21:02:12.177013 (+141311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0111 21:02:12.177019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0111 21:02:12.177061 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0111 21:02:12.177595 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0111 21:02:12.177614 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0111 21:02:12.237845 (+ 60231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":11341,"thread_start_us":11691,"threads_started":1}
W20250111 21:02:12.239326 18112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.240811 18805 connection.cc:537] server connection from 127.14.226.3:51951 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.241119 18805 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
W20250111 21:02:12.241421 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 21:02:12.241533 18805 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
W20250111 21:02:12.242179 18112 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
I20250111 21:02:12.242411 18112 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf2bd29095149dd9e4592ae2870d4ee; no voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8
I20250111 21:02:12.242713 19663 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:02:12.247170 16959 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.378327 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16832
I20250111 21:02:12.404733 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16694
I20250111 21:02:12.411810 16818 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.540218 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16694
I20250111 21:02:12.572142 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18094
I20250111 21:02:12.578969 18224 generic_service.cc:196] Checking for leaks (request via RPC)
W20250111 21:02:12.671893 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250111 21:02:13.046213 19663 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cc4301cbd2b84d22ab621b2c399022dc)
I20250111 21:02:13.046483 19663 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:13.046813 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18094
W20250111 21:02:13.117154 18805 connection.cc:537] client connection to 127.14.226.6:42445 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250111 21:02:13.117573 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16972
W20250111 21:02:13.118556 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250111 21:02:13.124231 17101 generic_service.cc:196] Checking for leaks (request via RPC)
W20250111 21:02:13.146845 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250111 21:02:13.254453 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16972
I20250111 21:02:13.279937 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18787
I20250111 21:02:13.286808 18913 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:13.701277 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18787
I20250111 21:02:13.764771 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18929
I20250111 21:02:13.771960 19060 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:13.876780 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18929
I20250111 21:02:13.900236 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15248
I20250111 21:02:13.907297 15308 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:14.100737 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15248
2025-01-11T21:02:14Z chronyd exiting
I20250111 21:02:14.158674 15240 test_util.cc:181] -----------------------------------------------
I20250111 21:02:14.158769 15240 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0
Note: This is test shard 8 of 8.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RollingRestartArgs/RollingRestartITest
[ RUN ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3
2025-01-11T20:58:30Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-11T20:58:30Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250111 20:58:30.019433 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.226.62:37841
--webserver_interface=127.14.226.62
--webserver_port=0
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.226.62:37841
--location_mapping_cmd=/tmp/dist-test-taskBASTb8/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/location-assignment.state --map /L0:3 --map /L1:4 --map /L2:4
--master_client_location_assignment_enabled=false with env {}
W20250111 20:58:30.357712 15248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:30.358028 15248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:30.358119 15248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:30.365978 15248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250111 20:58:30.366099 15248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:30.366143 15248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250111 20:58:30.366200 15248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250111 20:58:30.376071 15248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal
--location_mapping_cmd=/tmp/dist-test-taskBASTb8/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/location-assignment.state --map /L0:3 --map /L1:4 --map /L2:4
--ipki_ca_key_size=768
--master_addresses=127.14.226.62:37841
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.226.62:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.226.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:30.377712 15248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:30.379530 15248 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:30.391438 15253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:30.392726 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:30.393448 15256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:30.394796 15248 server_base.cc:1034] running on GCE node
I20250111 20:58:30.395686 15248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:30.397192 15248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:30.398414 15248 hybrid_clock.cc:648] HybridClock initialized: now 1736629110398315 us; error 77 us; skew 500 ppm
I20250111 20:58:30.398741 15248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:30.405674 15248 webserver.cc:458] Webserver started at http://127.14.226.62:40721/ using document root <none> and password file <none>
I20250111 20:58:30.406574 15248 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:30.406702 15248 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:30.407038 15248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:30.409734 15248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/instance:
uuid: "6701e95595e54e48a427659232b86234"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.410578 15248 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal/instance:
uuid: "6701e95595e54e48a427659232b86234"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.416098 15248 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20250111 20:58:30.419651 15262 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:30.421110 15248 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250111 20:58:30.421293 15248 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal
uuid: "6701e95595e54e48a427659232b86234"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.421499 15248 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:30.452598 15248 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:30.453518 15248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:30.453805 15248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:30.472028 15248 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.62:37841
I20250111 20:58:30.472056 15313 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.62:37841 every 8 connection(s)
I20250111 20:58:30.473590 15248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/info.pb
I20250111 20:58:30.477591 15314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 20:58:30.478689 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15248
I20250111 20:58:30.478981 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/wal/instance
I20250111 20:58:30.490165 15314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234: Bootstrap starting.
I20250111 20:58:30.493423 15314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234: Neither blocks nor log segments found. Creating new log.
I20250111 20:58:30.494587 15314 log.cc:826] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234: Log is configured to *not* fsync() on all Append() calls
I20250111 20:58:30.498212 15314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234: No bootstrap required, opened a new log
I20250111 20:58:30.506947 15314 raft_consensus.cc:357] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } }
I20250111 20:58:30.507454 15314 raft_consensus.cc:383] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 20:58:30.507576 15314 raft_consensus.cc:738] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6701e95595e54e48a427659232b86234, State: Initialized, Role: FOLLOWER
I20250111 20:58:30.508342 15314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } }
I20250111 20:58:30.508589 15314 raft_consensus.cc:397] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250111 20:58:30.508733 15314 raft_consensus.cc:491] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250111 20:58:30.508896 15314 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 0 FOLLOWER]: Advancing to term 1
I20250111 20:58:30.511371 15314 raft_consensus.cc:513] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } }
I20250111 20:58:30.511857 15314 leader_election.cc:304] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6701e95595e54e48a427659232b86234; no voters:
I20250111 20:58:30.512444 15314 leader_election.cc:290] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250111 20:58:30.512578 15319 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 1 FOLLOWER]: Leader election won for term 1
I20250111 20:58:30.513217 15319 raft_consensus.cc:695] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [term 1 LEADER]: Becoming Leader. State: Replica: 6701e95595e54e48a427659232b86234, State: Running, Role: LEADER
I20250111 20:58:30.513746 15319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } }
I20250111 20:58:30.514482 15314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [sys.catalog]: configured and running, proceeding with master startup.
I20250111 20:58:30.516523 15320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6701e95595e54e48a427659232b86234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } } }
I20250111 20:58:30.516851 15320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [sys.catalog]: This master's current role is: LEADER
I20250111 20:58:30.518761 15325 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250111 20:58:30.518996 15321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6701e95595e54e48a427659232b86234. Latest consensus state: current_term: 1 leader_uuid: "6701e95595e54e48a427659232b86234" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6701e95595e54e48a427659232b86234" member_type: VOTER last_known_addr { host: "127.14.226.62" port: 37841 } } }
I20250111 20:58:30.519421 15321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234 [sys.catalog]: This master's current role is: LEADER
I20250111 20:58:30.524770 15325 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250111 20:58:30.534420 15325 catalog_manager.cc:1348] Generated new cluster ID: c5cb489f1ace422ea8ff05100b53bdc6
I20250111 20:58:30.534564 15325 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250111 20:58:30.550392 15325 catalog_manager.cc:1371] Generated new certificate authority record
I20250111 20:58:30.551671 15325 catalog_manager.cc:1505] Loading token signing keys...
I20250111 20:58:30.567945 15325 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 6701e95595e54e48a427659232b86234: Generated new TSK 0
I20250111 20:58:30.568873 15325 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250111 20:58:30.575675 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.1:0
--local_ip_for_outbound_sockets=127.14.226.1
--webserver_interface=127.14.226.1
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:58:30.911373 15338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:30.911702 15338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:30.911852 15338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:30.919961 15338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:30.920265 15338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.1
I20250111 20:58:30.929937 15338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.226.1
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.1
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:30.931634 15338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:30.933480 15338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:30.945648 15343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:30.946997 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:30.948042 15344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:30.949093 15338 server_base.cc:1034] running on GCE node
I20250111 20:58:30.949716 15338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:30.950750 15338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:30.951979 15338 hybrid_clock.cc:648] HybridClock initialized: now 1736629110951895 us; error 58 us; skew 500 ppm
I20250111 20:58:30.952296 15338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:30.954710 15338 webserver.cc:458] Webserver started at http://127.14.226.1:44911/ using document root <none> and password file <none>
I20250111 20:58:30.955492 15338 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:30.955624 15338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:30.955935 15338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:30.958724 15338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/instance:
uuid: "09be37535c0a473a956abea447348515"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.959507 15338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal/instance:
uuid: "09be37535c0a473a956abea447348515"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.965020 15338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250111 20:58:30.968552 15352 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:30.969972 15338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20250111 20:58:30.970144 15338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
uuid: "09be37535c0a473a956abea447348515"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:58:30.970371 15338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:30.990234 15338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:30.991183 15338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:30.991463 15338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:30.992419 15338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:30.994060 15338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:30.994141 15338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:30.994298 15338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:30.994359 15338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20250111 20:58:31.026859 15338 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.1:38037
I20250111 20:58:31.026902 15464 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.1:38037 every 8 connection(s)
I20250111 20:58:31.028259 15338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
I20250111 20:58:31.034842 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15338
I20250111 20:58:31.035058 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal/instance
I20250111 20:58:31.038640 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.2:0
--local_ip_for_outbound_sockets=127.14.226.2
--webserver_interface=127.14.226.2
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:31.041752 15465 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:31.042107 15465 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:31.043216 15465 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:31.088757 15279 ts_manager.cc:194] Registered new tserver with Master: 09be37535c0a473a956abea447348515 (127.14.226.1:38037)
I20250111 20:58:31.090652 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.1:53229
W20250111 20:58:31.377033 15469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:31.377393 15469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:31.377547 15469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:31.385465 15469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:31.385825 15469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.2
I20250111 20:58:31.395450 15469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.226.2
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.2
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:31.397076 15469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:31.398871 15469 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:31.411029 15475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:31.412276 15476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:31.413318 15478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:31.414378 15469 server_base.cc:1034] running on GCE node
I20250111 20:58:31.415036 15469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:31.416072 15469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:31.417320 15469 hybrid_clock.cc:648] HybridClock initialized: now 1736629111417231 us; error 46 us; skew 500 ppm
I20250111 20:58:31.417652 15469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:31.419951 15469 webserver.cc:458] Webserver started at http://127.14.226.2:39013/ using document root <none> and password file <none>
I20250111 20:58:31.420714 15469 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:31.420817 15469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:31.421077 15469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:31.423828 15469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/instance:
uuid: "eb9ae591b5d24eb18f58490f46e7dfaf"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.424553 15469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal/instance:
uuid: "eb9ae591b5d24eb18f58490f46e7dfaf"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.430016 15469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20250111 20:58:31.433512 15484 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.434937 15469 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20250111 20:58:31.435091 15469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
uuid: "eb9ae591b5d24eb18f58490f46e7dfaf"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.435281 15469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:31.465093 15469 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:31.465965 15469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:31.466238 15469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:31.467350 15469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:31.468859 15469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:31.468940 15469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.469079 15469 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:31.469133 15469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.508702 15469 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.2:41707
I20250111 20:58:31.508759 15596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.2:41707 every 8 connection(s)
I20250111 20:58:31.510125 15469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
I20250111 20:58:31.518537 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15469
I20250111 20:58:31.518718 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal/instance
I20250111 20:58:31.521919 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.3:0
--local_ip_for_outbound_sockets=127.14.226.3
--webserver_interface=127.14.226.3
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:31.524351 15597 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:31.524677 15597 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:31.525524 15597 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:31.567824 15279 ts_manager.cc:194] Registered new tserver with Master: eb9ae591b5d24eb18f58490f46e7dfaf (127.14.226.2:41707)
I20250111 20:58:31.568809 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.2:38533
W20250111 20:58:31.860483 15600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:31.860827 15600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:31.860975 15600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:31.868791 15600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:31.869078 15600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.3
I20250111 20:58:31.878935 15600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.226.3
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.3
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:31.880568 15600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:31.882330 15600 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:31.894850 15607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:31.895980 15609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:31.897416 15606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:31.898629 15600 server_base.cc:1034] running on GCE node
I20250111 20:58:31.899340 15600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:31.900323 15600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:31.901544 15600 hybrid_clock.cc:648] HybridClock initialized: now 1736629111901480 us; error 43 us; skew 500 ppm
I20250111 20:58:31.901852 15600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:31.904261 15600 webserver.cc:458] Webserver started at http://127.14.226.3:34305/ using document root <none> and password file <none>
I20250111 20:58:31.905067 15600 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:31.905196 15600 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:31.905516 15600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:31.908284 15600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/instance:
uuid: "dc04179134174d03bd03d3db91b245f8"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.909150 15600 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/instance:
uuid: "dc04179134174d03bd03d3db91b245f8"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.914726 15600 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20250111 20:58:31.918403 15615 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.919930 15600 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250111 20:58:31.920120 15600 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
uuid: "dc04179134174d03bd03d3db91b245f8"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:58:31.920356 15600 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:31.934083 15600 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:31.935012 15600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:31.935302 15600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:31.936288 15600 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:31.937852 15600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:31.937937 15600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.938076 15600 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:31.938134 15600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:31.970790 15600 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.3:45529
I20250111 20:58:31.970849 15727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.3:45529 every 8 connection(s)
I20250111 20:58:31.972105 15600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
I20250111 20:58:31.980684 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15600
I20250111 20:58:31.980871 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/instance
I20250111 20:58:31.984362 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.4:0
--local_ip_for_outbound_sockets=127.14.226.4
--webserver_interface=127.14.226.4
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:31.985872 15728 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:31.986184 15728 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:31.987037 15728 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:32.047356 15279 ts_manager.cc:194] Registered new tserver with Master: dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 20:58:32.048352 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.3:32891
I20250111 20:58:32.093878 15465 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:32.339162 15731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:32.339519 15731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:32.339671 15731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:32.347869 15731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:32.348156 15731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.4
I20250111 20:58:32.357697 15731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.14.226.4
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.4
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:32.359393 15731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:32.361140 15731 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:32.373447 15737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:32.374745 15740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:32.375720 15738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:32.377173 15731 server_base.cc:1034] running on GCE node
I20250111 20:58:32.377790 15731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:32.378881 15731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:32.380206 15731 hybrid_clock.cc:648] HybridClock initialized: now 1736629112380135 us; error 48 us; skew 500 ppm
I20250111 20:58:32.380520 15731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:32.383033 15731 webserver.cc:458] Webserver started at http://127.14.226.4:38523/ using document root <none> and password file <none>
I20250111 20:58:32.383785 15731 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:32.383890 15731 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:32.384197 15731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:32.387075 15731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/instance:
uuid: "4e66492bc0e449a981045ff82ba305e8"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.389659 15731 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal/instance:
uuid: "4e66492bc0e449a981045ff82ba305e8"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.395375 15731 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250111 20:58:32.398908 15746 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.400307 15731 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250111 20:58:32.400467 15731 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
uuid: "4e66492bc0e449a981045ff82ba305e8"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.400655 15731 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:32.434811 15731 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:32.435647 15731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:32.435930 15731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:32.436878 15731 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:32.438413 15731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:32.438484 15731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.438602 15731 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:32.438652 15731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.470959 15731 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.4:33171
I20250111 20:58:32.471019 15858 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.4:33171 every 8 connection(s)
I20250111 20:58:32.472416 15731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
I20250111 20:58:32.474916 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15731
I20250111 20:58:32.475085 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal/instance
I20250111 20:58:32.478895 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.5:0
--local_ip_for_outbound_sockets=127.14.226.5
--webserver_interface=127.14.226.5
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:32.487830 15859 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:32.488148 15859 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:32.488973 15859 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:32.534511 15279 ts_manager.cc:194] Registered new tserver with Master: 4e66492bc0e449a981045ff82ba305e8 (127.14.226.4:33171)
I20250111 20:58:32.535796 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.4:54701
I20250111 20:58:32.571489 15597 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:32.822576 15862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:32.822890 15862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:32.823042 15862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:32.830981 15862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:32.831267 15862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.5
I20250111 20:58:32.841042 15862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
--webserver_interface=127.14.226.5
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.5
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:32.842728 15862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:32.844501 15862 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:32.857710 15868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:32.858378 15869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:32.859506 15871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:32.860538 15862 server_base.cc:1034] running on GCE node
I20250111 20:58:32.861218 15862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:32.862272 15862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:32.863520 15862 hybrid_clock.cc:648] HybridClock initialized: now 1736629112863446 us; error 48 us; skew 500 ppm
I20250111 20:58:32.863823 15862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:32.866168 15862 webserver.cc:458] Webserver started at http://127.14.226.5:46775/ using document root <none> and password file <none>
I20250111 20:58:32.866967 15862 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:32.867130 15862 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:32.867432 15862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:32.870091 15862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/instance:
uuid: "18452380c023464488c18995b91fa513"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.870999 15862 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal/instance:
uuid: "18452380c023464488c18995b91fa513"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.876420 15862 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250111 20:58:32.880046 15877 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.881417 15862 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250111 20:58:32.881578 15862 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
uuid: "18452380c023464488c18995b91fa513"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:32.881765 15862 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:32.904651 15862 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:32.905588 15862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:32.905871 15862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:32.907014 15862 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:32.908623 15862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:32.908711 15862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.908852 15862 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:32.908921 15862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:32.941488 15862 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.5:41645
I20250111 20:58:32.941529 15989 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.5:41645 every 8 connection(s)
I20250111 20:58:32.942899 15862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
I20250111 20:58:32.948895 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15862
I20250111 20:58:32.949136 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal/instance
I20250111 20:58:32.952919 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.6:0
--local_ip_for_outbound_sockets=127.14.226.6
--webserver_interface=127.14.226.6
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:32.957970 15990 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:32.958333 15990 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:32.959200 15990 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:33.018339 15279 ts_manager.cc:194] Registered new tserver with Master: 18452380c023464488c18995b91fa513 (127.14.226.5:41645)
I20250111 20:58:33.019376 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.5:59787
I20250111 20:58:33.051924 15728 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:33.314296 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:33.314639 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:33.314785 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:33.322917 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:33.323194 15993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.6
I20250111 20:58:33.332860 15993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.6:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
--webserver_interface=127.14.226.6
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.6
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:33.334581 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:33.336447 15993 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:33.349279 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:33.350370 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:33.351702 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:33.352870 15993 server_base.cc:1034] running on GCE node
I20250111 20:58:33.353518 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:33.354532 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:33.355736 15993 hybrid_clock.cc:648] HybridClock initialized: now 1736629113355662 us; error 55 us; skew 500 ppm
I20250111 20:58:33.356031 15993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:33.358327 15993 webserver.cc:458] Webserver started at http://127.14.226.6:33815/ using document root <none> and password file <none>
I20250111 20:58:33.359071 15993 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:33.359191 15993 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:33.359484 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:33.362154 15993 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/instance:
uuid: "edf2bd29095149dd9e4592ae2870d4ee"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.362929 15993 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal/instance:
uuid: "edf2bd29095149dd9e4592ae2870d4ee"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.368248 15993 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250111 20:58:33.371565 16008 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.372925 15993 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250111 20:58:33.373082 15993 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
uuid: "edf2bd29095149dd9e4592ae2870d4ee"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.373283 15993 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:33.392720 15993 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:33.393599 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:33.393872 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:33.394848 15993 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:33.396508 15993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:33.396601 15993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.396731 15993 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:33.396792 15993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.428892 15993 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.6:42445
I20250111 20:58:33.428931 16120 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.6:42445 every 8 connection(s)
I20250111 20:58:33.430354 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
I20250111 20:58:33.432713 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 15993
I20250111 20:58:33.432955 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal/instance
I20250111 20:58:33.436532 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.7:0
--local_ip_for_outbound_sockets=127.14.226.7
--webserver_interface=127.14.226.7
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:33.445040 16121 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:33.445395 16121 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:33.446292 16121 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:33.496158 15279 ts_manager.cc:194] Registered new tserver with Master: edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445)
I20250111 20:58:33.497462 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.6:51465
I20250111 20:58:33.539342 15859 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:33.783823 16124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:33.784127 16124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:33.784240 16124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:33.792044 16124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:33.792270 16124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.7
I20250111 20:58:33.801673 16124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.7:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
--webserver_interface=127.14.226.7
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.7
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:33.803277 16124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:33.804946 16124 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:33.817381 16131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:33.818531 16130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:33.819566 16133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:33.820946 16124 server_base.cc:1034] running on GCE node
I20250111 20:58:33.821588 16124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:33.822688 16124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:33.823917 16124 hybrid_clock.cc:648] HybridClock initialized: now 1736629113823847 us; error 50 us; skew 500 ppm
I20250111 20:58:33.824249 16124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:33.826792 16124 webserver.cc:458] Webserver started at http://127.14.226.7:40597/ using document root <none> and password file <none>
I20250111 20:58:33.827600 16124 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:33.827720 16124 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:33.827989 16124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:33.830735 16124 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/instance:
uuid: "49a14c4ba19447f8851a9a40990a4b91"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.831465 16124 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal/instance:
uuid: "49a14c4ba19447f8851a9a40990a4b91"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.837325 16124 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250111 20:58:33.843657 16139 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.845233 16124 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20250111 20:58:33.845413 16124 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
uuid: "49a14c4ba19447f8851a9a40990a4b91"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:33.845618 16124 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:33.882812 16124 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:33.883671 16124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:33.883949 16124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:33.884980 16124 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:33.886582 16124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:33.886660 16124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.886796 16124 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:33.886862 16124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:33.925926 16124 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.7:37147
I20250111 20:58:33.925948 16251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.7:37147 every 8 connection(s)
I20250111 20:58:33.927430 16124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
I20250111 20:58:33.936862 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16124
I20250111 20:58:33.937078 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal/instance
I20250111 20:58:33.941112 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.8:0
--local_ip_for_outbound_sockets=127.14.226.8
--webserver_interface=127.14.226.8
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:33.947392 16252 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:33.947842 16252 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:33.949009 16252 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:34.022589 15990 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 20:58:34.053395 15279 ts_manager.cc:194] Registered new tserver with Master: 49a14c4ba19447f8851a9a40990a4b91 (127.14.226.7:37147)
I20250111 20:58:34.054742 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.7:60705
W20250111 20:58:34.298851 16255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:34.299175 16255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:34.299319 16255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:34.307508 16255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:34.307775 16255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.8
I20250111 20:58:34.317520 16255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.8:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
--webserver_interface=127.14.226.8
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.8
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:34.319303 16255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:34.321167 16255 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:34.334450 16262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:34.334959 16261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:34.335932 16264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:34.337419 16255 server_base.cc:1034] running on GCE node
I20250111 20:58:34.338079 16255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:34.339143 16255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:34.340382 16255 hybrid_clock.cc:648] HybridClock initialized: now 1736629114340314 us; error 41 us; skew 500 ppm
I20250111 20:58:34.340708 16255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:34.343127 16255 webserver.cc:458] Webserver started at http://127.14.226.8:33315/ using document root <none> and password file <none>
I20250111 20:58:34.343899 16255 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:34.344028 16255 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:34.344332 16255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:34.347419 16255 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/instance:
uuid: "cc4301cbd2b84d22ab621b2c399022dc"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.348160 16255 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal/instance:
uuid: "cc4301cbd2b84d22ab621b2c399022dc"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.353531 16255 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250111 20:58:34.357065 16270 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.358605 16255 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250111 20:58:34.358780 16255 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
uuid: "cc4301cbd2b84d22ab621b2c399022dc"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.359009 16255 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:34.383177 16255 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:34.384054 16255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:34.384351 16255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:34.385337 16255 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:34.387041 16255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:34.387125 16255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.387271 16255 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:34.387327 16255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.421833 16255 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.8:41133
I20250111 20:58:34.421861 16382 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.8:41133 every 8 connection(s)
I20250111 20:58:34.423425 16255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
I20250111 20:58:34.432662 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16255
I20250111 20:58:34.432852 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal/instance
I20250111 20:58:34.436446 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.9:0
--local_ip_for_outbound_sockets=127.14.226.9
--webserver_interface=127.14.226.9
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20250111 20:58:34.437549 16383 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:34.437928 16383 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:34.439070 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:34.494418 15279 ts_manager.cc:194] Registered new tserver with Master: cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:58:34.495363 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.8:47157
I20250111 20:58:34.500224 16121 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:34.798058 16386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:34.798460 16386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:34.798619 16386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:34.806918 16386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:34.807191 16386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.9
I20250111 20:58:34.817248 16386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.9:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
--webserver_interface=127.14.226.9
--webserver_port=0
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.9
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:34.819159 16386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:34.820947 16386 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:34.835829 16392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:34.837052 16393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:34.837955 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:34.839417 16386 server_base.cc:1034] running on GCE node
I20250111 20:58:34.840229 16386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:34.842231 16386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:34.843560 16386 hybrid_clock.cc:648] HybridClock initialized: now 1736629114843477 us; error 37 us; skew 500 ppm
I20250111 20:58:34.844033 16386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:34.847352 16386 webserver.cc:458] Webserver started at http://127.14.226.9:33127/ using document root <none> and password file <none>
I20250111 20:58:34.848287 16386 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:34.848445 16386 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:34.848827 16386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250111 20:58:34.852455 16386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/instance:
uuid: "4c1123e925164e978604ac78771cb3ad"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.853473 16386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal/instance:
uuid: "4c1123e925164e978604ac78771cb3ad"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.862226 16386 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20250111 20:58:34.867733 16401 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.869724 16386 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20250111 20:58:34.869946 16386 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
uuid: "4c1123e925164e978604ac78771cb3ad"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 20:58:34.870234 16386 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:34.898336 16386 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:34.899250 16386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:34.899529 16386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:34.900507 16386 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:34.902122 16386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:34.902226 16386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.902380 16386 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:34.902439 16386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:34.934942 16386 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.9:39547
I20250111 20:58:34.935004 16513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.9:39547 every 8 connection(s)
I20250111 20:58:34.936298 16386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
I20250111 20:58:34.944089 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16386
I20250111 20:58:34.944283 15240 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal/instance
I20250111 20:58:34.950563 16514 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:34.950917 16514 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:34.951757 16514 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:35.001725 15279 ts_manager.cc:194] Registered new tserver with Master: 4c1123e925164e978604ac78771cb3ad (127.14.226.9:39547)
I20250111 20:58:35.002704 15279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.9:45843
I20250111 20:58:35.014377 15240 external_mini_cluster.cc:934] 9 TS(s) registered with all masters
I20250111 20:58:35.037226 15240 test_util.cc:274] Using random seed: 2038572538
I20250111 20:58:35.058517 16252 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 20:58:35.059618 15279 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51862:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250111 20:58:35.104276 16318 tablet_service.cc:1467] Processing CreateTablet for tablet 54a6628492fa41d593f0c59af04ad973 (DEFAULT_TABLE table=test-workload [id=36efa94b165a4490bd74234e3d893f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250111 20:58:35.104768 16056 tablet_service.cc:1467] Processing CreateTablet for tablet 54a6628492fa41d593f0c59af04ad973 (DEFAULT_TABLE table=test-workload [id=36efa94b165a4490bd74234e3d893f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250111 20:58:35.105969 16318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a6628492fa41d593f0c59af04ad973. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 20:58:35.106132 16056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a6628492fa41d593f0c59af04ad973. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 20:58:35.105983 15663 tablet_service.cc:1467] Processing CreateTablet for tablet 54a6628492fa41d593f0c59af04ad973 (DEFAULT_TABLE table=test-workload [id=36efa94b165a4490bd74234e3d893f5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250111 20:58:35.107183 15663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a6628492fa41d593f0c59af04ad973. 1 dirs total, 0 dirs full, 0 dirs failed
I20250111 20:58:35.119056 16539 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap starting.
I20250111 20:58:35.120769 16540 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap starting.
I20250111 20:58:35.122581 16541 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap starting.
I20250111 20:58:35.124619 16540 tablet_bootstrap.cc:654] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Neither blocks nor log segments found. Creating new log.
I20250111 20:58:35.124619 16539 tablet_bootstrap.cc:654] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Neither blocks nor log segments found. Creating new log.
I20250111 20:58:35.125339 16541 tablet_bootstrap.cc:654] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Neither blocks nor log segments found. Creating new log.
I20250111 20:58:35.126035 16539 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Log is configured to *not* fsync() on all Append() calls
I20250111 20:58:35.126034 16540 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Log is configured to *not* fsync() on all Append() calls
I20250111 20:58:35.126551 16541 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Log is configured to *not* fsync() on all Append() calls
I20250111 20:58:35.129424 16541 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: No bootstrap required, opened a new log
I20250111 20:58:35.129424 16540 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: No bootstrap required, opened a new log
I20250111 20:58:35.129735 16540 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20250111 20:58:35.129736 16541 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20250111 20:58:35.130683 16539 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: No bootstrap required, opened a new log
I20250111 20:58:35.130959 16539 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20250111 20:58:35.136988 16539 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.137427 16539 raft_consensus.cc:383] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 20:58:35.137553 16539 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc04179134174d03bd03d3db91b245f8, State: Initialized, Role: FOLLOWER
I20250111 20:58:35.138280 16539 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.139149 16541 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.139151 16540 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.139603 16540 raft_consensus.cc:383] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 20:58:35.139602 16541 raft_consensus.cc:383] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250111 20:58:35.139699 16540 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4301cbd2b84d22ab621b2c399022dc, State: Initialized, Role: FOLLOWER
I20250111 20:58:35.139700 16541 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf2bd29095149dd9e4592ae2870d4ee, State: Initialized, Role: FOLLOWER
I20250111 20:58:35.140321 16539 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Time spent starting tablet: real 0.009s user 0.004s sys 0.006s
I20250111 20:58:35.140441 16541 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.140440 16540 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.143206 16383 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 20:58:35.144852 16540 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20250111 20:58:35.146605 16541 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20250111 20:58:35.154471 16546 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250111 20:58:35.154712 16546 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.156457 16546 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:58:35.164351 15683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 20:58:35.164351 16338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 20:58:35.164836 15683 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edf2bd29095149dd9e4592ae2870d4ee in term 0.
I20250111 20:58:35.164841 16338 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edf2bd29095149dd9e4592ae2870d4ee in term 0.
I20250111 20:58:35.165520 16010 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 20:58:35.165928 16546 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250111 20:58:35.166095 16546 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250111 20:58:35.166217 16546 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 0 FOLLOWER]: Advancing to term 1
I20250111 20:58:35.168320 16546 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:58:35.168963 16546 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 1 election: Requested vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:58:35.169512 15683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8"
I20250111 20:58:35.169552 16338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc"
I20250111 20:58:35.169709 15683 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 0 FOLLOWER]: Advancing to term 1
I20250111 20:58:35.169742 16338 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 0 FOLLOWER]: Advancing to term 1
I20250111 20:58:35.171916 15683 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edf2bd29095149dd9e4592ae2870d4ee in term 1.
I20250111 20:58:35.171927 16338 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edf2bd29095149dd9e4592ae2870d4ee in term 1.
I20250111 20:58:35.172415 16010 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 20:58:35.172772 16546 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 FOLLOWER]: Leader election won for term 1
I20250111 20:58:35.173250 16546 raft_consensus.cc:695] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 LEADER]: Becoming Leader. State: Replica: edf2bd29095149dd9e4592ae2870d4ee, State: Running, Role: LEADER
I20250111 20:58:35.173825 16546 consensus_queue.cc:237] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
W20250111 20:58:35.175804 16384 tablet.cc:2367] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 20:58:35.178961 15278 catalog_manager.cc:5526] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee reported cstate change: term changed from 0 to 1, leader changed from <none> to edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6). New cstate: current_term: 1 leader_uuid: "edf2bd29095149dd9e4592ae2870d4ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } health_report { overall_health: UNKNOWN } } }
W20250111 20:58:35.183075 16122 tablet.cc:2367] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 20:58:35.206667 15240 maintenance_mode-itest.cc:745] Restarting batch of 3 tservers: 18452380c023464488c18995b91fa513,4e66492bc0e449a981045ff82ba305e8,49a14c4ba19447f8851a9a40990a4b91
W20250111 20:58:35.229804 15729 tablet.cc:2367] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250111 20:58:35.268484 16546 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 20:58:35.290515 16546 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250111 20:58:36.005499 16514 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 20:58:36.204377 15270 ts_manager.cc:292] Set tserver state for 4e66492bc0e449a981045ff82ba305e8 to MAINTENANCE_MODE
I20250111 20:58:36.607475 15271 ts_manager.cc:292] Set tserver state for 49a14c4ba19447f8851a9a40990a4b91 to MAINTENANCE_MODE
I20250111 20:58:36.620311 15273 ts_manager.cc:292] Set tserver state for 18452380c023464488c18995b91fa513 to MAINTENANCE_MODE
I20250111 20:58:39.290359 15794 tablet_service.cc:1422] Tablet server 4e66492bc0e449a981045ff82ba305e8 set to quiescing
I20250111 20:58:39.290815 15794 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:40.162158 15925 tablet_service.cc:1422] Tablet server 18452380c023464488c18995b91fa513 set to quiescing
I20250111 20:58:40.162470 15925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:41.368932 16187 tablet_service.cc:1422] Tablet server 49a14c4ba19447f8851a9a40990a4b91 set to quiescing
I20250111 20:58:41.369372 16187 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:43.412504 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15862
I20250111 20:58:43.471227 15985 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:58:43.700719 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15862
I20250111 20:58:43.765990 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.5:41645
--local_ip_for_outbound_sockets=127.14.226.5
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=46775
--webserver_interface=127.14.226.5
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:58:44.411934 16694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:44.412247 16694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:44.412398 16694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:44.422984 16694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:44.423249 16694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.5
I20250111 20:58:44.436477 16694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.5:41645
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
--webserver_interface=127.14.226.5
--webserver_port=46775
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.5
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:44.438825 16694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:44.441284 16694 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:58:44.552007 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:44.554330 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:44.585841 16703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:44.596136 16694 server_base.cc:1034] running on GCE node
I20250111 20:58:44.597398 16694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:44.615036 16694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:44.620254 16694 hybrid_clock.cc:648] HybridClock initialized: now 1736629124619572 us; error 634 us; skew 500 ppm
I20250111 20:58:44.620975 16694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:44.682684 16694 webserver.cc:458] Webserver started at http://127.14.226.5:46775/ using document root <none> and password file <none>
I20250111 20:58:44.683907 16694 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:44.684113 16694 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:44.701540 16694 fs_manager.cc:714] Time spent opening directory manager: real 0.015s user 0.003s sys 0.007s
I20250111 20:58:44.713575 16708 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:44.718358 16694 fs_manager.cc:730] Time spent opening block manager: real 0.012s user 0.006s sys 0.003s
I20250111 20:58:44.718638 16694 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
uuid: "18452380c023464488c18995b91fa513"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:44.720016 16694 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:44.794379 16694 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:44.795636 16694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:44.796077 16694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:44.803886 16694 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:44.806234 16694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:44.806509 16694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250111 20:58:44.806730 16694 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:44.806875 16694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:45.561192 16694 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.5:41645
I20250111 20:58:45.561226 16823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.5:41645 every 8 connection(s)
I20250111 20:58:45.563129 16694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb
I20250111 20:58:45.566820 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16694
I20250111 20:58:45.567111 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15731
I20250111 20:58:45.604403 15854 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:58:45.605443 16824 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:45.624260 16824 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:45.625348 16824 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:45.627306 15273 ts_manager.cc:194] Re-registered known tserver with Master: 18452380c023464488c18995b91fa513 (127.14.226.5:41645)
I20250111 20:58:45.628391 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.5:40257
I20250111 20:58:45.868373 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15731
I20250111 20:58:45.909325 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.4:33171
--local_ip_for_outbound_sockets=127.14.226.4
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=38523
--webserver_interface=127.14.226.4
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:58:46.501343 16832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:46.501847 16832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:46.502053 16832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:46.518575 16832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:46.518963 16832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.4
I20250111 20:58:46.534826 16832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.4:33171
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.14.226.4
--webserver_port=38523
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.4
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:46.537158 16832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:46.539652 16832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 20:58:46.631575 16824 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:46.667956 16841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:46.669353 16838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:46.674639 16839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:46.675629 16832 server_base.cc:1034] running on GCE node
I20250111 20:58:46.676824 16832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:46.681821 16832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:46.685323 16832 hybrid_clock.cc:648] HybridClock initialized: now 1736629126685236 us; error 49 us; skew 500 ppm
I20250111 20:58:46.685807 16832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:46.696069 16832 webserver.cc:458] Webserver started at http://127.14.226.4:38523/ using document root <none> and password file <none>
I20250111 20:58:46.697754 16832 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:46.697953 16832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:46.713220 16832 fs_manager.cc:714] Time spent opening directory manager: real 0.013s user 0.011s sys 0.000s
I20250111 20:58:46.724215 16847 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:46.726583 16832 fs_manager.cc:730] Time spent opening block manager: real 0.011s user 0.002s sys 0.007s
I20250111 20:58:46.726850 16832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
uuid: "4e66492bc0e449a981045ff82ba305e8"
format_stamp: "Formatted at 2025-01-11 20:58:32 on dist-test-slave-ntcq"
I20250111 20:58:46.728425 16832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:46.787986 16832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:46.789978 16832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:46.791793 16832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:46.793227 16832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:46.801784 16832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:46.802011 16832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250111 20:58:46.802196 16832 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:46.802556 16832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:48.127346 16832 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.4:33171
I20250111 20:58:48.127395 16964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.4:33171 every 8 connection(s)
I20250111 20:58:48.131683 16832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb
I20250111 20:58:48.135103 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16832
I20250111 20:58:48.135361 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16124
I20250111 20:58:48.176607 16247 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:58:48.230845 16965 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:48.231168 16965 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:48.232944 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:48.235186 15273 ts_manager.cc:194] Re-registered known tserver with Master: 4e66492bc0e449a981045ff82ba305e8 (127.14.226.4:33171)
I20250111 20:58:48.236325 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.4:45103
I20250111 20:58:48.372155 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16124
I20250111 20:58:48.412544 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.7:37147
--local_ip_for_outbound_sockets=127.14.226.7
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=40597
--webserver_interface=127.14.226.7
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:58:49.186959 16972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:58:49.187498 16972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:58:49.187703 16972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:58:49.206126 16972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:58:49.206596 16972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.7
I20250111 20:58:49.220871 16972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.7:37147
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
--webserver_interface=127.14.226.7
--webserver_port=40597
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.7
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:58:49.223198 16972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:58:49.225561 16972 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 20:58:49.241149 16965 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:58:49.360451 16980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:49.375505 16981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:58:49.376251 16984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:58:49.377233 16972 server_base.cc:1034] running on GCE node
I20250111 20:58:49.378293 16972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:58:49.396931 16972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:58:49.398526 16972 hybrid_clock.cc:648] HybridClock initialized: now 1736629129398390 us; error 108 us; skew 500 ppm
I20250111 20:58:49.399057 16972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:58:49.457005 16972 webserver.cc:458] Webserver started at http://127.14.226.7:40597/ using document root <none> and password file <none>
I20250111 20:58:49.458305 16972 fs_manager.cc:362] Metadata directory not provided
I20250111 20:58:49.458535 16972 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:58:49.530721 16972 fs_manager.cc:714] Time spent opening directory manager: real 0.069s user 0.016s sys 0.014s
I20250111 20:58:49.573043 16989 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:49.576988 16972 fs_manager.cc:730] Time spent opening block manager: real 0.041s user 0.011s sys 0.017s
I20250111 20:58:49.577257 16972 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
uuid: "49a14c4ba19447f8851a9a40990a4b91"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:58:49.578522 16972 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:58:49.807070 16972 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:58:49.808364 16972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:58:49.808719 16972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:58:49.849593 16972 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:58:49.859755 16972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:58:49.859906 16972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:49.860033 16972 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:58:49.860118 16972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:58:51.266721 16972 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.7:37147
I20250111 20:58:51.266793 17105 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.7:37147 every 8 connection(s)
I20250111 20:58:51.268798 16972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb
I20250111 20:58:51.279323 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 16972
I20250111 20:58:51.320771 17106 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:58:51.321089 17106 heartbeater.cc:463] Registering TS with master...
I20250111 20:58:51.322170 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:51.343425 15273 ts_manager.cc:194] Re-registered known tserver with Master: 49a14c4ba19447f8851a9a40990a4b91 (127.14.226.7:37147)
I20250111 20:58:51.344612 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.7:37261
I20250111 20:58:52.349443 17106 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 20:58:52.755782 15400 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:52.805367 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:52.828578 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:52.896677 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:52.899528 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:52.967180 15663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:53.042085 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:58:53.084733 16056 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:58:53.112957 15532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:58:55.119966 15273 ts_manager.cc:281] Unset tserver state for 18452380c023464488c18995b91fa513 from MAINTENANCE_MODE
I20250111 20:58:55.121560 16514 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.150136 15271 ts_manager.cc:281] Unset tserver state for 4e66492bc0e449a981045ff82ba305e8 from MAINTENANCE_MODE
I20250111 20:58:55.256789 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.270072 15465 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.354997 15728 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.361495 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.464833 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.514336 15271 ts_manager.cc:281] Unset tserver state for 49a14c4ba19447f8851a9a40990a4b91 from MAINTENANCE_MODE
I20250111 20:58:55.529414 16121 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.641265 15597 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:55.665378 16824 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:56.144287 16514 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:56.261014 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:56.274871 15465 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
W20250111 20:58:56.327029 16552 meta_cache.cc:1261] Time spent looking up entry by key: real 0.075s user 0.002s sys 0.000s
I20250111 20:58:56.360716 15728 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:56.366566 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:56.473551 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:58:58.627125 15240 maintenance_mode-itest.cc:745] Restarting batch of 3 tservers: eb9ae591b5d24eb18f58490f46e7dfaf,dc04179134174d03bd03d3db91b245f8,09be37535c0a473a956abea447348515
I20250111 20:58:59.801213 15271 ts_manager.cc:292] Set tserver state for eb9ae591b5d24eb18f58490f46e7dfaf to MAINTENANCE_MODE
I20250111 20:58:59.851548 15273 ts_manager.cc:292] Set tserver state for dc04179134174d03bd03d3db91b245f8 to MAINTENANCE_MODE
I20250111 20:59:00.645943 15270 ts_manager.cc:292] Set tserver state for 09be37535c0a473a956abea447348515 to MAINTENANCE_MODE
I20250111 20:59:02.564543 15663 tablet_service.cc:1422] Tablet server dc04179134174d03bd03d3db91b245f8 set to quiescing
I20250111 20:59:02.565135 15663 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:02.604017 15532 tablet_service.cc:1422] Tablet server eb9ae591b5d24eb18f58490f46e7dfaf set to quiescing
I20250111 20:59:02.604315 15532 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:05.118345 15400 tablet_service.cc:1422] Tablet server 09be37535c0a473a956abea447348515 set to quiescing
I20250111 20:59:05.118643 15400 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:07.289851 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15469
I20250111 20:59:07.316356 15591 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:59:07.586318 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15469
I20250111 20:59:07.616582 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.2:41707
--local_ip_for_outbound_sockets=127.14.226.2
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=39013
--webserver_interface=127.14.226.2
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:59:08.504496 17338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:59:08.513101 17338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:59:08.513384 17338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:59:08.529930 17338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:59:08.530364 17338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.2
I20250111 20:59:08.545986 17338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.2:41707
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.226.2
--webserver_port=39013
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.2
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:59:08.548342 17338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:59:08.550688 17338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:59:08.583966 17344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:08.585057 17345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:08.593932 17347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:08.600143 17338 server_base.cc:1034] running on GCE node
I20250111 20:59:08.601118 17338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:59:08.603035 17338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:59:08.604358 17338 hybrid_clock.cc:648] HybridClock initialized: now 1736629148604283 us; error 49 us; skew 500 ppm
I20250111 20:59:08.604775 17338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:59:08.608346 17338 webserver.cc:458] Webserver started at http://127.14.226.2:39013/ using document root <none> and password file <none>
I20250111 20:59:08.609216 17338 fs_manager.cc:362] Metadata directory not provided
I20250111 20:59:08.609377 17338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:59:08.647382 17338 fs_manager.cc:714] Time spent opening directory manager: real 0.036s user 0.006s sys 0.005s
I20250111 20:59:08.651633 17354 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:08.653484 17338 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20250111 20:59:08.653674 17338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
uuid: "eb9ae591b5d24eb18f58490f46e7dfaf"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:59:08.654598 17338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:59:08.753748 17338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:59:08.755918 17338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:59:08.756340 17338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:59:08.779211 17338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:59:08.786553 17338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:59:08.786772 17338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:08.786944 17338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:59:08.787079 17338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:09.432279 17338 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.2:41707
I20250111 20:59:09.434439 17338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb
I20250111 20:59:09.441622 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 17338
I20250111 20:59:09.441835 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15600
I20250111 20:59:09.455890 17468 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.2:41707 every 8 connection(s)
I20250111 20:59:09.489495 15723 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:59:09.491884 17469 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:59:09.492223 17469 heartbeater.cc:463] Registering TS with master...
I20250111 20:59:09.493253 17469 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:09.494963 15273 ts_manager.cc:194] Re-registered known tserver with Master: eb9ae591b5d24eb18f58490f46e7dfaf (127.14.226.2:41707)
I20250111 20:59:09.496101 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.2:44805
I20250111 20:59:09.892302 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15600
W20250111 20:59:09.977716 16010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.226.3:45529 (error 108)
I20250111 20:59:09.984180 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.3:45529
--local_ip_for_outbound_sockets=127.14.226.3
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=34305
--webserver_interface=127.14.226.3
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:59:09.998435 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 20:59:10.447165 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250111 20:59:10.499595 17469 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:59:10.667645 17479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:59:10.668138 17479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:59:10.668387 17479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:59:10.680881 17479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:59:10.681288 17479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.3
I20250111 20:59:10.696372 17479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.3:45529
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.226.3
--webserver_port=34305
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.3
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:59:10.698829 17479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:59:10.701310 17479 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:59:10.780619 17486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:10.763094 17485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:10.789862 17479 server_base.cc:1034] running on GCE node
W20250111 20:59:10.790788 17488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:10.791870 17479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:59:10.803643 17479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:59:10.807497 17479 hybrid_clock.cc:648] HybridClock initialized: now 1736629150807395 us; error 66 us; skew 500 ppm
I20250111 20:59:10.807942 17479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:59:10.866957 17479 webserver.cc:458] Webserver started at http://127.14.226.3:34305/ using document root <none> and password file <none>
I20250111 20:59:10.868965 17479 fs_manager.cc:362] Metadata directory not provided
I20250111 20:59:10.870079 17479 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:59:10.879566 17479 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20250111 20:59:10.899185 17497 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:10.903358 17479 fs_manager.cc:730] Time spent opening block manager: real 0.019s user 0.004s sys 0.005s
I20250111 20:59:10.903556 17479 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
uuid: "dc04179134174d03bd03d3db91b245f8"
format_stamp: "Formatted at 2025-01-11 20:58:31 on dist-test-slave-ntcq"
I20250111 20:59:10.904856 17479 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250111 20:59:10.960399 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250111 20:59:10.968667 17479 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:59:10.969837 17479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:59:10.970261 17479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:59:10.988799 17479 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:59:11.006145 17504 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20250111 20:59:11.017449 17479 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250111 20:59:11.017586 17479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.019s user 0.003s sys 0.004s
I20250111 20:59:11.017697 17479 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250111 20:59:11.028625 17504 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap starting.
I20250111 20:59:11.031414 17479 ts_tablet_manager.cc:610] Registered 1 tablets
I20250111 20:59:11.031533 17479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.014s user 0.004s sys 0.000s
W20250111 20:59:11.464069 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250111 20:59:11.899055 17610 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.3:45529 every 8 connection(s)
I20250111 20:59:11.899432 17479 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.3:45529
I20250111 20:59:11.901417 17479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb
I20250111 20:59:11.907229 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 17479
I20250111 20:59:11.907442 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15338
I20250111 20:59:11.986398 15459 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:59:12.062489 17611 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:59:12.062811 17611 heartbeater.cc:463] Registering TS with master...
I20250111 20:59:12.063970 17611 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:12.067503 15273 ts_manager.cc:194] Re-registered known tserver with Master: dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 20:59:12.069110 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.3:57043
W20250111 20:59:12.115332 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
I20250111 20:59:12.171880 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15338
I20250111 20:59:12.208096 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.1:38037
--local_ip_for_outbound_sockets=127.14.226.1
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=44911
--webserver_interface=127.14.226.1
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:59:12.651865 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
W20250111 20:59:13.000591 17622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:59:13.001093 17622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:59:13.001283 17622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:59:13.015766 17622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:59:13.016078 17622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.1
I20250111 20:59:13.017516 17504 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Log is configured to *not* fsync() on all Append() calls
I20250111 20:59:13.032541 17622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.1:38037
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.226.1
--webserver_port=44911
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.1
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:59:13.034857 17622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:59:13.037207 17622 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 20:59:13.075146 17611 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:59:13.079834 17628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:13.089221 17629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:13.102645 17622 server_base.cc:1034] running on GCE node
W20250111 20:59:13.107704 17631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:13.108765 17622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:59:13.115145 17622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:59:13.118643 17622 hybrid_clock.cc:648] HybridClock initialized: now 1736629153118527 us; error 77 us; skew 500 ppm
I20250111 20:59:13.119108 17622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:59:13.130545 17622 webserver.cc:458] Webserver started at http://127.14.226.1:44911/ using document root <none> and password file <none>
I20250111 20:59:13.134102 17622 fs_manager.cc:362] Metadata directory not provided
I20250111 20:59:13.134956 17622 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:59:13.186327 17622 fs_manager.cc:714] Time spent opening directory manager: real 0.046s user 0.004s sys 0.009s
W20250111 20:59:13.199105 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20250111 20:59:13.231256 17637 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:13.234653 17622 fs_manager.cc:730] Time spent opening block manager: real 0.046s user 0.008s sys 0.008s
I20250111 20:59:13.234843 17622 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
uuid: "09be37535c0a473a956abea447348515"
format_stamp: "Formatted at 2025-01-11 20:58:30 on dist-test-slave-ntcq"
I20250111 20:59:13.236022 17622 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 20:59:13.373235 17622 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:59:13.374440 17622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:59:13.374806 17622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:59:13.404273 17622 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:59:13.407876 17622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 20:59:13.408072 17622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250111 20:59:13.408228 17622 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 20:59:13.408350 17622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20250111 20:59:13.848448 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250111 20:59:14.317145 17270 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Peer dc04179134174d03bd03d3db91b245f8 is lagging by at least 4 ops behind the committed index
W20250111 20:59:14.461484 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
I20250111 20:59:14.560135 17754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.1:38037 every 8 connection(s)
I20250111 20:59:14.560472 17622 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.1:38037
I20250111 20:59:14.562561 17622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb
I20250111 20:59:14.567194 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 17622
I20250111 20:59:14.597337 17755 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:59:14.597678 17755 heartbeater.cc:463] Registering TS with master...
I20250111 20:59:14.598768 17755 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:14.601545 15273 ts_manager.cc:194] Re-registered known tserver with Master: 09be37535c0a473a956abea447348515 (127.14.226.1:38037)
I20250111 20:59:14.602728 15273 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.1:52681
W20250111 20:59:15.025889 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
I20250111 20:59:15.606560 17755 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:59:15.658692 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
I20250111 20:59:16.172266 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:16.222288 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:16.300230 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 20:59:16.321344 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
I20250111 20:59:16.358806 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:16.390504 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:16.487977 17546 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:16.537730 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:59:16.580426 16056 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:59:16.632543 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 20:59:16.922798 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
I20250111 20:59:17.411506 17771 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Peer dc04179134174d03bd03d3db91b245f8 is lagging by at least 234 ops behind the committed index [suppressed 25 similar messages]
W20250111 20:59:17.470075 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
I20250111 20:59:18.033812 17504 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap replaying log segment 1/1 (1.25M/3.78M this segment, stats: ops{read=746 overwritten=0 applied=742 ignored=0} inserts{seen=37050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 20:59:18.083130 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
W20250111 20:59:18.609544 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250111 20:59:19.168021 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250111 20:59:19.687810 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250111 20:59:20.141796 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250111 20:59:20.509814 17761 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Peer dc04179134174d03bd03d3db91b245f8 is lagging by at least 561 ops behind the committed index [suppressed 28 similar messages]
W20250111 20:59:20.664381 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
W20250111 20:59:21.162706 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
W20250111 20:59:21.569109 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.6:42445, user_credentials={real_user=slave}} blocked reactor thread for 94030.6us
W20250111 20:59:21.799288 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
W20250111 20:59:22.354030 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20250111 20:59:22.886811 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
I20250111 20:59:23.034528 17504 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap replaying log segment 1/1 (2.87M/3.78M this segment, stats: ops{read=1702 overwritten=0 applied=1702 ignored=0} inserts{seen=85050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 20:59:23.546532 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250111 20:59:23.561722 17805 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Peer dc04179134174d03bd03d3db91b245f8 is lagging by at least 793 ops behind the committed index [suppressed 26 similar messages]
W20250111 20:59:24.385138 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250111 20:59:25.069701 16010 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
I20250111 20:59:25.367924 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.422734 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.469338 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.471006 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.476500 17546 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.640799 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.663539 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:25.674856 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:59:25.678725 17504 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2240 overwritten=0 applied=2239 ignored=0} inserts{seen=111900 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250111 20:59:25.679625 17504 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Bootstrap complete.
I20250111 20:59:25.681737 17504 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Time spent bootstrapping tablet: real 14.654s user 10.038s sys 0.512s
I20250111 20:59:25.688786 17504 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:25.701320 17504 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc04179134174d03bd03d3db91b245f8, State: Initialized, Role: FOLLOWER
I20250111 20:59:25.702436 17504 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2239, Last appended: 1.2240, Last appended by leader: 2240, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:25.730057 17504 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Time spent starting tablet: real 0.048s user 0.032s sys 0.012s
I20250111 20:59:25.797291 16056 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:59:25.850207 17565 raft_consensus.cc:1212] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.2239->[1.2240-1.2664] Dedup: 1.2240->[1.2241-1.2664]
I20250111 20:59:26.271878 17873 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader edf2bd29095149dd9e4592ae2870d4ee)
I20250111 20:59:26.272161 17873 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:26.274487 17873 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:59:26.301584 16076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2664 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 20:59:26.358435 16336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2664 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 20:59:26.359345 17498 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc04179134174d03bd03d3db91b245f8; no voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee
I20250111 20:59:26.360153 17873 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 20:59:26.731859 17873 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader edf2bd29095149dd9e4592ae2870d4ee)
I20250111 20:59:26.732051 17873 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:26.732867 17873 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:59:26.734328 16076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3089 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 20:59:26.748385 17883 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [LEADER]: Peer dc04179134174d03bd03d3db91b245f8 is lagging by at least 152 ops behind the committed index [suppressed 15 similar messages]
I20250111 20:59:26.754990 16337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3089 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 20:59:26.755792 17498 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc04179134174d03bd03d3db91b245f8; no voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee
I20250111 20:59:26.756345 17873 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 20:59:27.138302 17873 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader edf2bd29095149dd9e4592ae2870d4ee)
I20250111 20:59:27.138477 17873 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:27.139220 17873 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:59:27.140197 16076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3514 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 20:59:27.140669 16338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3514 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 20:59:27.141563 17499 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc04179134174d03bd03d3db91b245f8; no voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee
I20250111 20:59:27.143069 17873 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250111 20:59:33.355460 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.510864 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.563218 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.661535 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.673836 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.697314 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.702836 17546 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:33.712072 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:59:33.836983 16056 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:59:35.482234 15273 ts_manager.cc:281] Unset tserver state for 09be37535c0a473a956abea447348515 from MAINTENANCE_MODE
I20250111 20:59:35.484644 16121 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.485065 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.485561 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.487435 17611 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.554443 17469 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.675465 17755 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.750397 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:35.857187 15277 ts_manager.cc:281] Unset tserver state for eb9ae591b5d24eb18f58490f46e7dfaf from MAINTENANCE_MODE
I20250111 20:59:35.901041 16824 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.044477 15277 ts_manager.cc:281] Unset tserver state for dc04179134174d03bd03d3db91b245f8 from MAINTENANCE_MODE
I20250111 20:59:36.236939 16514 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
W20250111 20:59:36.417183 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.6:42445, user_credentials={real_user=slave}} blocked reactor thread for 75143.7us
I20250111 20:59:36.490816 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.493752 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.503304 16121 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.514274 17611 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.558979 17469 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.679904 17755 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.789861 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:36.904857 16824 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:39.356505 15240 maintenance_mode-itest.cc:745] Restarting batch of 1 tservers: edf2bd29095149dd9e4592ae2870d4ee
I20250111 20:59:40.600843 15277 ts_manager.cc:292] Set tserver state for edf2bd29095149dd9e4592ae2870d4ee to MAINTENANCE_MODE
I20250111 20:59:43.061522 16056 tablet_service.cc:1422] Tablet server edf2bd29095149dd9e4592ae2870d4ee set to quiescing
I20250111 20:59:43.061762 16056 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:59:43.062459 18011 raft_consensus.cc:988] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: : Instructing follower dc04179134174d03bd03d3db91b245f8 to start an election
I20250111 20:59:43.062752 18011 raft_consensus.cc:1076] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 LEADER]: Signalling peer dc04179134174d03bd03d3db91b245f8 to start an election
I20250111 20:59:43.063870 17565 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973"
dest_uuid: "dc04179134174d03bd03d3db91b245f8"
from {username='slave'} at 127.14.226.6:37893
I20250111 20:59:43.064200 17565 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250111 20:59:43.064328 17565 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 1 FOLLOWER]: Advancing to term 2
I20250111 20:59:43.069515 17565 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:43.071053 16076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4896 } } ignore_live_leader: true dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee"
I20250111 20:59:43.071318 16076 raft_consensus.cc:3049] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 LEADER]: Stepping down as leader of term 1
I20250111 20:59:43.071444 16076 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 LEADER]: Becoming Follower/Learner. State: Replica: edf2bd29095149dd9e4592ae2870d4ee, State: Running, Role: LEADER
I20250111 20:59:43.071820 16076 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4896, Committed index: 4896, Last appended: 1.4897, Last appended by leader: 4897, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:43.072319 16336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "dc04179134174d03bd03d3db91b245f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4896 } } ignore_live_leader: true dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc"
I20250111 20:59:43.072378 16076 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 1 FOLLOWER]: Advancing to term 2
I20250111 20:59:43.072522 16336 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 1 FOLLOWER]: Advancing to term 2
I20250111 20:59:43.075618 16336 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc04179134174d03bd03d3db91b245f8 in term 2.
I20250111 20:59:43.077912 17565 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 election: Requested vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 20:59:43.079391 17498 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8; no voters:
I20250111 20:59:43.087706 16076 raft_consensus.cc:2405] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dc04179134174d03bd03d3db91b245f8 for term 2 because replica has last-logged OpId of term: 1 index: 4897, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4896.
W20250111 20:59:43.088675 16565 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56460: Illegal state: Replica edf2bd29095149dd9e4592ae2870d4ee is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } } }
I20250111 20:59:43.091503 18050 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 2 FOLLOWER]: Leader election won for term 2
I20250111 20:59:43.099850 18050 raft_consensus.cc:695] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [term 2 LEADER]: Becoming Leader. State: Replica: dc04179134174d03bd03d3db91b245f8, State: Running, Role: LEADER
I20250111 20:59:43.100721 18050 consensus_queue.cc:237] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4896, Committed index: 4896, Last appended: 1.4896, Last appended by leader: 4896, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 20:59:43.105732 15277 catalog_manager.cc:5526] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 reported cstate change: term changed from 1 to 2, leader changed from edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6) to dc04179134174d03bd03d3db91b245f8 (127.14.226.3). New cstate: current_term: 2 leader_uuid: "dc04179134174d03bd03d3db91b245f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } health_report { overall_health: UNKNOWN } } }
I20250111 20:59:43.164631 16336 raft_consensus.cc:1270] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Refusing update from remote peer dc04179134174d03bd03d3db91b245f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4896. Preceding OpId from leader: term: 2 index: 4899. (index mismatch)
I20250111 20:59:43.165405 18050 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4897, Last known committed idx: 4896, Time since last communication: 0.000s
I20250111 20:59:43.167577 16076 raft_consensus.cc:1270] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Refusing update from remote peer dc04179134174d03bd03d3db91b245f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4897. Preceding OpId from leader: term: 2 index: 4899. (index mismatch)
I20250111 20:59:43.168372 18050 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4897, Last known committed idx: 4896, Time since last communication: 0.000s
I20250111 20:59:43.168578 18050 consensus_queue.cc:1225] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee log is divergent from this leader: its last log entry 1.4897 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4896
I20250111 20:59:43.170076 16076 pending_rounds.cc:77] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Aborting all ops after (but not including) 4896
I20250111 20:59:43.170285 16076 pending_rounds.cc:99] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Aborting uncommitted WRITE_OP operation due to leader change: 1.4897
W20250111 20:59:43.170672 16076 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56460: Aborted: Op aborted by new leader
I20250111 20:59:43.213601 17872 mvcc.cc:204] Tried to move back new op lower bound from 7113233134231789568 to 7113233133987614720. Current Snapshot: MvccSnapshot[applied={T|T < 7113233134231789568 or (T in {7113233134231789568})}]
I20250111 20:59:43.226676 16570 mvcc.cc:204] Tried to move back new op lower bound from 7113233134231789568 to 7113233133987614720. Current Snapshot: MvccSnapshot[applied={T|T < 7113233134231789568 or (T in {7113233134231789568})}]
I20250111 20:59:45.300246 16056 tablet_service.cc:1422] Tablet server edf2bd29095149dd9e4592ae2870d4ee set to quiescing
I20250111 20:59:45.300531 16056 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:47.051021 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15993
I20250111 20:59:47.081967 16116 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 20:59:47.612425 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15993
W20250111 20:59:47.705358 17499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.226.6:42445 (error 108)
I20250111 20:59:47.706692 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.6:42445
--local_ip_for_outbound_sockets=127.14.226.6
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=33815
--webserver_interface=127.14.226.6
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 20:59:47.721382 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 20:59:48.160032 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250111 20:59:48.415146 18094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 20:59:48.415645 18094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 20:59:48.415911 18094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 20:59:48.428706 18094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 20:59:48.429119 18094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.6
I20250111 20:59:48.444391 18094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.6:42445
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
--webserver_interface=127.14.226.6
--webserver_port=33815
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.6
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 20:59:48.447315 18094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 20:59:48.449929 18094 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 20:59:48.486510 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:48.513290 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 20:59:48.591871 18104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 20:59:48.593329 18094 server_base.cc:1034] running on GCE node
I20250111 20:59:48.599871 18094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 20:59:48.610708 18094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 20:59:48.613343 18094 hybrid_clock.cc:648] HybridClock initialized: now 1736629188613229 us; error 75 us; skew 500 ppm
I20250111 20:59:48.615787 18094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 20:59:48.632015 18094 webserver.cc:458] Webserver started at http://127.14.226.6:33815/ using document root <none> and password file <none>
I20250111 20:59:48.633038 18094 fs_manager.cc:362] Metadata directory not provided
I20250111 20:59:48.633286 18094 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 20:59:48.641556 18094 fs_manager.cc:714] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20250111 20:59:48.647059 18110 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 20:59:48.649978 18094 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20250111 20:59:48.650353 18094 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
uuid: "edf2bd29095149dd9e4592ae2870d4ee"
format_stamp: "Formatted at 2025-01-11 20:58:33 on dist-test-slave-ntcq"
I20250111 20:59:48.653514 18094 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250111 20:59:48.660388 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250111 20:59:48.755026 18094 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 20:59:48.758934 18094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 20:59:48.759244 18094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 20:59:48.762177 18094 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 20:59:48.791517 18117 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20250111 20:59:48.806522 18094 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250111 20:59:48.806762 18094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.000s sys 0.001s
I20250111 20:59:48.806896 18094 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250111 20:59:48.811822 18094 ts_tablet_manager.cc:610] Registered 1 tablets
I20250111 20:59:48.812047 18094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.002s sys 0.002s
I20250111 20:59:48.812325 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap starting.
W20250111 20:59:49.166227 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250111 20:59:49.808066 18117 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Log is configured to *not* fsync() on all Append() calls
W20250111 20:59:49.822160 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20250111 20:59:50.224824 18094 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.6:42445
I20250111 20:59:50.224866 18228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.6:42445 every 8 connection(s)
I20250111 20:59:50.226797 18094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb
I20250111 20:59:50.228937 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 18094
I20250111 20:59:50.287240 18229 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 20:59:50.287640 18229 heartbeater.cc:463] Registering TS with master...
I20250111 20:59:50.288749 18229 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 20:59:50.292670 15277 ts_manager.cc:194] Re-registered known tserver with Master: edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445)
I20250111 20:59:50.294835 15277 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.6:58821
W20250111 20:59:50.339465 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
W20250111 20:59:50.821496 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20250111 20:59:51.323648 18229 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
W20250111 20:59:51.375856 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250111 20:59:51.604316 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.625795 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.784690 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.830941 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.867678 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:59:51.872431 18185 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 3 ops behind the committed index
I20250111 20:59:51.873378 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.950567 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:51.960075 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250111 20:59:52.061666 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
I20250111 20:59:52.093299 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 20:59:52.812856 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
W20250111 20:59:53.314461 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
W20250111 20:59:53.953668 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
W20250111 20:59:54.567644 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
I20250111 20:59:54.819630 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replaying log segment 1/2 (1.50M/8.00M this segment, stats: ops{read=869 overwritten=0 applied=866 ignored=0} inserts{seen=43250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 20:59:54.958050 18185 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 296 ops behind the committed index [suppressed 23 similar messages]
W20250111 20:59:55.106978 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
W20250111 20:59:55.769907 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
W20250111 20:59:56.212857 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250111 20:59:56.734884 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250111 20:59:57.199319 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250111 20:59:57.805538 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250111 20:59:58.010350 18185 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 612 ops behind the committed index [suppressed 28 similar messages]
W20250111 20:59:58.267462 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
W20250111 20:59:58.726750 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
W20250111 20:59:59.277837 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
I20250111 20:59:59.566356 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.578167 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 20:59:59.587751 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 20:59:59.595449 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.616379 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.629462 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.677361 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.698031 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.731420 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 20:59:59.822315 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replaying log segment 1/2 (2.94M/8.00M this segment, stats: ops{read=1700 overwritten=0 applied=1698 ignored=0} inserts{seen=84850 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 20:59:59.822690 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20250111 21:00:00.326526 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
W20250111 21:00:00.875545 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250111 21:00:01.019380 18185 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 828 ops behind the committed index [suppressed 28 similar messages]
W20250111 21:00:01.343420 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250111 21:00:01.846319 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
W20250111 21:00:02.402235 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 136: this message will repeat every 5th retry.
W20250111 21:00:02.880021 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 141: this message will repeat every 5th retry.
W20250111 21:00:03.405573 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 146: this message will repeat every 5th retry.
W20250111 21:00:03.974956 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 151: this message will repeat every 5th retry.
I20250111 21:00:04.044131 18358 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 1134 ops behind the committed index [suppressed 28 similar messages]
W20250111 21:00:04.520432 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 156: this message will repeat every 5th retry.
I20250111 21:00:04.827397 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replaying log segment 1/2 (4.77M/8.00M this segment, stats: ops{read=2758 overwritten=0 applied=2757 ignored=0} inserts{seen=137800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:00:05.092602 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 161: this message will repeat every 5th retry.
W20250111 21:00:05.653656 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 166: this message will repeat every 5th retry.
W20250111 21:00:06.240971 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 171: this message will repeat every 5th retry.
W20250111 21:00:06.956362 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 176: this message will repeat every 5th retry.
I20250111 21:00:07.117991 18185 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 1378 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:00:07.455756 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 181: this message will repeat every 5th retry.
I20250111 21:00:07.600246 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.627604 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.660301 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.674188 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.730520 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.758167 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:00:07.795405 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:00:07.800593 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:07.824558 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:00:08.034476 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 186: this message will repeat every 5th retry.
W20250111 21:00:08.598829 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 191: this message will repeat every 5th retry.
W20250111 21:00:09.045879 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 196: this message will repeat every 5th retry.
W20250111 21:00:09.581193 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 201: this message will repeat every 5th retry.
I20250111 21:00:09.834828 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replaying log segment 1/2 (6.40M/8.00M this segment, stats: ops{read=3696 overwritten=0 applied=3695 ignored=0} inserts{seen=184700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:00:10.173034 18411 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 1642 ops behind the committed index [suppressed 27 similar messages]
W20250111 21:00:10.226460 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 206: this message will repeat every 5th retry.
W20250111 21:00:10.829661 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 211: this message will repeat every 5th retry.
W20250111 21:00:11.384004 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 216: this message will repeat every 5th retry.
W20250111 21:00:11.909639 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 221: this message will repeat every 5th retry.
W20250111 21:00:12.419093 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 226: this message will repeat every 5th retry.
W20250111 21:00:13.074043 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 231: this message will repeat every 5th retry.
I20250111 21:00:13.212285 18436 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 1941 ops behind the committed index [suppressed 26 similar messages]
W20250111 21:00:13.601565 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 236: this message will repeat every 5th retry.
I20250111 21:00:14.128211 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replayed 1/2 log segments. Stats: ops{read=4621 overwritten=0 applied=4620 ignored=0} inserts{seen=230950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20250111 21:00:14.184835 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 241: this message will repeat every 5th retry.
W20250111 21:00:14.772667 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 246: this message will repeat every 5th retry.
I20250111 21:00:14.835062 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replaying log segment 2/2 (237.7K/1.04M this segment, stats: ops{read=4755 overwritten=0 applied=4753 ignored=0} inserts{seen=237600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:00:15.182165 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.188689 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.189819 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.198204 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.245873 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:00:15.248824 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.251108 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:00:15.297562 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:15.320711 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:00:15.349550 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 251: this message will repeat every 5th retry.
W20250111 21:00:15.845115 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 256: this message will repeat every 5th retry.
I20250111 21:00:16.224193 18462 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer edf2bd29095149dd9e4592ae2870d4ee is lagging by at least 2155 ops behind the committed index [suppressed 26 similar messages]
W20250111 21:00:16.359568 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 261: this message will repeat every 5th retry.
W20250111 21:00:16.882637 17499 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 266: this message will repeat every 5th retry.
I20250111 21:00:17.299926 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap replayed 2/2 log segments. Stats: ops{read=5231 overwritten=1 applied=5230 ignored=0} inserts{seen=261400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 21:00:17.300792 18117 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Bootstrap complete.
I20250111 21:00:17.305408 18117 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Time spent bootstrapping tablet: real 28.493s user 21.099s sys 0.767s
I20250111 21:00:17.312309 18117 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:00:17.313642 18117 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf2bd29095149dd9e4592ae2870d4ee, State: Initialized, Role: FOLLOWER
I20250111 21:00:17.314520 18117 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5230, Last appended: 2.5230, Last appended by leader: 5230, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:00:17.340312 18117 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Time spent starting tablet: real 0.035s user 0.024s sys 0.008s
I20250111 21:00:17.689954 18492 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:00:17.690227 18492 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:00:17.693059 18492 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:00:17.735026 16336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5655 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 21:00:17.743809 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 5655 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:00:17.745963 18112 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf2bd29095149dd9e4592ae2870d4ee; no voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8
I20250111 21:00:17.746448 18492 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:00:18.090296 18492 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:00:18.090484 18492 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:00:18.091115 18492 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:00:18.092023 16336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 6080 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 21:00:18.109575 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 6080 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:00:18.126458 18112 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf2bd29095149dd9e4592ae2870d4ee; no voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8
I20250111 21:00:18.127276 18492 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:00:18.520758 18492 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:00:18.520942 18492 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:00:18.521705 18492 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:00:18.522432 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 6505 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:00:18.523399 16336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 6505 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 21:00:18.528702 18111 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf2bd29095149dd9e4592ae2870d4ee; no voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8
I20250111 21:00:18.534361 18492 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250111 21:00:20.383872 18495 log.cc:926] Time spent T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
W20250111 21:00:20.880867 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 132040us
I20250111 21:00:22.844717 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.871150 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.884545 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.913621 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.935659 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.943810 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:22.946486 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:00:23.041954 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:00:23.114470 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:00:24.570389 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 160681us
I20250111 21:00:25.399627 15277 ts_manager.cc:281] Unset tserver state for edf2bd29095149dd9e4592ae2870d4ee from MAINTENANCE_MODE
I20250111 21:00:25.403362 16514 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.469564 16383 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.651230 17106 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.664963 16965 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.693292 17469 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.737807 18229 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.760258 17611 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:25.806810 17755 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:26.024134 16824 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:27.812351 15240 maintenance_mode-itest.cc:745] Restarting batch of 2 tservers: cc4301cbd2b84d22ab621b2c399022dc,4c1123e925164e978604ac78771cb3ad
I20250111 21:00:29.455294 15272 ts_manager.cc:292] Set tserver state for cc4301cbd2b84d22ab621b2c399022dc to MAINTENANCE_MODE
I20250111 21:00:29.803956 15277 ts_manager.cc:292] Set tserver state for 4c1123e925164e978604ac78771cb3ad to MAINTENANCE_MODE
I20250111 21:00:30.725816 15268 maintenance_manager.cc:418] P 6701e95595e54e48a427659232b86234: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033392
I20250111 21:00:31.348831 15267 maintenance_manager.cc:642] P 6701e95595e54e48a427659232b86234: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.622s user 0.034s sys 0.010s Metrics: {"bytes_written":7317,"cfile_init":1,"drs_written":1,"lbm_read_time_us":103,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1611,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":14,"thread_start_us":137606,"threads_started":1,"wal-append.queue_time_us":56006}
I20250111 21:00:31.377548 15268 maintenance_manager.cc:418] P 6701e95595e54e48a427659232b86234: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 666 bytes on disk
I20250111 21:00:31.378751 15267 maintenance_manager.cc:642] P 6701e95595e54e48a427659232b86234: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4}
I20250111 21:00:31.541850 16449 tablet_service.cc:1422] Tablet server 4c1123e925164e978604ac78771cb3ad set to quiescing
I20250111 21:00:31.542111 16449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:34.427660 16318 tablet_service.cc:1422] Tablet server cc4301cbd2b84d22ab621b2c399022dc set to quiescing
I20250111 21:00:34.427919 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:00:35.203303 16384 maintenance_manager.cc:418] P cc4301cbd2b84d22ab621b2c399022dc: Scheduling FlushMRSOp(54a6628492fa41d593f0c59af04ad973): perf score=0.047439
I20250111 21:00:39.945304 16318 tablet_service.cc:1422] Tablet server cc4301cbd2b84d22ab621b2c399022dc set to quiescing
I20250111 21:00:39.945607 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
W20250111 21:00:44.240432 18495 log.cc:494] Time spent T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee: Log roll took a long time: real 0.060s user 0.010s sys 0.000s
I20250111 21:00:44.992815 16318 tablet_service.cc:1422] Tablet server cc4301cbd2b84d22ab621b2c399022dc set to quiescing
I20250111 21:00:44.993113 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 2 scanners
W20250111 21:00:45.492846 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 53655.4us
I20250111 21:00:48.114518 16318 tablet_service.cc:1422] Tablet server cc4301cbd2b84d22ab621b2c399022dc set to quiescing
I20250111 21:00:48.114797 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 1 scanners
I20250111 21:00:52.034832 16318 tablet_service.cc:1422] Tablet server cc4301cbd2b84d22ab621b2c399022dc set to quiescing
I20250111 21:00:52.035125 16318 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:00:53.839093 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16255
I20250111 21:00:53.874158 16378 generic_service.cc:196] Checking for leaks (request via RPC)
W20250111 21:00:54.358434 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 49095us
I20250111 21:00:54.416163 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16255
W20250111 21:00:54.508649 17498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.226.8:41133 (error 108) [suppressed 24 similar messages]
I20250111 21:00:54.511309 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.8:41133
--local_ip_for_outbound_sockets=127.14.226.8
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=33315
--webserver_interface=127.14.226.8
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 21:00:54.524361 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.8:41133: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 21:00:54.948833 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.8:41133: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250111 21:00:55.192302 18787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 21:00:55.192625 18787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 21:00:55.192785 18787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 21:00:55.205971 18787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 21:00:55.206427 18787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.8
I20250111 21:00:55.222220 18787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.8:41133
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
--webserver_interface=127.14.226.8
--webserver_port=33315
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.8
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 21:00:55.224275 18787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 21:00:55.226562 18787 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250111 21:00:55.278538 18793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 21:00:55.279645 18794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 21:00:55.279955 18796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 21:00:55.304003 18787 server_base.cc:1034] running on GCE node
I20250111 21:00:55.304996 18787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 21:00:55.322211 18787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250111 21:00:55.330641 18787 hybrid_clock.cc:648] HybridClock initialized: now 1736629255330432 us; error 145 us; skew 500 ppm
I20250111 21:00:55.333721 18787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 21:00:55.359580 18787 webserver.cc:458] Webserver started at http://127.14.226.8:33315/ using document root <none> and password file <none>
I20250111 21:00:55.360847 18787 fs_manager.cc:362] Metadata directory not provided
I20250111 21:00:55.361388 18787 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 21:00:55.377820 18787 fs_manager.cc:714] Time spent opening directory manager: real 0.011s user 0.005s sys 0.002s
I20250111 21:00:55.414995 18803 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 21:00:55.418058 18787 fs_manager.cc:730] Time spent opening block manager: real 0.038s user 0.002s sys 0.003s
I20250111 21:00:55.418427 18787 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
uuid: "cc4301cbd2b84d22ab621b2c399022dc"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 21:00:55.421131 18787 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 6 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 21:00:55.457828 18787 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 21:00:55.464406 18787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 21:00:55.464857 18787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 21:00:55.471249 18787 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 21:00:55.475898 18810 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20250111 21:00:55.497753 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.8:41133: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250111 21:00:55.504832 18787 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250111 21:00:55.505002 18787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.030s user 0.000s sys 0.001s
I20250111 21:00:55.505151 18787 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250111 21:00:55.511224 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap starting.
I20250111 21:00:55.520612 18787 ts_tablet_manager.cc:610] Registered 1 tablets
I20250111 21:00:55.520810 18787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.016s user 0.003s sys 0.002s
W20250111 21:00:55.978677 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.8:41133: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20250111 21:00:56.506556 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.8:41133: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20250111 21:00:56.642145 18787 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.8:41133
I20250111 21:00:56.642220 18918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.8:41133 every 8 connection(s)
I20250111 21:00:56.644357 18787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb
I20250111 21:00:56.649916 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 18787
I20250111 21:00:56.650199 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16386
I20250111 21:00:56.689818 18810 log.cc:826] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Log is configured to *not* fsync() on all Append() calls
I20250111 21:00:56.709836 16509 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:00:56.786509 18920 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 21:00:56.786921 18920 heartbeater.cc:463] Registering TS with master...
I20250111 21:00:56.788081 18920 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:56.793287 15277 ts_manager.cc:194] Re-registered known tserver with Master: cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:00:56.794867 15277 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.8:34609
I20250111 21:00:57.037523 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16386
I20250111 21:00:57.085373 15240 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskBASTb8/build/asan/bin/kudu
/tmp/dist-test-taskBASTb8/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.226.9:39547
--local_ip_for_outbound_sockets=127.14.226.9
--tserver_master_addrs=127.14.226.62:37841
--webserver_port=33127
--webserver_interface=127.14.226.9
--builtin_ntp_servers=127.14.226.20:36611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20250111 21:00:57.286458 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 26: this message will repeat every 5th retry.
W20250111 21:00:57.754372 18929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 21:00:57.754901 18929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250111 21:00:57.755125 18929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 21:00:57.767881 18929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250111 21:00:57.768277 18929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.226.9
I20250111 21:00:57.785786 18929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.226.20:36611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.226.9:39547
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
--webserver_interface=127.14.226.9
--webserver_port=33127
--tserver_master_addrs=127.14.226.62:37841
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.226.9
--log_dir=/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision bda246888fb83d6b8131ddbb04f062f3e03180a8
build type FASTDEBUG
built by None at 11 Jan 2025 20:43:24 UTC on 24a791456cd2
build id 4131
ASAN enabled
I20250111 21:00:57.789412 18929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250111 21:00:57.792927 18929 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250111 21:00:57.799530 18920 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 21:00:57.831928 18929 server_base.cc:1034] running on GCE node
W20250111 21:00:57.832794 18937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 21:00:57.833344 18940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250111 21:00:57.837378 18938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250111 21:00:57.838557 18929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250111 21:00:57.870661 18929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20250111 21:00:57.872304 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 31: this message will repeat every 5th retry.
I20250111 21:00:57.872661 18929 hybrid_clock.cc:648] HybridClock initialized: now 1736629257872556 us; error 81 us; skew 500 ppm
I20250111 21:00:57.873024 18929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250111 21:00:57.936278 18929 webserver.cc:458] Webserver started at http://127.14.226.9:33127/ using document root <none> and password file <none>
I20250111 21:00:57.937651 18929 fs_manager.cc:362] Metadata directory not provided
I20250111 21:00:57.937911 18929 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250111 21:00:57.960696 18929 fs_manager.cc:714] Time spent opening directory manager: real 0.020s user 0.005s sys 0.001s
I20250111 21:00:57.972391 18946 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250111 21:00:57.974408 18929 fs_manager.cc:730] Time spent opening block manager: real 0.012s user 0.009s sys 0.000s
I20250111 21:00:57.974618 18929 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data,/tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
uuid: "4c1123e925164e978604ac78771cb3ad"
format_stamp: "Formatted at 2025-01-11 20:58:34 on dist-test-slave-ntcq"
I20250111 21:00:57.975909 18929 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
metadata directory: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/wal
1 data directories: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250111 21:00:58.024806 18929 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250111 21:00:58.026027 18929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250111 21:00:58.026559 18929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250111 21:00:58.100204 18929 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250111 21:00:58.113219 18929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250111 21:00:58.113432 18929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250111 21:00:58.113600 18929 ts_tablet_manager.cc:610] Registered 0 tablets
I20250111 21:00:58.113724 18929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20250111 21:00:58.503489 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 36: this message will repeat every 5th retry.
I20250111 21:00:58.695256 18972 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 2 ops behind the committed index
W20250111 21:00:59.278518 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 41: this message will repeat every 5th retry.
I20250111 21:00:59.414292 19064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.226.9:39547 every 8 connection(s)
I20250111 21:00:59.414633 18929 rpc_server.cc:307] RPC server started. Bound to: 127.14.226.9:39547
I20250111 21:00:59.416759 18929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb
I20250111 21:00:59.419425 15240 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskBASTb8/build/asan/bin/kudu as pid 18929
I20250111 21:00:59.477414 19065 heartbeater.cc:346] Connected to a master server at 127.14.226.62:37841
I20250111 21:00:59.477788 19065 heartbeater.cc:463] Registering TS with master...
I20250111 21:00:59.480607 19065 heartbeater.cc:510] Master 127.14.226.62:37841 requested a full tablet report, sending...
I20250111 21:00:59.482887 15277 ts_manager.cc:194] Re-registered known tserver with Master: 4c1123e925164e978604ac78771cb3ad (127.14.226.9:39547)
I20250111 21:00:59.484005 15277 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.226.9:46945
W20250111 21:00:59.890906 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 46: this message will repeat every 5th retry.
W20250111 21:01:00.378050 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 51: this message will repeat every 5th retry.
I20250111 21:01:00.491952 19065 heartbeater.cc:502] Master 127.14.226.62:37841 was elected leader, sending a full tablet report...
I20250111 21:01:00.898885 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:00.923177 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:00.941932 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 56: this message will repeat every 5th retry.
I20250111 21:01:00.945076 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:01.027801 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:01.084286 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:01.168839 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:01.190438 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:01.211819 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:01.235185 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250111 21:01:01.537708 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 61: this message will repeat every 5th retry.
I20250111 21:01:01.694043 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 1/3 (1.27M/8.00M this segment, stats: ops{read=747 overwritten=0 applied=747 ignored=0} inserts{seen=37300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:01.720280 18973 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 228 ops behind the committed index [suppressed 24 similar messages]
W20250111 21:01:02.011271 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 66: this message will repeat every 5th retry.
W20250111 21:01:02.538872 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 71: this message will repeat every 5th retry.
W20250111 21:01:03.061058 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 76: this message will repeat every 5th retry.
W20250111 21:01:03.600114 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 81: this message will repeat every 5th retry.
W20250111 21:01:04.052011 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 86: this message will repeat every 5th retry.
W20250111 21:01:04.600085 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 91: this message will repeat every 5th retry.
I20250111 21:01:04.792196 18704 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 554 ops behind the committed index [suppressed 29 similar messages]
W20250111 21:01:05.139725 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 96: this message will repeat every 5th retry.
W20250111 21:01:05.673491 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 101: this message will repeat every 5th retry.
W20250111 21:01:06.194504 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 106: this message will repeat every 5th retry.
I20250111 21:01:06.694702 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 1/3 (2.68M/8.00M this segment, stats: ops{read=1581 overwritten=0 applied=1580 ignored=0} inserts{seen=78950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:06.752334 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20250111 21:01:07.511551 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 116: this message will repeat every 5th retry.
I20250111 21:01:07.851359 18704 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 863 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:01:08.124384 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20250111 21:01:08.275816 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.299860 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.417799 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.446619 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.496793 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.498447 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:08.506759 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:08.534420 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:08.577873 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:08.706499 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 126: this message will repeat every 5th retry.
W20250111 21:01:09.159718 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 131: this message will repeat every 5th retry.
W20250111 21:01:09.674503 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 136: this message will repeat every 5th retry.
W20250111 21:01:10.149489 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 141: this message will repeat every 5th retry.
W20250111 21:01:10.662220 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 146: this message will repeat every 5th retry.
I20250111 21:01:10.852843 19127 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 1135 ops behind the committed index [suppressed 28 similar messages]
W20250111 21:01:11.167714 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 151: this message will repeat every 5th retry.
W20250111 21:01:11.654474 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 156: this message will repeat every 5th retry.
I20250111 21:01:11.695415 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 1/3 (4.33M/8.00M this segment, stats: ops{read=2548 overwritten=0 applied=2547 ignored=0} inserts{seen=127300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:12.180712 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 161: this message will repeat every 5th retry.
W20250111 21:01:12.689249 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 166: this message will repeat every 5th retry.
W20250111 21:01:13.244280 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 171: this message will repeat every 5th retry.
W20250111 21:01:13.733002 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 176: this message will repeat every 5th retry.
I20250111 21:01:13.940712 19184 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 1455 ops behind the committed index [suppressed 29 similar messages]
W20250111 21:01:14.312352 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 181: this message will repeat every 5th retry.
W20250111 21:01:14.934850 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 186: this message will repeat every 5th retry.
W20250111 21:01:15.596817 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 191: this message will repeat every 5th retry.
I20250111 21:01:15.961534 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:15.962359 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:15.998070 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:16.059106 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:16.147895 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:16.152613 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:16.214323 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:16.226868 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:16.240177 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 196: this message will repeat every 5th retry.
I20250111 21:01:16.294689 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:16.696435 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 1/3 (6.15M/8.00M this segment, stats: ops{read=3619 overwritten=0 applied=3618 ignored=0} inserts{seen=180850 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:16.816009 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 201: this message will repeat every 5th retry.
I20250111 21:01:17.017067 19190 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 1706 ops behind the committed index [suppressed 24 similar messages]
W20250111 21:01:17.391371 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 206: this message will repeat every 5th retry.
W20250111 21:01:17.882156 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 211: this message will repeat every 5th retry.
W20250111 21:01:18.371193 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 216: this message will repeat every 5th retry.
W20250111 21:01:18.856873 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 221: this message will repeat every 5th retry.
W20250111 21:01:19.267488 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 44842us
W20250111 21:01:19.422659 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 226: this message will repeat every 5th retry.
W20250111 21:01:19.976524 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 231: this message will repeat every 5th retry.
I20250111 21:01:20.065493 19190 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 2020 ops behind the committed index [suppressed 28 similar messages]
W20250111 21:01:20.581465 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 236: this message will repeat every 5th retry.
W20250111 21:01:21.136068 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 241: this message will repeat every 5th retry.
W20250111 21:01:21.674237 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 246: this message will repeat every 5th retry.
I20250111 21:01:21.699433 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 1/3 (7.97M/8.00M this segment, stats: ops{read=4687 overwritten=0 applied=4686 ignored=0} inserts{seen=234250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:21.779783 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replayed 1/3 log segments. Stats: ops{read=4706 overwritten=0 applied=4704 ignored=0} inserts{seen=235150 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
W20250111 21:01:22.200645 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 251: this message will repeat every 5th retry.
W20250111 21:01:22.795861 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 256: this message will repeat every 5th retry.
I20250111 21:01:23.108363 19179 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 2317 ops behind the committed index [suppressed 26 similar messages]
W20250111 21:01:23.418459 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 261: this message will repeat every 5th retry.
I20250111 21:01:23.547838 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.625989 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.634992 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.681737 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.693123 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.721333 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:23.741564 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:23.820302 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:23.861279 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:24.041002 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 266: this message will repeat every 5th retry.
W20250111 21:01:24.564774 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 271: this message will repeat every 5th retry.
W20250111 21:01:25.070647 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 276: this message will repeat every 5th retry.
W20250111 21:01:25.605774 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 281: this message will repeat every 5th retry.
I20250111 21:01:25.812206 17614 maintenance_manager.cc:418] P dc04179134174d03bd03d3db91b245f8: Scheduling FlushMRSOp(54a6628492fa41d593f0c59af04ad973): perf score=0.068756
W20250111 21:01:26.138649 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 286: this message will repeat every 5th retry.
I20250111 21:01:26.176144 19310 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 2603 ops behind the committed index [suppressed 27 similar messages]
W20250111 21:01:26.669456 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 291: this message will repeat every 5th retry.
I20250111 21:01:26.696842 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 2/3 (1.46M/8.00M this segment, stats: ops{read=5562 overwritten=0 applied=5561 ignored=0} inserts{seen=277950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:27.193100 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 296: this message will repeat every 5th retry.
W20250111 21:01:27.880772 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 301: this message will repeat every 5th retry.
W20250111 21:01:28.616200 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 306: this message will repeat every 5th retry.
W20250111 21:01:29.082449 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 311: this message will repeat every 5th retry.
I20250111 21:01:29.284598 19316 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 2836 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:01:29.719251 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 316: this message will repeat every 5th retry.
W20250111 21:01:30.325160 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 321: this message will repeat every 5th retry.
W20250111 21:01:30.925019 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 326: this message will repeat every 5th retry.
I20250111 21:01:31.201891 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.213889 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.283946 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.368382 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:31.447358 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 331: this message will repeat every 5th retry.
I20250111 21:01:31.497011 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.547750 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.601130 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:31.698910 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 2/3 (2.96M/8.00M this segment, stats: ops{read=6446 overwritten=0 applied=6446 ignored=0} inserts{seen=322200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:31.701301 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:31.719571 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
W20250111 21:01:31.989306 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 336: this message will repeat every 5th retry.
I20250111 21:01:32.312583 19367 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 3021 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:01:32.720700 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 341: this message will repeat every 5th retry.
W20250111 21:01:33.238706 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 346: this message will repeat every 5th retry.
W20250111 21:01:33.773622 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 351: this message will repeat every 5th retry.
W20250111 21:01:34.260769 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 356: this message will repeat every 5th retry.
W20250111 21:01:34.864055 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 361: this message will repeat every 5th retry.
I20250111 21:01:35.321978 19319 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 3256 ops behind the committed index [suppressed 26 similar messages]
W20250111 21:01:35.355540 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 366: this message will repeat every 5th retry.
W20250111 21:01:35.985899 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 371: this message will repeat every 5th retry.
W20250111 21:01:36.660756 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 376: this message will repeat every 5th retry.
I20250111 21:01:36.699419 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 2/3 (4.75M/8.00M this segment, stats: ops{read=7498 overwritten=0 applied=7498 ignored=0} inserts{seen=374800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:37.138414 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 381: this message will repeat every 5th retry.
W20250111 21:01:37.913300 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 386: this message will repeat every 5th retry.
I20250111 21:01:38.393342 19378 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 3449 ops behind the committed index [suppressed 24 similar messages]
W20250111 21:01:38.478505 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 391: this message will repeat every 5th retry.
W20250111 21:01:39.046550 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 396: this message will repeat every 5th retry.
I20250111 21:01:39.437156 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.459308 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.498605 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.508399 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.511263 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:39.539206 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.540670 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:39.644192 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:39.646320 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20250111 21:01:39.855324 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 401: this message will repeat every 5th retry.
W20250111 21:01:40.450861 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 406: this message will repeat every 5th retry.
W20250111 21:01:41.001276 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 411: this message will repeat every 5th retry.
I20250111 21:01:41.433311 19378 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 3649 ops behind the committed index [suppressed 24 similar messages]
W20250111 21:01:41.465822 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 416: this message will repeat every 5th retry.
I20250111 21:01:41.700464 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 2/3 (6.20M/8.00M this segment, stats: ops{read=8356 overwritten=0 applied=8343 ignored=0} inserts{seen=417050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:42.036298 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 421: this message will repeat every 5th retry.
W20250111 21:01:42.553038 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 426: this message will repeat every 5th retry.
W20250111 21:01:43.002458 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 431: this message will repeat every 5th retry.
W20250111 21:01:43.496445 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 436: this message will repeat every 5th retry.
W20250111 21:01:44.018458 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 441: this message will repeat every 5th retry.
W20250111 21:01:44.490698 16525 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.14.226.3:45529, user_credentials={real_user=slave}} blocked reactor thread for 91221.1us
I20250111 21:01:44.519766 19437 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 3905 ops behind the committed index [suppressed 29 similar messages]
W20250111 21:01:44.588510 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 446: this message will repeat every 5th retry.
W20250111 21:01:45.099190 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 451: this message will repeat every 5th retry.
W20250111 21:01:45.581351 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 456: this message will repeat every 5th retry.
W20250111 21:01:46.054672 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 461: this message will repeat every 5th retry.
W20250111 21:01:46.636058 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 466: this message will repeat every 5th retry.
I20250111 21:01:46.718748 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 2/3 (7.58M/8.00M this segment, stats: ops{read=9164 overwritten=0 applied=9163 ignored=0} inserts{seen=458050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:46.775913 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:46.787418 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:46.880275 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:46.892625 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:46.895350 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:46.989630 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:47.007251 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:47.013536 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20250111 21:01:47.038506 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
W20250111 21:01:47.171280 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 471: this message will repeat every 5th retry.
I20250111 21:01:47.573451 19437 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 4100 ops behind the committed index [suppressed 28 similar messages]
W20250111 21:01:47.736337 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 476: this message will repeat every 5th retry.
W20250111 21:01:48.249912 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 481: this message will repeat every 5th retry.
W20250111 21:01:48.805160 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 486: this message will repeat every 5th retry.
I20250111 21:01:48.834193 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replayed 2/3 log segments. Stats: ops{read=9410 overwritten=0 applied=9409 ignored=0} inserts{seen=470350 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20250111 21:01:49.331324 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 491: this message will repeat every 5th retry.
W20250111 21:01:50.030437 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 496: this message will repeat every 5th retry.
W20250111 21:01:50.526360 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 501: this message will repeat every 5th retry.
I20250111 21:01:50.613945 19328 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 4351 ops behind the committed index [suppressed 26 similar messages]
W20250111 21:01:51.164003 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 506: this message will repeat every 5th retry.
I20250111 21:01:51.710495 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 3/3 (579.2K/1.77M this segment, stats: ops{read=9742 overwritten=0 applied=9741 ignored=0} inserts{seen=486950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
W20250111 21:01:51.711535 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 511: this message will repeat every 5th retry.
W20250111 21:01:52.241742 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 516: this message will repeat every 5th retry.
W20250111 21:01:52.711491 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 521: this message will repeat every 5th retry.
W20250111 21:01:53.242506 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 526: this message will repeat every 5th retry.
I20250111 21:01:53.671723 19502 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 4575 ops behind the committed index [suppressed 27 similar messages]
W20250111 21:01:53.888170 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 531: this message will repeat every 5th retry.
W20250111 21:01:54.486828 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 536: this message will repeat every 5th retry.
W20250111 21:01:55.000485 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 541: this message will repeat every 5th retry.
I20250111 21:01:55.083194 16756 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.087261 17689 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.121188 19000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.139308 17039 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.157052 18852 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.242862 18159 tablet_service.cc:1429] Tablet server has 0 leaders and 3 scanners
I20250111 21:01:55.304369 17402 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.329993 16897 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20250111 21:01:55.393018 17546 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20250111 21:01:55.732434 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 546: this message will repeat every 5th retry.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:753: Failure
Failed
Bad status: Timed out: cluster didn't become healthy: Running 'cluster ksck 127.14.226.62:37841' did not succeed in 60 seconds: Runtime error: error running 'cluster ksck 127.14.226.62:37841': stdout: Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
6701e95595e54e48a427659232b86234 | 127.14.226.62:37841 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.14.226.20:36611 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server States
Server | State
----------------------------------+------------------
4c1123e925164e978604ac78771cb3ad | MAINTENANCE_MODE
cc4301cbd2b84d22ab621b2c399022dc | MAINTENANCE_MODE
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
09be37535c0a473a956abea447348515 | 127.14.226.1:38037 | HEALTHY | /L0 | 0 | 0
18452380c023464488c18995b91fa513 | 127.14.226.5:41645 | HEALTHY | /L1 | 0 | 0
49a14c4ba19447f8851a9a40990a4b91 | 127.14.226.7:37147 | HEALTHY | /L1 | 0 | 0
4c1123e925164e978604ac78771cb3ad | 127.14.226.9:39547 | HEALTHY | /L2 | 0 | 0
4e66492bc0e449a981045ff82ba305e8 | 127.14.226.4:33171 | HEALTHY | /L1 | 0 | 0
cc4301cbd2b84d22ab621b2c399022dc | 127.14.226.8:41133 | HEALTHY | /L2 | 0 | 0
dc04179134174d03bd03d3db91b245f8 | 127.14.226.3:45529 | HEALTHY | /L0 | 1 | 0
eb9ae591b5d24eb18f58490f46e7dfaf | 127.14.226.2:41707 | HEALTHY | /L0 | 0 | 0
edf2bd29095149dd9e4592ae2870d4ee | 127.14.226.6:42445 | HEALTHY | /L1 | 0 | 3
Tablet Server Location Summary
Location | Count
----------+---------
/L2 | 2
/L0 | 3
/L1 | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.14.226.1 | experimental | 127.14.226.1:38037
local_ip_for_outbound_sockets | 127.14.226.2 | experimental | 127.14.226.2:41707
local_ip_for_outbound_sockets | 127.14.226.3 | experimental | 127.14.226.3:45529
local_ip_for_outbound_sockets | 127.14.226.4 | experimental | 127.14.226.4:33171
local_ip_for_outbound_sockets | 127.14.226.5 | experimental | 127.14.226.5:41645
local_ip_for_outbound_sockets | 127.14.226.6 | experimental | 127.14.226.6:42445
local_ip_for_outbound_sockets | 127.14.226.7 | experimental | 127.14.226.7:37147
local_ip_for_outbound_sockets | 127.14.226.8 | experimental | 127.14.226.8:41133
local_ip_for_outbound_sockets | 127.14.226.9 | experimental | 127.14.226.9:39547
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-0/data/info.pb | hidden | 127.14.226.1:38037
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-1/data/info.pb | hidden | 127.14.226.2:41707
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/data/info.pb | hidden | 127.14.226.3:45529
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-3/data/info.pb | hidden | 127.14.226.4:33171
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-4/data/info.pb | hidden | 127.14.226.5:41645
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-5/data/info.pb | hidden | 127.14.226.6:42445
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-6/data/info.pb | hidden | 127.14.226.7:37147
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-7/data/info.pb | hidden | 127.14.226.8:41133
server_dump_info_path | /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-8/data/info.pb | hidden | 127.14.226.9:39547
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.14.226.20:36611 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
Tablet 54a6628492fa41d593f0c59af04ad973 of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): RUNNING
dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): RUNNING [LEADER]
cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap replaying log segment 3/3 (579.2K/1.77M this segment, stats: ops{read=9742 overwritten=0 applied=9741 ignored=0} inserts{seen=486950 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
All reported replicas are:
A = edf2bd29095149dd9e4592ae2870d4ee
B = dc04179134174d03bd03d3db91b245f8
C = cc4301cbd2b84d22ab621b2c399022dc
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B* C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNDER_REPLICATED | 1 | 0 | 0 | 1 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
, stderr: W20250111 21:01:54.147111 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250111 21:01:54.147701 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250111 21:01:54.161664 19505 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Runtime error: ksck discovered errors
: /tmp/dist-test-taskBASTb8/build/asan/bin/kudu: process exited with non-zero status 1
W20250111 21:01:56.234683 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 551: this message will repeat every 5th retry.
I20250111 21:01:56.712366 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replaying log segment 3/3 (1.72M/1.77M this segment, stats: ops{read=10423 overwritten=0 applied=10422 ignored=0} inserts{seen=521000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20250111 21:01:56.724895 19557 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 4725 ops behind the committed index [suppressed 25 similar messages]
W20250111 21:01:56.744405 17498 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 -> Peer cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133): Couldn't send request to peer cc4301cbd2b84d22ab621b2c399022dc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 556: this message will repeat every 5th retry.
I20250111 21:01:56.829862 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap replayed 3/3 log segments. Stats: ops{read=10452 overwritten=0 applied=10452 ignored=0} inserts{seen=522500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250111 21:01:56.830878 18810 tablet_bootstrap.cc:492] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Bootstrap complete.
I20250111 21:01:56.837132 18810 ts_tablet_manager.cc:1397] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent bootstrapping tablet: real 61.326s user 42.496s sys 1.360s
I20250111 21:01:56.843802 18810 raft_consensus.cc:357] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:56.845100 18810 raft_consensus.cc:738] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc4301cbd2b84d22ab621b2c399022dc, State: Initialized, Role: FOLLOWER
I20250111 21:01:56.846081 18810 consensus_queue.cc:260] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10452, Last appended: 2.10452, Last appended by leader: 10452, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:56.860318 18810 ts_tablet_manager.cc:1428] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc: Time spent starting tablet: real 0.023s user 0.005s sys 0.016s
I20250111 21:01:57.463286 19561 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:57.463577 19561 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:57.465703 19561 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:57.498972 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 11302 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:57.516265 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 11302 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:57.517169 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:57.759892 19561 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:58.590710 19573 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:58.590938 19573 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:58.591871 19573 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:58.592552 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13002 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:58.594054 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13002 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:58.594775 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:58.595048 19573 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.236052 19573 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:59.236269 19573 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:59.237141 19573 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:59.237602 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13852 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:59.237715 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 13852 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:59.238401 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:59.238818 19573 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.861016 19575 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:01:59.861251 19575 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:01:59.862351 19575 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:01:59.863061 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 14702 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:01:59.863251 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 14702 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:01:59.864004 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:01:59.864497 19575 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:01:59.871767 19574 consensus_queue.cc:786] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8 [LEADER]: Peer cc4301cbd2b84d22ab621b2c399022dc is lagging by at least 475 ops behind the committed index [suppressed 11 similar messages]
I20250111 21:02:00.216840 19575 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:00.217058 19575 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:00.217801 19575 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:00.218720 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15127 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:02:00.220028 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15127 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:02:00.221004 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc; no voters: dc04179134174d03bd03d3db91b245f8, edf2bd29095149dd9e4592ae2870d4ee
I20250111 21:02:00.221526 19575 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:02:01.796489 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: FlushMRSOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 35.984s user 23.499s sys 0.728s Metrics: {"bytes_written":18669649,"cfile_init":1,"compiler_manager_pool.queue_time_us":718,"compiler_manager_pool.run_cpu_time_us":705010,"compiler_manager_pool.run_wall_time_us":1338006,"dirs.queue_time_us":6719,"dirs.run_cpu_time_us":288,"dirs.run_wall_time_us":6333,"drs_written":1,"lbm_read_time_us":151,"lbm_reads_lt_1ms":4,"lbm_write_time_us":142541,"lbm_writes_1-10_ms":8,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":806,"peak_mem_usage":0,"rows_written":672750,"spinlock_wait_cycles":653696,"thread_start_us":12492,"threads_started":3,"wal-append.queue_time_us":2587}
I20250111 21:02:01.798480 17614 maintenance_manager.cc:418] P dc04179134174d03bd03d3db91b245f8: Scheduling LogGCOp(54a6628492fa41d593f0c59af04ad973): free 16781455 bytes of WAL
I20250111 21:02:01.798970 17502 log_reader.cc:385] T 54a6628492fa41d593f0c59af04ad973: removed 2 log segments from log reader
I20250111 21:02:01.799135 17502 log.cc:1078] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Deleting log segment in path: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/wals/54a6628492fa41d593f0c59af04ad973/wal-000000001 (ops 1-4908)
I20250111 21:02:01.799324 17502 log.cc:1078] T 54a6628492fa41d593f0c59af04ad973 P dc04179134174d03bd03d3db91b245f8: Deleting log segment in path: /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0/minicluster-data/ts-2/wal/wals/54a6628492fa41d593f0c59af04ad973/wal-000000002 (ops 4909-9529)
I20250111 21:02:01.804133 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: LogGCOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 0.005s user 0.000s sys 0.003s Metrics: {}
I20250111 21:02:01.804726 17614 maintenance_manager.cc:418] P dc04179134174d03bd03d3db91b245f8: Scheduling UndoDeltaBlockGCOp(54a6628492fa41d593f0c59af04ad973): 7913644 bytes on disk
I20250111 21:02:01.805950 17502 maintenance_manager.cc:642] P dc04179134174d03bd03d3db91b245f8: UndoDeltaBlockGCOp(54a6628492fa41d593f0c59af04ad973) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":110,"lbm_reads_lt_1ms":4}
I20250111 21:02:02.717296 15240 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250111 21:02:02.717495 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 09be37535c0a473a956abea447348515 and pid 17622
************************ BEGIN STACKS **************************
[New LWP 17623]
[New LWP 17624]
[New LWP 17625]
[New LWP 17627]
[New LWP 17633]
[New LWP 17634]
[New LWP 17635]
[New LWP 17638]
[New LWP 17639]
[New LWP 17640]
[New LWP 17641]
[New LWP 17643]
[New LWP 17646]
[New LWP 17647]
[New LWP 17648]
[New LWP 17650]
[New LWP 17651]
[New LWP 17652]
[New LWP 17653]
[New LWP 17654]
[New LWP 17655]
[New LWP 17656]
[New LWP 17657]
[New LWP 17658]
[New LWP 17659]
[New LWP 17660]
[New LWP 17661]
[New LWP 17662]
[New LWP 17663]
[New LWP 17664]
[New LWP 17665]
[New LWP 17666]
[New LWP 17667]
[New LWP 17668]
[New LWP 17669]
[New LWP 17670]
[New LWP 17671]
[New LWP 17672]
[New LWP 17673]
[New LWP 17674]
[New LWP 17675]
[New LWP 17676]
[New LWP 17677]
[New LWP 17678]
[New LWP 17679]
[New LWP 17680]
[New LWP 17681]
[New LWP 17682]
[New LWP 17683]
[New LWP 17684]
[New LWP 17685]
[New LWP 17686]
[New LWP 17687]
[New LWP 17688]
[New LWP 17689]
[New LWP 17690]
[New LWP 17691]
[New LWP 17692]
[New LWP 17693]
[New LWP 17694]
[New LWP 17695]
[New LWP 17696]
[New LWP 17697]
[New LWP 17698]
[New LWP 17699]
[New LWP 17700]
[New LWP 17701]
[New LWP 17702]
[New LWP 17704]
[New LWP 17705]
[New LWP 17706]
[New LWP 17707]
[New LWP 17708]
[New LWP 17709]
[New LWP 17710]
[New LWP 17711]
[New LWP 17712]
[New LWP 17713]
[New LWP 17714]
[New LWP 17715]
[New LWP 17716]
[New LWP 17717]
[New LWP 17718]
[New LWP 17719]
[New LWP 17720]
[New LWP 17721]
[New LWP 17722]
[New LWP 17723]
[New LWP 17724]
[New LWP 17725]
[New LWP 17726]
[New LWP 17727]
[New LWP 17728]
[New LWP 17729]
[New LWP 17730]
[New LWP 17731]
[New LWP 17732]
[New LWP 17733]
[New LWP 17734]
[New LWP 17735]
[New LWP 17736]
[New LWP 17737]
[New LWP 17738]
[New LWP 17739]
[New LWP 17740]
[New LWP 17741]
[New LWP 17742]
[New LWP 17743]
[New LWP 17744]
[New LWP 17745]
[New LWP 17746]
[New LWP 17747]
[New LWP 17748]
[New LWP 17749]
[New LWP 17750]
[New LWP 17751]
[New LWP 17752]
[New LWP 17753]
[New LWP 17754]
[New LWP 17755]
[New LWP 17758]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fc62e46ad50 in ?? ()
Id Target Id Frame
* 1 LWP 17622 "kudu" 0x00007fc62e46ad50 in ?? ()
2 LWP 17623 "kudu" 0x00007fc62e466fb9 in ?? ()
3 LWP 17624 "kudu" 0x00007fc62e466fb9 in ?? ()
4 LWP 17625 "kudu" 0x00007fc62e466fb9 in ?? ()
5 LWP 17627 "kernel-watcher-" 0x00007fc62e466fb9 in ?? ()
6 LWP 17633 "ntp client-1763" 0x00007fc62e46a9e2 in ?? ()
7 LWP 17634 "file cache-evic" 0x00007fc62e466fb9 in ?? ()
8 LWP 17635 "sq_acceptor" 0x00007fc6271c3cb9 in ?? ()
9 LWP 17638 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
10 LWP 17639 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
11 LWP 17640 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
12 LWP 17641 "rpc reactor-176" 0x00007fc6271d0a47 in ?? ()
13 LWP 17643 "MaintenanceMgr " 0x00007fc62e466ad3 in ?? ()
14 LWP 17646 "txn-status-mana" 0x00007fc62e466fb9 in ?? ()
15 LWP 17647 "collect_and_rem" 0x00007fc62e466fb9 in ?? ()
16 LWP 17648 "tc-session-exp-" 0x00007fc62e466fb9 in ?? ()
17 LWP 17650 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
18 LWP 17651 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
19 LWP 17652 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
20 LWP 17653 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
21 LWP 17654 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
22 LWP 17655 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
23 LWP 17656 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
24 LWP 17657 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
25 LWP 17658 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
26 LWP 17659 "rpc worker-1765" 0x00007fc62e466ad3 in ?? ()
27 LWP 17660 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
28 LWP 17661 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
29 LWP 17662 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
30 LWP 17663 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
31 LWP 17664 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
32 LWP 17665 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
33 LWP 17666 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
34 LWP 17667 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
35 LWP 17668 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
36 LWP 17669 "rpc worker-1766" 0x00007fc62e466ad3 in ?? ()
37 LWP 17670 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
38 LWP 17671 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
39 LWP 17672 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
40 LWP 17673 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
41 LWP 17674 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
42 LWP 17675 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
43 LWP 17676 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
44 LWP 17677 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
45 LWP 17678 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
46 LWP 17679 "rpc worker-1767" 0x00007fc62e466ad3 in ?? ()
47 LWP 17680 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
48 LWP 17681 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
49 LWP 17682 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
50 LWP 17683 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
51 LWP 17684 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
52 LWP 17685 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
53 LWP 17686 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
54 LWP 17687 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
55 LWP 17688 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
56 LWP 17689 "rpc worker-1768" 0x00007fc62e466ad3 in ?? ()
57 LWP 17690 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
58 LWP 17691 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
59 LWP 17692 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
60 LWP 17693 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
61 LWP 17694 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
62 LWP 17695 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
63 LWP 17696 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
64 LWP 17697 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
65 LWP 17698 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
66 LWP 17699 "rpc worker-1769" 0x00007fc62e466ad3 in ?? ()
67 LWP 17700 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
68 LWP 17701 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
69 LWP 17702 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
70 LWP 17704 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
71 LWP 17705 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
72 LWP 17706 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
73 LWP 17707 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
74 LWP 17708 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
75 LWP 17709 "rpc worker-1770" 0x00007fc62e466ad3 in ?? ()
76 LWP 17710 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
77 LWP 17711 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
78 LWP 17712 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
79 LWP 17713 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
80 LWP 17714 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
81 LWP 17715 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
82 LWP 17716 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
83 LWP 17717 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
84 LWP 17718 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
85 LWP 17719 "rpc worker-1771" 0x00007fc62e466ad3 in ?? ()
86 LWP 17720 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
87 LWP 17721 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
88 LWP 17722 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
89 LWP 17723 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
90 LWP 17724 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
91 LWP 17725 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
92 LWP 17726 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
93 LWP 17727 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
94 LWP 17728 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
95 LWP 17729 "rpc worker-1772" 0x00007fc62e466ad3 in ?? ()
96 LWP 17730 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
97 LWP 17731 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
98 LWP 17732 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
99 LWP 17733 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
100 LWP 17734 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
101 LWP 17735 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
102 LWP 17736 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
103 LWP 17737 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
104 LWP 17738 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
105 LWP 17739 "rpc worker-1773" 0x00007fc62e466ad3 in ?? ()
106 LWP 17740 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
107 LWP 17741 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
108 LWP 17742 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
109 LWP 17743 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
110 LWP 17744 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
111 LWP 17745 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
112 LWP 17746 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
113 LWP 17747 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
114 LWP 17748 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
115 LWP 17749 "rpc worker-1774" 0x00007fc62e466ad3 in ?? ()
116 LWP 17750 "rpc worker-1775" 0x00007fc62e466ad3 in ?? ()
117 LWP 17751 "diag-logger-177" 0x00007fc62e466fb9 in ?? ()
118 LWP 17752 "result-tracker-" 0x00007fc62e466fb9 in ?? ()
119 LWP 17753 "excess-log-dele" 0x00007fc62e466fb9 in ?? ()
120 LWP 17754 "acceptor-17754" 0x00007fc6271d20c7 in ?? ()
121 LWP 17755 "heartbeat-17755" 0x00007fc62e466fb9 in ?? ()
122 LWP 17758 "maintenance_sch" 0x00007fc62e466fb9 in ?? ()
Thread 122 (LWP 17758):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002a4 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fc5e353b550 in ?? ()
#6 0x0000000000000548 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17755):
#0 0x00007fc62e466fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000a8 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fc5e45625b0 in ?? ()
#6 0x0000000000000150 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17754):
#0 0x00007fc6271d20c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007fc5e4d89d50 in ?? ()
#3 0x00007fc5e4d89d80 in ?? ()
#4 0x00007fc5e4d89e80 in ?? ()
#5 0x00007fc5e4d89d70 in ?? ()
#6 0x00007fc5e4d89de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007fc62b05c620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17753):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc5e55a3f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe8a18b9d0 in ?? ()
#5 0x00007fc5e55a3ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17752):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc5e5dbc0c0 in ?? ()
#2 0x00000000000000a9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fc5e5dbc0b0 in ?? ()
#6 0x0000000000000152 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17751):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17750):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x000000000000001c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007fc5e6debe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5e6debe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 17749):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000024 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007fc5e7603e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5e7603e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 17748):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17747):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17746):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17745):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17744):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17743):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17742):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17741):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17740):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17739):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17738):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17737):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17736):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17735):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17734):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17733):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17732):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17731):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17730):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17729):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17728):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17727):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17726):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17725):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17724):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17723):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17722):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17721):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17720):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17719):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17718):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17717):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17716):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17715):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17714):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17713):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17712):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17711):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17710):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007fc5fb1bce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc5fb1bce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17709):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17708):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17707):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17706):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17705):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17704):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17702):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17701):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17700):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17699):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17698):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17697):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17696):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17695):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17694):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17693):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17692):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17691):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17690):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17689):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fc60539de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc60539de70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17688):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17687):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17686):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17685):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17684):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17683):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17682):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17681):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17680):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17679):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17678):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17677):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17676):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17675):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17674):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17673):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17672):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17671):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17670):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17669):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007fc6201b7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fc6201b7e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17668):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17667):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17666):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17665):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17664):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17663):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17662):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17661):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17660):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17659):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17658):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17657):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17656):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17655):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17654):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17653):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17652):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17651):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17650):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17648):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc618f61c80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fc618f61c70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17647):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17646):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc619f9f1c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fc619f9f1b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17643):
#0 0x00007fc62e466ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 17641):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61afb22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fc61afb22d0 in ?? ()
#4 0x00007fc61afb24e0 in ?? ()
#5 0x00007fc61afb2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fc61afb23a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb96c9c0220c000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000da000000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000ff9435ee474 in ?? ()
#17 0x00007fc61afb2380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61afb2350 in ?? ()
#20 0x3fb96c9c0220c000 in ?? ()
#21 0x000000001afb23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb96c9c0220c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17640):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61b7b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fc61b7b32d0 in ?? ()
#4 0x00007fc61b7b34e0 in ?? ()
#5 0x00007fc61b7b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fc61b7b33a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb96b18eca48000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000005e800000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff9436ee674 in ?? ()
#17 0x00007fc61b7b3380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61b7b3350 in ?? ()
#20 0x3fb96b18eca48000 in ?? ()
#21 0x000000001b7b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96b18eca48000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17639):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61bfb42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fc61bfb42d0 in ?? ()
#4 0x00007fc61bfb44e0 in ?? ()
#5 0x00007fc61bfb4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fc61bfb43a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb95de1b7034000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000051000000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff9437ee874 in ?? ()
#17 0x00007fc61bfb4380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x00007fc61bfb4350 in ?? ()
#20 0x3fb95de1b7034000 in ?? ()
#21 0x000000001bfb43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb95de1b7034000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 17638):
#0 0x00007fc6271d0a47 in ?? ()
#1 0x00007fc61dfad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fc61dfad2d0 in ?? ()
#4 0x00007fc61dfad4e0 in ?? ()
#5 0x00007fc61dfad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fc61dfad3a0 in ?? ()
#8 0x00007fc62983025d in ?? ()
#9 0x3fb988e27896a000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000f7800000 in ?? ()
#14 0x00007fc600000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000ff943beda74 in ?? ()
#17 0x00007fc61dfad380 in ?? ()
#18 0x00007fc629834ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17635):
#0 0x00007fc6271c3cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fc61f9b7788 in ?? ()
#3 0x00006020000167d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17634):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc61f1b6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fc61f1b6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17633):
#0 0x00007fc62e46a9e2 in ?? ()
#1 0x00007fc61e9b3b60 in ?? ()
#2 0x00007fc61e9b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fc61e9b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fc61e9b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fc61e9b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fc633d8ef0c in ?? ()
#12 0x00007fc633d7e7b9 in ?? ()
#13 0x00007fc633d82da6 in ?? ()
#14 0x00007fc633d877e0 in ?? ()
#15 0x00007fc633d87609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fc62b07d0be in ?? ()
#18 0x00007fc62e4606db in ?? ()
#19 0x00000ff943d2e67c in ?? ()
#20 0x00007fc61e9b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17627):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x00007fc6209b8c40 in ?? ()
#2 0x0000000000000353 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fc6209b8c30 in ?? ()
#6 0x00000000000006a6 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17625):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17624):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17623):
#0 0x00007fc62e466fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fc6221bc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17622):
#0 0x00007fc62e46ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:04.004901 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID eb9ae591b5d24eb18f58490f46e7dfaf and pid 17338
************************ BEGIN STACKS **************************
[New LWP 17340]
[New LWP 17341]
[New LWP 17342]
[New LWP 17343]
[New LWP 17350]
[New LWP 17351]
[New LWP 17352]
[New LWP 17355]
[New LWP 17356]
[New LWP 17357]
[New LWP 17358]
[New LWP 17359]
[New LWP 17360]
[New LWP 17361]
[New LWP 17362]
[New LWP 17363]
[New LWP 17364]
[New LWP 17365]
[New LWP 17366]
[New LWP 17367]
[New LWP 17368]
[New LWP 17369]
[New LWP 17370]
[New LWP 17371]
[New LWP 17372]
[New LWP 17373]
[New LWP 17374]
[New LWP 17375]
[New LWP 17376]
[New LWP 17377]
[New LWP 17378]
[New LWP 17379]
[New LWP 17380]
[New LWP 17381]
[New LWP 17382]
[New LWP 17383]
[New LWP 17384]
[New LWP 17385]
[New LWP 17386]
[New LWP 17387]
[New LWP 17388]
[New LWP 17389]
[New LWP 17390]
[New LWP 17391]
[New LWP 17392]
[New LWP 17393]
[New LWP 17394]
[New LWP 17395]
[New LWP 17396]
[New LWP 17397]
[New LWP 17398]
[New LWP 17399]
[New LWP 17400]
[New LWP 17401]
[New LWP 17402]
[New LWP 17403]
[New LWP 17404]
[New LWP 17405]
[New LWP 17406]
[New LWP 17407]
[New LWP 17408]
[New LWP 17409]
[New LWP 17410]
[New LWP 17411]
[New LWP 17412]
[New LWP 17413]
[New LWP 17414]
[New LWP 17415]
[New LWP 17416]
[New LWP 17417]
[New LWP 17418]
[New LWP 17419]
[New LWP 17420]
[New LWP 17421]
[New LWP 17423]
[New LWP 17424]
[New LWP 17425]
[New LWP 17426]
[New LWP 17427]
[New LWP 17428]
[New LWP 17429]
[New LWP 17430]
[New LWP 17432]
[New LWP 17433]
[New LWP 17434]
[New LWP 17435]
[New LWP 17436]
[New LWP 17437]
[New LWP 17438]
[New LWP 17439]
[New LWP 17440]
[New LWP 17441]
[New LWP 17442]
[New LWP 17443]
[New LWP 17444]
[New LWP 17445]
[New LWP 17446]
[New LWP 17447]
[New LWP 17448]
[New LWP 17449]
[New LWP 17450]
[New LWP 17451]
[New LWP 17452]
[New LWP 17453]
[New LWP 17454]
[New LWP 17455]
[New LWP 17456]
[New LWP 17457]
[New LWP 17458]
[New LWP 17459]
[New LWP 17460]
[New LWP 17461]
[New LWP 17462]
[New LWP 17463]
[New LWP 17464]
[New LWP 17465]
[New LWP 17466]
[New LWP 17467]
[New LWP 17468]
[New LWP 17469]
[New LWP 17472]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f921f414d50 in ?? ()
Id Target Id Frame
* 1 LWP 17338 "kudu" 0x00007f921f414d50 in ?? ()
2 LWP 17340 "kudu" 0x00007f921f410fb9 in ?? ()
3 LWP 17341 "kudu" 0x00007f921f410fb9 in ?? ()
4 LWP 17342 "kudu" 0x00007f921f410fb9 in ?? ()
5 LWP 17343 "kernel-watcher-" 0x00007f921f410fb9 in ?? ()
6 LWP 17350 "ntp client-1735" 0x00007f921f4149e2 in ?? ()
7 LWP 17351 "file cache-evic" 0x00007f921f410fb9 in ?? ()
8 LWP 17352 "sq_acceptor" 0x00007f921816dcb9 in ?? ()
9 LWP 17355 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
10 LWP 17356 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
11 LWP 17357 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
12 LWP 17358 "rpc reactor-173" 0x00007f921817aa47 in ?? ()
13 LWP 17359 "MaintenanceMgr " 0x00007f921f410ad3 in ?? ()
14 LWP 17360 "txn-status-mana" 0x00007f921f410fb9 in ?? ()
15 LWP 17361 "collect_and_rem" 0x00007f921f410fb9 in ?? ()
16 LWP 17362 "tc-session-exp-" 0x00007f921f410fb9 in ?? ()
17 LWP 17363 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
18 LWP 17364 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
19 LWP 17365 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
20 LWP 17366 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
21 LWP 17367 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
22 LWP 17368 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
23 LWP 17369 "rpc worker-1736" 0x00007f921f410ad3 in ?? ()
24 LWP 17370 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
25 LWP 17371 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
26 LWP 17372 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
27 LWP 17373 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
28 LWP 17374 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
29 LWP 17375 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
30 LWP 17376 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
31 LWP 17377 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
32 LWP 17378 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
33 LWP 17379 "rpc worker-1737" 0x00007f921f410ad3 in ?? ()
34 LWP 17380 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
35 LWP 17381 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
36 LWP 17382 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
37 LWP 17383 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
38 LWP 17384 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
39 LWP 17385 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
40 LWP 17386 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
41 LWP 17387 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
42 LWP 17388 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
43 LWP 17389 "rpc worker-1738" 0x00007f921f410ad3 in ?? ()
44 LWP 17390 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
45 LWP 17391 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
46 LWP 17392 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
47 LWP 17393 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
48 LWP 17394 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
49 LWP 17395 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
50 LWP 17396 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
51 LWP 17397 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
52 LWP 17398 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
53 LWP 17399 "rpc worker-1739" 0x00007f921f410ad3 in ?? ()
54 LWP 17400 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
55 LWP 17401 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
56 LWP 17402 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
57 LWP 17403 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
58 LWP 17404 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
59 LWP 17405 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
60 LWP 17406 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
61 LWP 17407 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
62 LWP 17408 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
63 LWP 17409 "rpc worker-1740" 0x00007f921f410ad3 in ?? ()
64 LWP 17410 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
65 LWP 17411 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
66 LWP 17412 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
67 LWP 17413 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
68 LWP 17414 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
69 LWP 17415 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
70 LWP 17416 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
71 LWP 17417 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
72 LWP 17418 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
73 LWP 17419 "rpc worker-1741" 0x00007f921f410ad3 in ?? ()
74 LWP 17420 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
75 LWP 17421 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
76 LWP 17423 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
77 LWP 17424 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
78 LWP 17425 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
79 LWP 17426 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
80 LWP 17427 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
81 LWP 17428 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
82 LWP 17429 "rpc worker-1742" 0x00007f921f410ad3 in ?? ()
83 LWP 17430 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
84 LWP 17432 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
85 LWP 17433 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
86 LWP 17434 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
87 LWP 17435 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
88 LWP 17436 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
89 LWP 17437 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
90 LWP 17438 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
91 LWP 17439 "rpc worker-1743" 0x00007f921f410ad3 in ?? ()
92 LWP 17440 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
93 LWP 17441 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
94 LWP 17442 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
95 LWP 17443 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
96 LWP 17444 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
97 LWP 17445 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
98 LWP 17446 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
99 LWP 17447 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
100 LWP 17448 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
101 LWP 17449 "rpc worker-1744" 0x00007f921f410ad3 in ?? ()
102 LWP 17450 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
103 LWP 17451 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
104 LWP 17452 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
105 LWP 17453 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
106 LWP 17454 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
107 LWP 17455 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
108 LWP 17456 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
109 LWP 17457 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
110 LWP 17458 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
111 LWP 17459 "rpc worker-1745" 0x00007f921f410ad3 in ?? ()
112 LWP 17460 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
113 LWP 17461 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
114 LWP 17462 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
115 LWP 17463 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
116 LWP 17464 "rpc worker-1746" 0x00007f921f410ad3 in ?? ()
117 LWP 17465 "diag-logger-174" 0x00007f921f410fb9 in ?? ()
118 LWP 17466 "result-tracker-" 0x00007f921f410fb9 in ?? ()
119 LWP 17467 "excess-log-dele" 0x00007f921f410fb9 in ?? ()
120 LWP 17468 "acceptor-17468" 0x00007f921817c0c7 in ?? ()
121 LWP 17469 "heartbeat-17469" 0x00007f921f410fb9 in ?? ()
122 LWP 17472 "maintenance_sch" 0x00007f921f410fb9 in ?? ()
Thread 122 (LWP 17472):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002bd in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f91d4cf4550 in ?? ()
#6 0x000000000000057a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17469):
#0 0x00007f921f410fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000af in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f91d550c5b0 in ?? ()
#6 0x000000000000015e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17468):
#0 0x00007f921817c0c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f91d5d33d50 in ?? ()
#3 0x00007f91d5d33d80 in ?? ()
#4 0x00007f91d5d33e80 in ?? ()
#5 0x00007f91d5d33d70 in ?? ()
#6 0x00007f91d5d33de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f921c006620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17467):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f91d654df00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffefea1d730 in ?? ()
#5 0x00007f91d654def0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17466):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f91d6d660c0 in ?? ()
#2 0x00000000000000af in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f91d6d660b0 in ?? ()
#6 0x000000000000015e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17465):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17464):
#0 0x00007f921f410ad3 in ?? ()
#1 0x000000000000002c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007f91d7d95e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91d7d95e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 17463):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007f91d85ade50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91d85ade70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 17462):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17461):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17460):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17459):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17458):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17457):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17456):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17455):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17454):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17453):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17452):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17451):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17450):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17449):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17448):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17447):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17446):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17445):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17444):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17443):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17442):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17441):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17440):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17439):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17438):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17437):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17436):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17435):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17434):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17433):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17432):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17430):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17429):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17428):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17427):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17426):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17425):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17424):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17423):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f91ec166e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91ec166e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17421):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17420):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17419):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17418):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17417):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17416):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17415):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17414):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17413):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17412):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17411):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17410):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17409):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17408):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17407):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17406):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17405):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17404):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17403):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17402):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f91f6347e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91f6347e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17401):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17400):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17399):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17398):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17397):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17396):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17395):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17394):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17393):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17392):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17391):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17390):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17389):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17388):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17387):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17386):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17385):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17384):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17383):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17382):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007f91ffd10e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91ffd10e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17381):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17380):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17379):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17378):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17377):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17376):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17375):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17374):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17373):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17372):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17371):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17370):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17369):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17368):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17367):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17366):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17365):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17364):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17363):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17362):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f9209f0fc80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f9209f0fc70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17361):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17360):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f920af4d1c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f920af4d1b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17359):
#0 0x00007f921f410ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 17358):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920bf9c2e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f920bf9c2d0 in ?? ()
#4 0x00007f920bf9c4e0 in ?? ()
#5 0x00007f920bf9c320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f920bf9c3a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb96ea77db42000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000ef800000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ff2c17eb874 in ?? ()
#17 0x00007f920bf9c380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920bf9c350 in ?? ()
#20 0x3fb96ea77db42000 in ?? ()
#21 0x000000000bf9c3a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb96ea77db42000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17357):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920c7b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f920c7b32d0 in ?? ()
#4 0x00007f920c7b34e0 in ?? ()
#5 0x00007f920c7b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f920c7b33a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb96cf145286000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000064000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ff2c18ee674 in ?? ()
#17 0x00007f920c7b3380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920c7b3350 in ?? ()
#20 0x3fb96cf145286000 in ?? ()
#21 0x000000000c7b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96cf145286000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17356):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920cfb42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f920cfb42d0 in ?? ()
#4 0x00007f920cfb44e0 in ?? ()
#5 0x00007f920cfb4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f920cfb43a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb952fad3356000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000005b000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2c19ee874 in ?? ()
#17 0x00007f920cfb4380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x00007f920cfb4350 in ?? ()
#20 0x3fb952fad3356000 in ?? ()
#21 0x000000000cfb43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb952fad3356000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 17355):
#0 0x00007f921817aa47 in ?? ()
#1 0x00007f920efad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f920efad2d0 in ?? ()
#4 0x00007f920efad4e0 in ?? ()
#5 0x00007f920efad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f920efad3a0 in ?? ()
#8 0x00007f921a7da25d in ?? ()
#9 0x3fb98ca5f3f9c000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000077000000 in ?? ()
#14 0x00007f9200000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff2c1deda74 in ?? ()
#17 0x00007f920efad380 in ?? ()
#18 0x00007f921a7deba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17352):
#0 0x00007f921816dcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f92109b7788 in ?? ()
#3 0x0000602000015cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17351):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f92101b6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f92101b6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17350):
#0 0x00007f921f4149e2 in ?? ()
#1 0x00007f920f9b3b60 in ?? ()
#2 0x00007f920f9b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f920f9b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f920f9b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f920f9b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f9224d38f0c in ?? ()
#12 0x00007f9224d287b9 in ?? ()
#13 0x00007f9224d2cda6 in ?? ()
#14 0x00007f9224d317e0 in ?? ()
#15 0x00007f9224d31609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f921c0270be in ?? ()
#18 0x00007f921f40a6db in ?? ()
#19 0x00000ff2c1f2e67c in ?? ()
#20 0x00007f920f9b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17343):
#0 0x00007f921f410fb9 in ?? ()
#1 0x00007f92119b8c40 in ?? ()
#2 0x000000000000036f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f92119b8c30 in ?? ()
#6 0x00000000000006de in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17342):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17341):
#0 0x00007f921f410fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17340):
#0 0x00007f921f410fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f92131bc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17338):
#0 0x00007f921f414d50 in ?? ()
#1 0x0000000000000000 in ?? ()
W20250111 21:02:05.228107 17465 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
************************* END STACKS ***************************
I20250111 21:02:05.244814 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID dc04179134174d03bd03d3db91b245f8 and pid 17479
************************ BEGIN STACKS **************************
[New LWP 17481]
[New LWP 17482]
[New LWP 17483]
[New LWP 17484]
[New LWP 17490]
[New LWP 17491]
[New LWP 17492]
[New LWP 17498]
[New LWP 17499]
[New LWP 17500]
[New LWP 17501]
[New LWP 17502]
[New LWP 17503]
[New LWP 17505]
[New LWP 17506]
[New LWP 17507]
[New LWP 17508]
[New LWP 17509]
[New LWP 17510]
[New LWP 17511]
[New LWP 17512]
[New LWP 17513]
[New LWP 17514]
[New LWP 17515]
[New LWP 17516]
[New LWP 17517]
[New LWP 17518]
[New LWP 17519]
[New LWP 17520]
[New LWP 17521]
[New LWP 17522]
[New LWP 17523]
[New LWP 17524]
[New LWP 17525]
[New LWP 17526]
[New LWP 17527]
[New LWP 17528]
[New LWP 17529]
[New LWP 17530]
[New LWP 17531]
[New LWP 17532]
[New LWP 17533]
[New LWP 17534]
[New LWP 17535]
[New LWP 17536]
[New LWP 17537]
[New LWP 17538]
[New LWP 17539]
[New LWP 17540]
[New LWP 17541]
[New LWP 17542]
[New LWP 17543]
[New LWP 17544]
[New LWP 17545]
[New LWP 17546]
[New LWP 17547]
[New LWP 17548]
[New LWP 17549]
[New LWP 17550]
[New LWP 17551]
[New LWP 17552]
[New LWP 17553]
[New LWP 17554]
[New LWP 17555]
[New LWP 17556]
[New LWP 17557]
[New LWP 17558]
[New LWP 17559]
[New LWP 17560]
[New LWP 17561]
[New LWP 17562]
[New LWP 17563]
[New LWP 17564]
[New LWP 17565]
[New LWP 17566]
[New LWP 17567]
[New LWP 17568]
[New LWP 17569]
[New LWP 17570]
[New LWP 17571]
[New LWP 17572]
[New LWP 17573]
[New LWP 17574]
[New LWP 17575]
[New LWP 17576]
[New LWP 17577]
[New LWP 17578]
[New LWP 17579]
[New LWP 17580]
[New LWP 17581]
[New LWP 17582]
[New LWP 17583]
[New LWP 17584]
[New LWP 17585]
[New LWP 17586]
[New LWP 17587]
[New LWP 17588]
[New LWP 17589]
[New LWP 17590]
[New LWP 17591]
[New LWP 17592]
[New LWP 17593]
[New LWP 17594]
[New LWP 17595]
[New LWP 17596]
[New LWP 17597]
[New LWP 17598]
[New LWP 17599]
[New LWP 17600]
[New LWP 17601]
[New LWP 17602]
[New LWP 17603]
[New LWP 17604]
[New LWP 17605]
[New LWP 17606]
[New LWP 17607]
[New LWP 17608]
[New LWP 17609]
[New LWP 17610]
[New LWP 17611]
[New LWP 17614]
[New LWP 19593]
[New LWP 19602]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f87e30dbd50 in ?? ()
Id Target Id Frame
* 1 LWP 17479 "kudu" 0x00007f87e30dbd50 in ?? ()
2 LWP 17481 "kudu" 0x00007f87e30d7fb9 in ?? ()
3 LWP 17482 "kudu" 0x00007f87e30d7fb9 in ?? ()
4 LWP 17483 "kudu" 0x00007f87e30d7fb9 in ?? ()
5 LWP 17484 "kernel-watcher-" 0x00007f87e30d7fb9 in ?? ()
6 LWP 17490 "ntp client-1749" 0x00007f87e30db9e2 in ?? ()
7 LWP 17491 "file cache-evic" 0x00007f87e30d7fb9 in ?? ()
8 LWP 17492 "sq_acceptor" 0x00007f87dbe34cb9 in ?? ()
9 LWP 17498 "rpc reactor-174" 0x00007f87dbe41a47 in ?? ()
10 LWP 17499 "rpc reactor-174" 0x0000000000926cb7 in __sanitizer::theDepot ()
11 LWP 17500 "rpc reactor-175" 0x00007f87dbe41a47 in ?? ()
12 LWP 17501 "rpc reactor-175" 0x00007f87dbe41a47 in ?? ()
13 LWP 17502 "MaintenanceMgr " 0x00007f87e30d7ad3 in ?? ()
14 LWP 17503 "txn-status-mana" 0x00007f87e30d7fb9 in ?? ()
15 LWP 17505 "collect_and_rem" 0x00007f87e30d7fb9 in ?? ()
16 LWP 17506 "tc-session-exp-" 0x00007f87e30d7fb9 in ?? ()
17 LWP 17507 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
18 LWP 17508 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
19 LWP 17509 "rpc worker-1750" 0x00007f87e30d7ad3 in ?? ()
20 LWP 17510 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
21 LWP 17511 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
22 LWP 17512 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
23 LWP 17513 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
24 LWP 17514 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
25 LWP 17515 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
26 LWP 17516 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
27 LWP 17517 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
28 LWP 17518 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
29 LWP 17519 "rpc worker-1751" 0x00007f87e30d7ad3 in ?? ()
30 LWP 17520 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
31 LWP 17521 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
32 LWP 17522 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
33 LWP 17523 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
34 LWP 17524 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
35 LWP 17525 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
36 LWP 17526 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
37 LWP 17527 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
38 LWP 17528 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
39 LWP 17529 "rpc worker-1752" 0x00007f87e30d7ad3 in ?? ()
40 LWP 17530 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
41 LWP 17531 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
42 LWP 17532 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
43 LWP 17533 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
44 LWP 17534 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
45 LWP 17535 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
46 LWP 17536 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
47 LWP 17537 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
48 LWP 17538 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
49 LWP 17539 "rpc worker-1753" 0x00007f87e30d7ad3 in ?? ()
50 LWP 17540 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
51 LWP 17541 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
52 LWP 17542 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
53 LWP 17543 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
54 LWP 17544 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
55 LWP 17545 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
56 LWP 17546 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
57 LWP 17547 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
58 LWP 17548 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
59 LWP 17549 "rpc worker-1754" 0x00007f87e30d7ad3 in ?? ()
60 LWP 17550 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
61 LWP 17551 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
62 LWP 17552 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
63 LWP 17553 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
64 LWP 17554 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
65 LWP 17555 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
66 LWP 17556 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
67 LWP 17557 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
68 LWP 17558 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
69 LWP 17559 "rpc worker-1755" 0x00007f87e30d7ad3 in ?? ()
70 LWP 17560 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
71 LWP 17561 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
72 LWP 17562 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
73 LWP 17563 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
74 LWP 17564 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
75 LWP 17565 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
76 LWP 17566 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
77 LWP 17567 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
78 LWP 17568 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
79 LWP 17569 "rpc worker-1756" 0x00007f87e30d7ad3 in ?? ()
80 LWP 17570 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
81 LWP 17571 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
82 LWP 17572 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
83 LWP 17573 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
84 LWP 17574 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
85 LWP 17575 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
86 LWP 17576 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
87 LWP 17577 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
88 LWP 17578 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
89 LWP 17579 "rpc worker-1757" 0x00007f87e30d7ad3 in ?? ()
90 LWP 17580 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
91 LWP 17581 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
92 LWP 17582 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
93 LWP 17583 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
94 LWP 17584 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
95 LWP 17585 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
96 LWP 17586 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
97 LWP 17587 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
98 LWP 17588 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
99 LWP 17589 "rpc worker-1758" 0x00007f87e30d7ad3 in ?? ()
100 LWP 17590 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
101 LWP 17591 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
102 LWP 17592 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
103 LWP 17593 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
104 LWP 17594 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
105 LWP 17595 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
106 LWP 17596 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
107 LWP 17597 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
108 LWP 17598 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
109 LWP 17599 "rpc worker-1759" 0x00007f87e30d7ad3 in ?? ()
110 LWP 17600 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
111 LWP 17601 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
112 LWP 17602 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
113 LWP 17603 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
114 LWP 17604 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
115 LWP 17605 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
116 LWP 17606 "rpc worker-1760" 0x00007f87e30d7ad3 in ?? ()
117 LWP 17607 "diag-logger-176" 0x00007f87e30d7fb9 in ?? ()
118 LWP 17608 "result-tracker-" 0x00007f87e30d7fb9 in ?? ()
119 LWP 17609 "excess-log-dele" 0x00007f87e30d7fb9 in ?? ()
120 LWP 17610 "acceptor-17610" 0x00007f87dbe430c7 in ?? ()
121 LWP 17611 "heartbeat-17611" 0x00007f87e30d7fb9 in ?? ()
122 LWP 17614 "maintenance_sch" 0x00007f87e30d7fb9 in ?? ()
123 LWP 19593 "raft [worker]-1" 0x00007f87e30d7fb9 in ?? ()
124 LWP 19602 "rpc reactor-174" 0x00007f87dbe41711 in ?? ()
Thread 124 (LWP 19602):
#0 0x00007f87dbe41711 in ?? ()
#1 0x00007f87e30d1600 in ?? ()
#2 0x00007f878b0fa700 in ?? ()
#3 0x0000000000000000 in ?? ()
Thread 123 (LWP 19593):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000045 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f878cbf4b24 in ?? ()
#5 0x00007f878cbf47b0 in ?? ()
#6 0x000000000000008b in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f878cbf47d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 17614):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x00000000000002ba in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000616000017584 in ?? ()
#5 0x00007f8798161550 in ?? ()
#6 0x0000000000000575 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f8798161570 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0xffffffff0000006e in ?? ()
#12 0x00007f8798162030 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 121 (LWP 17611):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000b2 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061300001aac4 in ?? ()
#5 0x00007f87989795b0 in ?? ()
#6 0x0000000000000165 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f87989795d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f8798979650 in ?? ()
#12 0x00000ff0f312f2cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 17610):
#0 0x00007f87dbe430c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f87991a0d50 in ?? ()
#3 0x00007f87991a0d80 in ?? ()
#4 0x00007f87991a0e80 in ?? ()
#5 0x00007f87991a0d70 in ?? ()
#6 0x00007f87991a0de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f87dfccd620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17609):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87999b9f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc6591bef0 in ?? ()
#5 0x00007f87999b9ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17608):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f879a1d30c0 in ?? ()
#2 0x00000000000000ae in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f879a1d30b0 in ?? ()
#6 0x000000000000015c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17607):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17606):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000027 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002d682c in ?? ()
#4 0x00007f879b202e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f879b202e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002d67e0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f879b202e70 in ?? ()
#11 0x00000ff0f36405d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 17605):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000019 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002d002c in ?? ()
#4 0x00007f879ba1ae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f879ba1ae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002cffe0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f879ba1ae70 in ?? ()
#11 0x00000ff0f37435d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 17604):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17603):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17602):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17601):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17600):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17599):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17598):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17597):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17596):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17595):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17594):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17593):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17592):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17591):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17590):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17589):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17588):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17587):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17586):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17585):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17584):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17583):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17582):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17581):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17580):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17579):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17578):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17577):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17576):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17575):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17574):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17573):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17572):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17571):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17570):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17569):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17568):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17567):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17566):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000330 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001c0068 in ?? ()
#4 0x00007f87af5d3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87af5d3e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 17565):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000304 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001b6878 in ?? ()
#4 0x00007f87afdebe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87afdebe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 17564):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001b007c in ?? ()
#4 0x00007f87b0603e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87b0603e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001b0030 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87b0603e70 in ?? ()
#11 0x00000ff0f60c07d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 17563):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17562):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17561):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17560):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17559):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17558):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17557):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17556):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17555):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17554):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17553):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17552):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17551):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17550):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17549):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17548):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17547):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17546):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000010 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000136828 in ?? ()
#4 0x00007f87b8f9ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87b8f9ce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 17545):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17544):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17543):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17542):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17541):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17540):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17539):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17538):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17537):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17536):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17535):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17534):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17533):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17532):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17531):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17530):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17529):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17528):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17527):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17526):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000001016 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007f87c317de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c317de70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 17525):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000dfc in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007f87c3995e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c3995e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 17524):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000bcf in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f87c41ade50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87c41ade70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87c41ade70 in ?? ()
#11 0x00000ff0f8835bd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 17523):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17522):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17521):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17520):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17519):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17518):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17517):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17516):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17515):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17514):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17513):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17512):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17511):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17510):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17509):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17508):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 17507):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 17506):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87cd384c80 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f87cd384c70 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 17505):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 17503):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87cec191c0 in ?? ()
#2 0x0000000000000011 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f87cec191b0 in ?? ()
#6 0x0000000000000022 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 17502):
#0 0x00007f87e30d7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x00007f87cf440b24 in ?? ()
#4 0x00007f87cf440830 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f87cf440850 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00006160000177c0 in ?? ()
#9 0x00007f87e30d7770 in ?? ()
#10 0x00007f87cf440850 in ?? ()
#11 0x00007f87cf440850 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 12 (LWP 17501):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87cfc682e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f87cfc682d0 in ?? ()
#4 0x00007f87cfc684e0 in ?? ()
#5 0x00007f87cfc68320 in ?? ()
#6 0x0000614000021a98 in ?? ()
#7 0x00007f87cfc683a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3f95ec6d96790000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000ca000000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff179f85074 in ?? ()
#17 0x00007f87cfc68380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x00007f87cfc68350 in ?? ()
#20 0x3f95ec6d96790000 in ?? ()
#21 0x00000000cfc683a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021a98 in ?? ()
#24 0x3f95ec6d96790000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 17500):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87d047f2e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f87d047f2d0 in ?? ()
#4 0x00007f87d047f4e0 in ?? ()
#5 0x00007f87d047f320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f87d047f3a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3fb97224309e0000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000009d000000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff17a087e74 in ?? ()
#17 0x00007f87d047f380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x00007f87d047f350 in ?? ()
#20 0x3fb97224309e0000 in ?? ()
#21 0x00000000d047f3a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb97224309e0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 17499):
#0 0x0000000000926cb7 in __sanitizer::theDepot ()
#1 0x000000000088c36d in __sanitizer::theDepot ()
#2 0x00007f87f313a2ee in ?? ()
#3 0x00000000e69ae515 in ?? ()
#4 0x00007f87d7a88000 in ?? ()
#5 0x0000000000000000 in ?? ()
Thread 9 (LWP 17498):
#0 0x00007f87dbe41a47 in ?? ()
#1 0x00007f87d2cad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f87d2cad2d0 in ?? ()
#4 0x00007f87d2cad4e0 in ?? ()
#5 0x00007f87d2cad320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f87d2cad3a0 in ?? ()
#8 0x00007f87de4a125d in ?? ()
#9 0x3f930eb6fdb18000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000068f00000 in ?? ()
#14 0x00007f8700000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ff17a58da74 in ?? ()
#17 0x00007f87d2cad380 in ?? ()
#18 0x00007f87de4a5ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 17492):
#0 0x00007f87dbe34cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f87d46b7788 in ?? ()
#3 0x0000602000015a10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 17491):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87d3eb6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f87d3eb6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 17490):
#0 0x00007f87e30db9e2 in ?? ()
#1 0x00007f87d36b3b60 in ?? ()
#2 0x00007f87d36b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f87d36b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f87d36b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f87d36b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f87e89fff0c in ?? ()
#12 0x00007f87e89ef7b9 in ?? ()
#13 0x00007f87e89f3da6 in ?? ()
#14 0x00007f87e89f87e0 in ?? ()
#15 0x00007f87e89f8609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f87dfcee0be in ?? ()
#18 0x00007f87e30d16db in ?? ()
#19 0x00000ff17a6ce67c in ?? ()
#20 0x00007f87d36b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 17484):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x00007f87d56b8c40 in ?? ()
#2 0x000000000000036c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f87d56b8c30 in ?? ()
#6 0x00000000000006d8 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 17483):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 17482):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 17481):
#0 0x00007f87e30d7fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f87d6ebc410 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 17479):
#0 0x00007f87e30dbd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:06.637135 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 3 with UUID 4e66492bc0e449a981045ff82ba305e8 and pid 16832
************************ BEGIN STACKS **************************
[New LWP 16834]
[New LWP 16835]
[New LWP 16836]
[New LWP 16837]
[New LWP 16843]
[New LWP 16844]
[New LWP 16845]
[New LWP 16848]
[New LWP 16849]
[New LWP 16850]
[New LWP 16851]
[New LWP 16852]
[New LWP 16853]
[New LWP 16854]
[New LWP 16855]
[New LWP 16856]
[New LWP 16857]
[New LWP 16858]
[New LWP 16859]
[New LWP 16860]
[New LWP 16861]
[New LWP 16862]
[New LWP 16863]
[New LWP 16864]
[New LWP 16865]
[New LWP 16866]
[New LWP 16867]
[New LWP 16869]
[New LWP 16870]
[New LWP 16871]
[New LWP 16872]
[New LWP 16873]
[New LWP 16874]
[New LWP 16876]
[New LWP 16877]
[New LWP 16878]
[New LWP 16879]
[New LWP 16880]
[New LWP 16881]
[New LWP 16882]
[New LWP 16883]
[New LWP 16884]
[New LWP 16885]
[New LWP 16886]
[New LWP 16887]
[New LWP 16888]
[New LWP 16889]
[New LWP 16890]
[New LWP 16891]
[New LWP 16892]
[New LWP 16893]
[New LWP 16894]
[New LWP 16895]
[New LWP 16896]
[New LWP 16897]
[New LWP 16898]
[New LWP 16899]
[New LWP 16900]
[New LWP 16901]
[New LWP 16902]
[New LWP 16903]
[New LWP 16904]
[New LWP 16905]
[New LWP 16906]
[New LWP 16907]
[New LWP 16908]
[New LWP 16909]
[New LWP 16910]
[New LWP 16911]
[New LWP 16912]
[New LWP 16913]
[New LWP 16914]
[New LWP 16915]
[New LWP 16916]
[New LWP 16917]
[New LWP 16918]
[New LWP 16919]
[New LWP 16921]
[New LWP 16922]
[New LWP 16923]
[New LWP 16924]
[New LWP 16925]
[New LWP 16926]
[New LWP 16928]
[New LWP 16929]
[New LWP 16930]
[New LWP 16931]
[New LWP 16932]
[New LWP 16933]
[New LWP 16934]
[New LWP 16935]
[New LWP 16937]
[New LWP 16938]
[New LWP 16939]
[New LWP 16940]
[New LWP 16941]
[New LWP 16942]
[New LWP 16943]
[New LWP 16944]
[New LWP 16945]
[New LWP 16946]
[New LWP 16947]
[New LWP 16948]
[New LWP 16949]
[New LWP 16950]
[New LWP 16951]
[New LWP 16952]
[New LWP 16953]
[New LWP 16954]
[New LWP 16955]
[New LWP 16956]
[New LWP 16957]
[New LWP 16958]
[New LWP 16959]
[New LWP 16960]
[New LWP 16961]
[New LWP 16962]
[New LWP 16963]
[New LWP 16964]
[New LWP 16965]
[New LWP 16968]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f05c5d4ad50 in ?? ()
Id Target Id Frame
* 1 LWP 16832 "kudu" 0x00007f05c5d4ad50 in ?? ()
2 LWP 16834 "kudu" 0x00007f05c5d46fb9 in ?? ()
3 LWP 16835 "kudu" 0x00007f05c5d46fb9 in ?? ()
4 LWP 16836 "kudu" 0x00007f05c5d46fb9 in ?? ()
5 LWP 16837 "kernel-watcher-" 0x00007f05c5d46fb9 in ?? ()
6 LWP 16843 "ntp client-1684" 0x00007f05c5d4a9e2 in ?? ()
7 LWP 16844 "file cache-evic" 0x00007f05c5d46fb9 in ?? ()
8 LWP 16845 "sq_acceptor" 0x00007f05beaa3cb9 in ?? ()
9 LWP 16848 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
10 LWP 16849 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
11 LWP 16850 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
12 LWP 16851 "rpc reactor-168" 0x00007f05beab0a47 in ?? ()
13 LWP 16852 "MaintenanceMgr " 0x00007f05c5d46ad3 in ?? ()
14 LWP 16853 "txn-status-mana" 0x00007f05c5d46fb9 in ?? ()
15 LWP 16854 "collect_and_rem" 0x00007f05c5d46fb9 in ?? ()
16 LWP 16855 "tc-session-exp-" 0x00007f05c5d46fb9 in ?? ()
17 LWP 16856 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
18 LWP 16857 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
19 LWP 16858 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
20 LWP 16859 "rpc worker-1685" 0x00007f05c5d46ad3 in ?? ()
21 LWP 16860 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
22 LWP 16861 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
23 LWP 16862 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
24 LWP 16863 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
25 LWP 16864 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
26 LWP 16865 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
27 LWP 16866 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
28 LWP 16867 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
29 LWP 16869 "rpc worker-1686" 0x00007f05c5d46ad3 in ?? ()
30 LWP 16870 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
31 LWP 16871 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
32 LWP 16872 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
33 LWP 16873 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
34 LWP 16874 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
35 LWP 16876 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
36 LWP 16877 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
37 LWP 16878 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
38 LWP 16879 "rpc worker-1687" 0x00007f05c5d46ad3 in ?? ()
39 LWP 16880 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
40 LWP 16881 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
41 LWP 16882 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
42 LWP 16883 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
43 LWP 16884 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
44 LWP 16885 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
45 LWP 16886 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
46 LWP 16887 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
47 LWP 16888 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
48 LWP 16889 "rpc worker-1688" 0x00007f05c5d46ad3 in ?? ()
49 LWP 16890 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
50 LWP 16891 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
51 LWP 16892 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
52 LWP 16893 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
53 LWP 16894 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
54 LWP 16895 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
55 LWP 16896 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
56 LWP 16897 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
57 LWP 16898 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
58 LWP 16899 "rpc worker-1689" 0x00007f05c5d46ad3 in ?? ()
59 LWP 16900 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
60 LWP 16901 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
61 LWP 16902 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
62 LWP 16903 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
63 LWP 16904 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
64 LWP 16905 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
65 LWP 16906 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
66 LWP 16907 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
67 LWP 16908 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
68 LWP 16909 "rpc worker-1690" 0x00007f05c5d46ad3 in ?? ()
69 LWP 16910 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
70 LWP 16911 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
71 LWP 16912 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
72 LWP 16913 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
73 LWP 16914 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
74 LWP 16915 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
75 LWP 16916 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
76 LWP 16917 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
77 LWP 16918 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
78 LWP 16919 "rpc worker-1691" 0x00007f05c5d46ad3 in ?? ()
79 LWP 16921 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
80 LWP 16922 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
81 LWP 16923 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
82 LWP 16924 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
83 LWP 16925 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
84 LWP 16926 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
85 LWP 16928 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
86 LWP 16929 "rpc worker-1692" 0x00007f05c5d46ad3 in ?? ()
87 LWP 16930 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
88 LWP 16931 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
89 LWP 16932 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
90 LWP 16933 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
91 LWP 16934 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
92 LWP 16935 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
93 LWP 16937 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
94 LWP 16938 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
95 LWP 16939 "rpc worker-1693" 0x00007f05c5d46ad3 in ?? ()
96 LWP 16940 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
97 LWP 16941 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
98 LWP 16942 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
99 LWP 16943 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
100 LWP 16944 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
101 LWP 16945 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
102 LWP 16946 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
103 LWP 16947 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
104 LWP 16948 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
105 LWP 16949 "rpc worker-1694" 0x00007f05c5d46ad3 in ?? ()
106 LWP 16950 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
107 LWP 16951 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
108 LWP 16952 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
109 LWP 16953 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
110 LWP 16954 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
111 LWP 16955 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
112 LWP 16956 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
113 LWP 16957 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
114 LWP 16958 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
115 LWP 16959 "rpc worker-1695" 0x00007f05c5d46ad3 in ?? ()
116 LWP 16960 "rpc worker-1696" 0x00007f05c5d46ad3 in ?? ()
117 LWP 16961 "diag-logger-169" 0x00007f05c5d46fb9 in ?? ()
118 LWP 16962 "result-tracker-" 0x00007f05c5d46fb9 in ?? ()
119 LWP 16963 "excess-log-dele" 0x00007f05c5d46fb9 in ?? ()
120 LWP 16964 "acceptor-16964" 0x00007f05beab20c7 in ?? ()
121 LWP 16965 "heartbeat-16965" 0x00007f05c5d46fb9 in ?? ()
122 LWP 16968 "maintenance_sch" 0x00007f05c5d46fb9 in ?? ()
Thread 122 (LWP 16968):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000031b in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f057b62a550 in ?? ()
#6 0x0000000000000636 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 16965):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f057be425b0 in ?? ()
#6 0x000000000000018c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 16964):
#0 0x00007f05beab20c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007f057c669d50 in ?? ()
#3 0x00007f057c669d80 in ?? ()
#4 0x00007f057c669e80 in ?? ()
#5 0x00007f057c669d70 in ?? ()
#6 0x00007f057c669de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f05c293c620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 16963):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f057ce83f00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007fff7624fbb0 in ?? ()
#5 0x00007f057ce83ef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 16962):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f057d69c0c0 in ?? ()
#2 0x00000000000000c7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f057d69c0b0 in ?? ()
#6 0x000000000000018e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 16961):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 16960):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000027 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f057e6cbe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f057e6cbe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f057e6cbe70 in ?? ()
#11 0x00000fe0afcd97d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 16959):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x000000000000001d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f057eee3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f057eee3e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f057eee3e70 in ?? ()
#11 0x00000fe0afddc7d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 16958):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 16957):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 16956):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 16955):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 16954):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 16953):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 16952):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 16951):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 16950):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 16949):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 16948):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 16947):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 16946):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 16945):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 16944):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 16943):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 16942):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 16941):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 16940):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 16939):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 16938):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 16937):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 16935):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 16934):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 16933):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 16932):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 16931):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 16930):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 16929):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 16928):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 16926):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 16925):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 16924):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 16923):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 16922):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 16921):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 16919):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 16918):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 16917):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f0592a9ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f0592a9ce70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f0592a9ce70 in ?? ()
#11 0x00000fe0b25539d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 16916):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 16915):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 16914):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 16913):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 16912):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 16911):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 16910):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 16909):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 16908):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 16907):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 16906):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 16905):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 16904):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 16903):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 16902):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 16901):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 16900):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 16899):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 16898):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 16897):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f059cc7de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f059cc7de70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f059cc7de70 in ?? ()
#11 0x00000fe0b398fbd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 16896):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 16895):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 16894):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 16893):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 16892):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 16891):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 16890):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 16889):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 16888):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 16887):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 16886):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 16885):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 16884):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 16883):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 16882):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 16881):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 16880):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 16879):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 16878):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 16877):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f05a6647e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f05a6647e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f05c5d46770 in ?? ()
#10 0x00007f05a6647e70 in ?? ()
#11 0x00000fe0b4cc8fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 16876):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 16874):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 16873):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 16872):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 16871):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 16870):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 16869):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 16867):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 16866):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 16865):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 16864):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 16863):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 16862):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 16861):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 16860):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 16859):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 16858):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 16857):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16856):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16855):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b084bc80 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f05b084bc70 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16854):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16853):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b18891c0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f05b18891b0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16852):
#0 0x00007f05c5d46ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16851):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b28b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f05b28b22d0 in ?? ()
#4 0x00007f05b28b24e0 in ?? ()
#5 0x00007f05b28b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f05b28b23a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb970ceac376000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000068800000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000fe13650e474 in ?? ()
#17 0x00007f05b28b2380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b28b2350 in ?? ()
#20 0x3fb970ceac376000 in ?? ()
#21 0x00000000b28b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb970ceac376000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16850):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b30b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f05b30b32d0 in ?? ()
#4 0x00007f05b30b34e0 in ?? ()
#5 0x00007f05b30b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f05b30b33a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb97cc4d9b80000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000ce000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000fe13660e674 in ?? ()
#17 0x00007f05b30b3380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b30b3350 in ?? ()
#20 0x3fb97cc4d9b80000 in ?? ()
#21 0x00000000b30b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb97cc4d9b80000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16849):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b38b42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f05b38b42d0 in ?? ()
#4 0x00007f05b38b44e0 in ?? ()
#5 0x00007f05b38b4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f05b38b43a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb97682f1f22000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000022000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000fe13670e874 in ?? ()
#17 0x00007f05b38b4380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x00007f05b38b4350 in ?? ()
#20 0x3fb97682f1f22000 in ?? ()
#21 0x00000000b38b43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb97682f1f22000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16848):
#0 0x00007f05beab0a47 in ?? ()
#1 0x00007f05b58ad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f05b58ad2d0 in ?? ()
#4 0x00007f05b58ad4e0 in ?? ()
#5 0x00007f05b58ad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f05b58ad3a0 in ?? ()
#8 0x00007f05c111025d in ?? ()
#9 0x3fb956bbb4e4e000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000f0000000 in ?? ()
#14 0x00007f0500000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000fe136b0da74 in ?? ()
#17 0x00007f05b58ad380 in ?? ()
#18 0x00007f05c1114ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16845):
#0 0x00007f05beaa3cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f05b72b7788 in ?? ()
#3 0x0000602000015a10 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16844):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b6ab6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f05b6ab6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16843):
#0 0x00007f05c5d4a9e2 in ?? ()
#1 0x00007f05b62b3b60 in ?? ()
#2 0x00007f05b62b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f05b62b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f05b62b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f05b62b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f05cb66ef0c in ?? ()
#12 0x00007f05cb65e7b9 in ?? ()
#13 0x00007f05cb662da6 in ?? ()
#14 0x00007f05cb6677e0 in ?? ()
#15 0x00007f05cb667609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f05c295d0be in ?? ()
#18 0x00007f05c5d406db in ?? ()
#19 0x00000fe136c4e67c in ?? ()
#20 0x00007f05b62b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16837):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x00007f05b82b8c40 in ?? ()
#2 0x00000000000003e8 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f05b82b8c30 in ?? ()
#6 0x00000000000007d0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16836):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16835):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16834):
#0 0x00007f05c5d46fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f05b9abc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16832):
#0 0x00007f05c5d4ad50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:07.464926 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 4 with UUID 18452380c023464488c18995b91fa513 and pid 16694
************************ BEGIN STACKS **************************
[New LWP 16695]
[New LWP 16696]
[New LWP 16697]
[New LWP 16698]
[New LWP 16704]
[New LWP 16705]
[New LWP 16706]
[New LWP 16709]
[New LWP 16710]
[New LWP 16711]
[New LWP 16712]
[New LWP 16713]
[New LWP 16714]
[New LWP 16715]
[New LWP 16716]
[New LWP 16717]
[New LWP 16718]
[New LWP 16719]
[New LWP 16720]
[New LWP 16721]
[New LWP 16722]
[New LWP 16723]
[New LWP 16724]
[New LWP 16725]
[New LWP 16726]
[New LWP 16727]
[New LWP 16728]
[New LWP 16729]
[New LWP 16730]
[New LWP 16731]
[New LWP 16732]
[New LWP 16733]
[New LWP 16734]
[New LWP 16735]
[New LWP 16736]
[New LWP 16737]
[New LWP 16738]
[New LWP 16739]
[New LWP 16740]
[New LWP 16741]
[New LWP 16742]
[New LWP 16743]
[New LWP 16744]
[New LWP 16745]
[New LWP 16746]
[New LWP 16747]
[New LWP 16748]
[New LWP 16749]
[New LWP 16750]
[New LWP 16751]
[New LWP 16752]
[New LWP 16753]
[New LWP 16754]
[New LWP 16755]
[New LWP 16756]
[New LWP 16757]
[New LWP 16758]
[New LWP 16759]
[New LWP 16760]
[New LWP 16761]
[New LWP 16762]
[New LWP 16763]
[New LWP 16764]
[New LWP 16765]
[New LWP 16766]
[New LWP 16767]
[New LWP 16768]
[New LWP 16769]
[New LWP 16770]
[New LWP 16771]
[New LWP 16772]
[New LWP 16773]
[New LWP 16774]
[New LWP 16775]
[New LWP 16776]
[New LWP 16777]
[New LWP 16778]
[New LWP 16779]
[New LWP 16780]
[New LWP 16781]
[New LWP 16782]
[New LWP 16784]
[New LWP 16785]
[New LWP 16786]
[New LWP 16787]
[New LWP 16788]
[New LWP 16789]
[New LWP 16790]
[New LWP 16791]
[New LWP 16792]
[New LWP 16793]
[New LWP 16794]
[New LWP 16795]
[New LWP 16796]
[New LWP 16797]
[New LWP 16799]
[New LWP 16801]
[New LWP 16802]
[New LWP 16803]
[New LWP 16804]
[New LWP 16805]
[New LWP 16806]
[New LWP 16807]
[New LWP 16808]
[New LWP 16809]
[New LWP 16810]
[New LWP 16811]
[New LWP 16812]
[New LWP 16813]
[New LWP 16814]
[New LWP 16815]
[New LWP 16816]
[New LWP 16817]
[New LWP 16818]
[New LWP 16819]
[New LWP 16820]
[New LWP 16821]
[New LWP 16822]
[New LWP 16823]
[New LWP 16824]
[New LWP 16825]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fe24f60ed50 in ?? ()
Id Target Id Frame
* 1 LWP 16694 "kudu" 0x00007fe24f60ed50 in ?? ()
2 LWP 16695 "kudu" 0x00007fe24f60afb9 in ?? ()
3 LWP 16696 "kudu" 0x00007fe24f60afb9 in ?? ()
4 LWP 16697 "kudu" 0x00007fe24f60afb9 in ?? ()
5 LWP 16698 "kernel-watcher-" 0x00007fe24f60afb9 in ?? ()
6 LWP 16704 "ntp client-1670" 0x00007fe24f60e9e2 in ?? ()
7 LWP 16705 "file cache-evic" 0x00007fe24f60afb9 in ?? ()
8 LWP 16706 "sq_acceptor" 0x00007fe248367cb9 in ?? ()
9 LWP 16709 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
10 LWP 16710 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
11 LWP 16711 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
12 LWP 16712 "rpc reactor-167" 0x00007fe248374a47 in ?? ()
13 LWP 16713 "MaintenanceMgr " 0x00007fe24f60aad3 in ?? ()
14 LWP 16714 "txn-status-mana" 0x00007fe24f60afb9 in ?? ()
15 LWP 16715 "collect_and_rem" 0x00007fe24f60afb9 in ?? ()
16 LWP 16716 "tc-session-exp-" 0x00007fe24f60afb9 in ?? ()
17 LWP 16717 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
18 LWP 16718 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
19 LWP 16719 "rpc worker-1671" 0x00007fe24f60aad3 in ?? ()
20 LWP 16720 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
21 LWP 16721 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
22 LWP 16722 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
23 LWP 16723 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
24 LWP 16724 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
25 LWP 16725 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
26 LWP 16726 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
27 LWP 16727 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
28 LWP 16728 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
29 LWP 16729 "rpc worker-1672" 0x00007fe24f60aad3 in ?? ()
30 LWP 16730 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
31 LWP 16731 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
32 LWP 16732 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
33 LWP 16733 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
34 LWP 16734 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
35 LWP 16735 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
36 LWP 16736 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
37 LWP 16737 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
38 LWP 16738 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
39 LWP 16739 "rpc worker-1673" 0x00007fe24f60aad3 in ?? ()
40 LWP 16740 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
41 LWP 16741 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
42 LWP 16742 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
43 LWP 16743 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
44 LWP 16744 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
45 LWP 16745 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
46 LWP 16746 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
47 LWP 16747 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
48 LWP 16748 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
49 LWP 16749 "rpc worker-1674" 0x00007fe24f60aad3 in ?? ()
50 LWP 16750 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
51 LWP 16751 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
52 LWP 16752 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
53 LWP 16753 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
54 LWP 16754 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
55 LWP 16755 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
56 LWP 16756 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
57 LWP 16757 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
58 LWP 16758 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
59 LWP 16759 "rpc worker-1675" 0x00007fe24f60aad3 in ?? ()
60 LWP 16760 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
61 LWP 16761 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
62 LWP 16762 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
63 LWP 16763 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
64 LWP 16764 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
65 LWP 16765 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
66 LWP 16766 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
67 LWP 16767 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
68 LWP 16768 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
69 LWP 16769 "rpc worker-1676" 0x00007fe24f60aad3 in ?? ()
70 LWP 16770 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
71 LWP 16771 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
72 LWP 16772 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
73 LWP 16773 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
74 LWP 16774 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
75 LWP 16775 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
76 LWP 16776 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
77 LWP 16777 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
78 LWP 16778 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
79 LWP 16779 "rpc worker-1677" 0x00007fe24f60aad3 in ?? ()
80 LWP 16780 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
81 LWP 16781 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
82 LWP 16782 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
83 LWP 16784 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
84 LWP 16785 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
85 LWP 16786 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
86 LWP 16787 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
87 LWP 16788 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
88 LWP 16789 "rpc worker-1678" 0x00007fe24f60aad3 in ?? ()
89 LWP 16790 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
90 LWP 16791 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
91 LWP 16792 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
92 LWP 16793 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
93 LWP 16794 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
94 LWP 16795 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
95 LWP 16796 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
96 LWP 16797 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
97 LWP 16799 "rpc worker-1679" 0x00007fe24f60aad3 in ?? ()
98 LWP 16801 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
99 LWP 16802 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
100 LWP 16803 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
101 LWP 16804 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
102 LWP 16805 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
103 LWP 16806 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
104 LWP 16807 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
105 LWP 16808 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
106 LWP 16809 "rpc worker-1680" 0x00007fe24f60aad3 in ?? ()
107 LWP 16810 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
108 LWP 16811 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
109 LWP 16812 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
110 LWP 16813 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
111 LWP 16814 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
112 LWP 16815 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
113 LWP 16816 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
114 LWP 16817 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
115 LWP 16818 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
116 LWP 16819 "rpc worker-1681" 0x00007fe24f60aad3 in ?? ()
117 LWP 16820 "diag-logger-168" 0x00007fe24f60afb9 in ?? ()
118 LWP 16821 "result-tracker-" 0x00007fe24f60afb9 in ?? ()
119 LWP 16822 "excess-log-dele" 0x00007fe24f60afb9 in ?? ()
120 LWP 16823 "acceptor-16823" 0x00007fe2483760c7 in ?? ()
121 LWP 16824 "heartbeat-16824" 0x00007fe24f60afb9 in ?? ()
122 LWP 16825 "maintenance_sch" 0x00007fe24f60afb9 in ?? ()
Thread 122 (LWP 16825):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000329 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fe204eee550 in ?? ()
#6 0x0000000000000652 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 16824):
#0 0x00007fe24f60afb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c9 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fe2057065b0 in ?? ()
#6 0x0000000000000192 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 16823):
#0 0x00007fe2483760c7 in ?? ()
#1 0x00006140000212a8 in ?? ()
#2 0x00007fe205f2dd50 in ?? ()
#3 0x00007fe205f2dd80 in ?? ()
#4 0x00007fe205f2de80 in ?? ()
#5 0x00007fe205f2dd70 in ?? ()
#6 0x00007fe205f2dde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007fe24c200620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 16822):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe206747f00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffe9d1fdc90 in ?? ()
#5 0x00007fe206747ef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 16821):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe206f600c0 in ?? ()
#2 0x00000000000000ca in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fe206f600b0 in ?? ()
#6 0x0000000000000194 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 16820):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 16819):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x000000000000001f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007fe207f8fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe207f8fe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe207f8fe70 in ?? ()
#11 0x00000ffc40ff1fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 16818):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000025 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007fe2087a7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe2087a7e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe2087a7e70 in ?? ()
#11 0x00000ffc410f4fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 16817):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 16816):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 16815):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 16814):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 16813):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 16812):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 16811):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 16810):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 16809):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 16808):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 16807):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 16806):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 16805):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 16804):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 16803):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 16802):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 16801):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 16799):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 16797):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 16796):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 16795):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 16794):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 16793):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 16792):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 16791):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 16790):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 16789):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 16788):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 16787):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 16786):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 16785):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 16784):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 16782):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 16781):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 16780):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 16779):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 16778):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 16777):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 16776):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007fe21bb48e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe21bb48e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe21bb48e70 in ?? ()
#11 0x00000ffc437691d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 16775):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 16774):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 16773):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 16772):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 16771):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 16770):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 16769):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 16768):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 16767):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 16766):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 16765):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 16764):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 16763):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 16762):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 16761):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 16760):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 16759):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 16758):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 16757):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 16756):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007fe225d29e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe225d29e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe225d29e70 in ?? ()
#11 0x00000ffc44ba53d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 16755):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 16754):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 16753):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 16752):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 16751):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 16750):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 16749):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 16748):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 16747):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 16746):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 16745):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 16744):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 16743):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 16742):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 16741):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 16740):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 16739):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 16738):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 16737):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 16736):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007fe22ff0ae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe22ff0ae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007fe24f60a770 in ?? ()
#10 0x00007fe22ff0ae70 in ?? ()
#11 0x00000ffc45fe15d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 16735):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 16734):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 16733):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 16732):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 16731):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 16730):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 16729):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 16728):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 16727):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 16726):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 16725):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 16724):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 16723):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 16722):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 16721):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 16720):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 16719):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 16718):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16717):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16716):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe239fc3c80 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fe239fc3c70 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16715):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16714):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe23afc51c0 in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fe23afc51b0 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16713):
#0 0x00007fe24f60aad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16712):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23bfc82e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fe23bfc82d0 in ?? ()
#4 0x00007fe23bfc84e0 in ?? ()
#5 0x00007fe23bfc8320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fe23bfc83a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb9895fb68d0000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000080800000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ffcc77f1074 in ?? ()
#17 0x00007fe23bfc8380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23bfc8350 in ?? ()
#20 0x3fb9895fb68d0000 in ?? ()
#21 0x000000003bfc83a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb9895fb68d0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16711):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23c7c92e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fe23c7c92d0 in ?? ()
#4 0x00007fe23c7c94e0 in ?? ()
#5 0x00007fe23c7c9320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fe23c7c93a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb98a38391aa000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000038800000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffcc78f1274 in ?? ()
#17 0x00007fe23c7c9380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23c7c9350 in ?? ()
#20 0x3fb98a38391aa000 in ?? ()
#21 0x000000003c7c93a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb98a38391aa000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16710):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23cfca2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fe23cfca2d0 in ?? ()
#4 0x00007fe23cfca4e0 in ?? ()
#5 0x00007fe23cfca320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fe23cfca3a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb982d359b6c000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000003e000000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ffcc79f1474 in ?? ()
#17 0x00007fe23cfca380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x00007fe23cfca350 in ?? ()
#20 0x3fb982d359b6c000 in ?? ()
#21 0x000000003cfca3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb982d359b6c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16709):
#0 0x00007fe248374a47 in ?? ()
#1 0x00007fe23ebb32e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fe23ebb32d0 in ?? ()
#4 0x00007fe23ebb34e0 in ?? ()
#5 0x00007fe23ebb3320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fe23ebb33a0 in ?? ()
#8 0x00007fe24a9d425d in ?? ()
#9 0x3fb9646078a9e000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000041000000 in ?? ()
#14 0x00007fe200000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffcc7d6e674 in ?? ()
#17 0x00007fe23ebb3380 in ?? ()
#18 0x00007fe24a9d8ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16706):
#0 0x00007fe248367cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fe240bb7788 in ?? ()
#3 0x0000602000017cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16705):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe2403b6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fe2403b6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16704):
#0 0x00007fe24f60e9e2 in ?? ()
#1 0x00007fe23f3b2b60 in ?? ()
#2 0x00007fe23f3b2c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fe23f3b2bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fe23f3b2c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fe23f3b2c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fe254f32f0c in ?? ()
#12 0x00007fe254f227b9 in ?? ()
#13 0x00007fe254f26da6 in ?? ()
#14 0x00007fe254f2b7e0 in ?? ()
#15 0x00007fe254f2b609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fe24c2210be in ?? ()
#18 0x00007fe24f6046db in ?? ()
#19 0x00000ffcc7e6e47c in ?? ()
#20 0x00007fe23f3b2400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16698):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x00007fe241bb8c40 in ?? ()
#2 0x00000000000003f7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fe241bb8c30 in ?? ()
#6 0x00000000000007ee in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16697):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16696):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16695):
#0 0x00007fe24f60afb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fe2433bc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16694):
#0 0x00007fe24f60ed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:08.285482 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 5 with UUID edf2bd29095149dd9e4592ae2870d4ee and pid 18094
************************ BEGIN STACKS **************************
[New LWP 18097]
[New LWP 18098]
[New LWP 18099]
[New LWP 18100]
[New LWP 18106]
[New LWP 18107]
[New LWP 18108]
[New LWP 18111]
[New LWP 18112]
[New LWP 18113]
[New LWP 18114]
[New LWP 18115]
[New LWP 18116]
[New LWP 18118]
[New LWP 18119]
[New LWP 18120]
[New LWP 18121]
[New LWP 18122]
[New LWP 18123]
[New LWP 18124]
[New LWP 18125]
[New LWP 18126]
[New LWP 18127]
[New LWP 18128]
[New LWP 18129]
[New LWP 18130]
[New LWP 18131]
[New LWP 18132]
[New LWP 18133]
[New LWP 18134]
[New LWP 18135]
[New LWP 18136]
[New LWP 18137]
[New LWP 18138]
[New LWP 18139]
[New LWP 18140]
[New LWP 18141]
[New LWP 18142]
[New LWP 18143]
[New LWP 18144]
[New LWP 18145]
[New LWP 18146]
[New LWP 18147]
[New LWP 18148]
[New LWP 18149]
[New LWP 18150]
[New LWP 18151]
[New LWP 18152]
[New LWP 18153]
[New LWP 18154]
[New LWP 18155]
[New LWP 18156]
[New LWP 18157]
[New LWP 18158]
[New LWP 18159]
[New LWP 18160]
[New LWP 18161]
[New LWP 18162]
[New LWP 18163]
[New LWP 18164]
[New LWP 18165]
[New LWP 18166]
[New LWP 18167]
[New LWP 18168]
[New LWP 18169]
[New LWP 18170]
[New LWP 18171]
[New LWP 18172]
[New LWP 18173]
[New LWP 18174]
[New LWP 18175]
[New LWP 18176]
[New LWP 18177]
[New LWP 18179]
[New LWP 18180]
[New LWP 18181]
[New LWP 18182]
[New LWP 18186]
[New LWP 18187]
[New LWP 18188]
[New LWP 18189]
[New LWP 18190]
[New LWP 18191]
[New LWP 18192]
[New LWP 18193]
[New LWP 18194]
[New LWP 18195]
[New LWP 18196]
[New LWP 18197]
[New LWP 18198]
[New LWP 18199]
[New LWP 18200]
[New LWP 18201]
[New LWP 18202]
[New LWP 18203]
[New LWP 18205]
[New LWP 18206]
[New LWP 18207]
[New LWP 18208]
[New LWP 18209]
[New LWP 18210]
[New LWP 18211]
[New LWP 18212]
[New LWP 18213]
[New LWP 18214]
[New LWP 18215]
[New LWP 18216]
[New LWP 18217]
[New LWP 18218]
[New LWP 18219]
[New LWP 18220]
[New LWP 18221]
[New LWP 18222]
[New LWP 18223]
[New LWP 18224]
[New LWP 18225]
[New LWP 18226]
[New LWP 18227]
[New LWP 18228]
[New LWP 18229]
[New LWP 18230]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f91bfd7dd50 in ?? ()
Id Target Id Frame
* 1 LWP 18094 "kudu" 0x00007f91bfd7dd50 in ?? ()
2 LWP 18097 "kudu" 0x00007f91bfd79fb9 in ?? ()
3 LWP 18098 "kudu" 0x00007f91bfd79fb9 in ?? ()
4 LWP 18099 "kudu" 0x00007f91bfd79fb9 in ?? ()
5 LWP 18100 "kernel-watcher-" 0x00007f91bfd79fb9 in ?? ()
6 LWP 18106 "ntp client-1810" 0x00007f91bfd7d9e2 in ?? ()
7 LWP 18107 "file cache-evic" 0x00007f91bfd79fb9 in ?? ()
8 LWP 18108 "sq_acceptor" 0x00007f91b8ad6cb9 in ?? ()
9 LWP 18111 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
10 LWP 18112 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
11 LWP 18113 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
12 LWP 18114 "rpc reactor-181" 0x00007f91b8ae3a47 in ?? ()
13 LWP 18115 "MaintenanceMgr " 0x00007f91bfd79ad3 in ?? ()
14 LWP 18116 "txn-status-mana" 0x00007f91bfd79fb9 in ?? ()
15 LWP 18118 "collect_and_rem" 0x00007f91bfd79fb9 in ?? ()
16 LWP 18119 "tc-session-exp-" 0x00007f91bfd79fb9 in ?? ()
17 LWP 18120 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
18 LWP 18121 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
19 LWP 18122 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
20 LWP 18123 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
21 LWP 18124 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
22 LWP 18125 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
23 LWP 18126 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
24 LWP 18127 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
25 LWP 18128 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
26 LWP 18129 "rpc worker-1812" 0x00007f91bfd79ad3 in ?? ()
27 LWP 18130 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
28 LWP 18131 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
29 LWP 18132 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
30 LWP 18133 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
31 LWP 18134 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
32 LWP 18135 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
33 LWP 18136 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
34 LWP 18137 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
35 LWP 18138 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
36 LWP 18139 "rpc worker-1813" 0x00007f91bfd79ad3 in ?? ()
37 LWP 18140 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
38 LWP 18141 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
39 LWP 18142 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
40 LWP 18143 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
41 LWP 18144 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
42 LWP 18145 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
43 LWP 18146 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
44 LWP 18147 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
45 LWP 18148 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
46 LWP 18149 "rpc worker-1814" 0x00007f91bfd79ad3 in ?? ()
47 LWP 18150 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
48 LWP 18151 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
49 LWP 18152 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
50 LWP 18153 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
51 LWP 18154 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
52 LWP 18155 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
53 LWP 18156 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
54 LWP 18157 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
55 LWP 18158 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
56 LWP 18159 "rpc worker-1815" 0x00007f91bfd79ad3 in ?? ()
57 LWP 18160 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
58 LWP 18161 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
59 LWP 18162 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
60 LWP 18163 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
61 LWP 18164 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
62 LWP 18165 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
63 LWP 18166 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
64 LWP 18167 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
65 LWP 18168 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
66 LWP 18169 "rpc worker-1816" 0x00007f91bfd79ad3 in ?? ()
67 LWP 18170 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
68 LWP 18171 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
69 LWP 18172 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
70 LWP 18173 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
71 LWP 18174 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
72 LWP 18175 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
73 LWP 18176 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
74 LWP 18177 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
75 LWP 18179 "rpc worker-1817" 0x00007f91bfd79ad3 in ?? ()
76 LWP 18180 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
77 LWP 18181 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
78 LWP 18182 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
79 LWP 18186 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
80 LWP 18187 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
81 LWP 18188 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
82 LWP 18189 "rpc worker-1818" 0x00007f91bfd79ad3 in ?? ()
83 LWP 18190 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
84 LWP 18191 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
85 LWP 18192 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
86 LWP 18193 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
87 LWP 18194 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
88 LWP 18195 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
89 LWP 18196 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
90 LWP 18197 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
91 LWP 18198 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
92 LWP 18199 "rpc worker-1819" 0x00007f91bfd79ad3 in ?? ()
93 LWP 18200 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
94 LWP 18201 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
95 LWP 18202 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
96 LWP 18203 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
97 LWP 18205 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
98 LWP 18206 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
99 LWP 18207 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
100 LWP 18208 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
101 LWP 18209 "rpc worker-1820" 0x00007f91bfd79ad3 in ?? ()
102 LWP 18210 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
103 LWP 18211 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
104 LWP 18212 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
105 LWP 18213 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
106 LWP 18214 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
107 LWP 18215 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
108 LWP 18216 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
109 LWP 18217 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
110 LWP 18218 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
111 LWP 18219 "rpc worker-1821" 0x00007f91bfd79ad3 in ?? ()
112 LWP 18220 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
113 LWP 18221 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
114 LWP 18222 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
115 LWP 18223 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
116 LWP 18224 "rpc worker-1822" 0x00007f91bfd79ad3 in ?? ()
117 LWP 18225 "diag-logger-182" 0x00007f91bfd79fb9 in ?? ()
118 LWP 18226 "result-tracker-" 0x00007f91bfd79fb9 in ?? ()
119 LWP 18227 "excess-log-dele" 0x00007f91bfd79fb9 in ?? ()
120 LWP 18228 "acceptor-18228" 0x00007f91b8ae50c7 in ?? ()
121 LWP 18229 "heartbeat-18229" 0x00007f91bfd79fb9 in ?? ()
122 LWP 18230 "maintenance_sch" 0x00007f91bfd79fb9 in ?? ()
Thread 122 (LWP 18230):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000228 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f91744cb550 in ?? ()
#6 0x0000000000000450 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18229):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x000000000000008a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f9174ce35b0 in ?? ()
#6 0x0000000000000114 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18228):
#0 0x00007f91b8ae50c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f917550ad50 in ?? ()
#3 0x00007f917550ad80 in ?? ()
#4 0x00007f917550ae80 in ?? ()
#5 0x00007f917550ad70 in ?? ()
#6 0x00007f917550ade0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f91bc96f620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 18227):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f9175d23f00 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffde77dcef0 in ?? ()
#5 0x00007f9175d23ef0 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18226):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f917653c0c0 in ?? ()
#2 0x000000000000008a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f917653c0b0 in ?? ()
#6 0x0000000000000114 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18225):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18224):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000020 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00032ffc8 in ?? ()
#4 0x00007f917756ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f917756ce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 18223):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00031ffd8 in ?? ()
#4 0x00007f9177d84e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9177d84e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 18222):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18221):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18220):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18219):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18218):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18217):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18216):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18215):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18214):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18213):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18212):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18211):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18210):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18209):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18208):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18207):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18206):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18205):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18203):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18202):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18201):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18200):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18199):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18198):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18197):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18196):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18195):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18194):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18193):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18192):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18191):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18190):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18189):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18188):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18187):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18186):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18182):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18181):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18180):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x00000000000011e1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001c006c in ?? ()
#4 0x00007f918c277e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918c277e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001c0020 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f918c277e70 in ?? ()
#11 0x00000ff23184efd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18179):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001440 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001b6878 in ?? ()
#4 0x00007f918ca8fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918ca8fe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 18177):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000042f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001b007c in ?? ()
#4 0x00007f918d2b0e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f918d2b0e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001b0030 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f918d2b0e70 in ?? ()
#11 0x00000ff231a561d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 18176):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18175):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18174):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18173):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18172):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18171):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18170):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18169):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18168):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18167):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18166):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18165):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18164):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18163):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18162):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18161):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18160):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18159):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000000d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00013682c in ?? ()
#4 0x00007f9196458e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9196458e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001367e0 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f9196458e70 in ?? ()
#11 0x00000ff232c8b1d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 18158):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18157):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18156):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18155):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18154):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18153):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18152):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18151):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18150):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18149):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18148):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18147):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18146):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18145):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18144):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18143):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18142):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18141):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18140):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18139):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001b90 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000affd8 in ?? ()
#4 0x00007f919fe21e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f919fe21e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 18138):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001a3e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000acfe8 in ?? ()
#4 0x00007f91a0639e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a0639e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 18137):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001594 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f91a0e51e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a0e51e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 33 (LWP 18136):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x000000000000114a in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000967f8 in ?? ()
#4 0x00007f91a1669e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a1669e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 18135):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000001495 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008fffc in ?? ()
#4 0x00007f91a1e81e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a1e81e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008ffb0 in ?? ()
#9 0x00007f91bfd79770 in ?? ()
#10 0x00007f91a1e81e70 in ?? ()
#11 0x00000ff2343d03d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 31 (LWP 18134):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000612 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008d008 in ?? ()
#4 0x00007f91a2699e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f91a2699e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 30 (LWP 18133):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18132):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18131):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18130):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18129):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18128):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18127):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18126):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18125):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18124):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18123):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18122):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18121):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18120):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18119):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91aa036c80 in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f91aa036c70 in ?? ()
#6 0x000000000000001c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18118):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18116):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91ab8af1c0 in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f91ab8af1b0 in ?? ()
#6 0x000000000000001c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18115):
#0 0x00007f91bfd79ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18114):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ac8b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f91ac8b22d0 in ?? ()
#4 0x00007f91ac8b24e0 in ?? ()
#5 0x00007f91ac8b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f91ac8b23a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb95d6275a80000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000da800000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ff2b590e474 in ?? ()
#17 0x00007f91ac8b2380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ac8b2350 in ?? ()
#20 0x3fb95d6275a80000 in ?? ()
#21 0x00000000ac8b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb95d6275a80000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18113):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ad0b32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f91ad0b32d0 in ?? ()
#4 0x00007f91ad0b34e0 in ?? ()
#5 0x00007f91ad0b3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f91ad0b33a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb96c4f2f718000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000006b000000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2b5a0e674 in ?? ()
#17 0x00007f91ad0b3380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ad0b3350 in ?? ()
#20 0x3fb96c4f2f718000 in ?? ()
#21 0x00000000ad0b33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb96c4f2f718000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18112):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91ad8b42e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f91ad8b42d0 in ?? ()
#4 0x00007f91ad8b44e0 in ?? ()
#5 0x00007f91ad8b4320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f91ad8b43a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb96195ba67c000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x00000000a2000000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318d24 in ?? ()
#16 0x00000ff2b5b0e874 in ?? ()
#17 0x00007f91ad8b4380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x00007f91ad8b4350 in ?? ()
#20 0x3fb96195ba67c000 in ?? ()
#21 0x00000000ad8b43a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb96195ba67c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18111):
#0 0x00007f91b8ae3a47 in ?? ()
#1 0x00007f91af8ad2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f91af8ad2d0 in ?? ()
#4 0x00007f91af8ad4e0 in ?? ()
#5 0x00007f91af8ad320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f91af8ad3a0 in ?? ()
#8 0x00007f91bb14325d in ?? ()
#9 0x3fb95d6594850000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x0000000029800000 in ?? ()
#14 0x00007f9100000000 in ?? ()
#15 0x41d9e0b657318dbf in ?? ()
#16 0x00000ff2b5f0da74 in ?? ()
#17 0x00007f91af8ad380 in ?? ()
#18 0x00007f91bb147ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18108):
#0 0x00007f91b8ad6cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f91b12b7788 in ?? ()
#3 0x0000602000017cd0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18107):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91b0ab6080 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f91b0ab6070 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18106):
#0 0x00007f91bfd7d9e2 in ?? ()
#1 0x00007f91b02b3b60 in ?? ()
#2 0x00007f91b02b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f91b02b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f91b02b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f91b02b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f91c56a1f0c in ?? ()
#12 0x00007f91c56917b9 in ?? ()
#13 0x00007f91c5695da6 in ?? ()
#14 0x00007f91c569a7e0 in ?? ()
#15 0x00007f91c569a609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f91bc9900be in ?? ()
#18 0x00007f91bfd736db in ?? ()
#19 0x00000ff2b604e67c in ?? ()
#20 0x00007f91b02b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18100):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x00007f91b22b8c40 in ?? ()
#2 0x00000000000002bc in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f91b22b8c30 in ?? ()
#6 0x0000000000000578 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18099):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18098):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18097):
#0 0x00007f91bfd79fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000006 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f91b3abc410 in ?? ()
#6 0x000000000000000c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18094):
#0 0x00007f91bfd7dd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:09.073047 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 6 with UUID 49a14c4ba19447f8851a9a40990a4b91 and pid 16972
************************ BEGIN STACKS **************************
[New LWP 16976]
[New LWP 16977]
[New LWP 16978]
[New LWP 16979]
[New LWP 16985]
[New LWP 16986]
[New LWP 16987]
[New LWP 16991]
[New LWP 16992]
[New LWP 16993]
[New LWP 16994]
[New LWP 16995]
[New LWP 16996]
[New LWP 16997]
[New LWP 16998]
[New LWP 16999]
[New LWP 17000]
[New LWP 17001]
[New LWP 17002]
[New LWP 17003]
[New LWP 17004]
[New LWP 17005]
[New LWP 17006]
[New LWP 17007]
[New LWP 17008]
[New LWP 17009]
[New LWP 17010]
[New LWP 17011]
[New LWP 17012]
[New LWP 17013]
[New LWP 17014]
[New LWP 17015]
[New LWP 17016]
[New LWP 17017]
[New LWP 17019]
[New LWP 17020]
[New LWP 17021]
[New LWP 17022]
[New LWP 17023]
[New LWP 17024]
[New LWP 17025]
[New LWP 17026]
[New LWP 17027]
[New LWP 17028]
[New LWP 17029]
[New LWP 17030]
[New LWP 17031]
[New LWP 17032]
[New LWP 17033]
[New LWP 17034]
[New LWP 17035]
[New LWP 17036]
[New LWP 17037]
[New LWP 17038]
[New LWP 17039]
[New LWP 17040]
[New LWP 17041]
[New LWP 17042]
[New LWP 17043]
[New LWP 17044]
[New LWP 17045]
[New LWP 17046]
[New LWP 17047]
[New LWP 17048]
[New LWP 17049]
[New LWP 17050]
[New LWP 17051]
[New LWP 17052]
[New LWP 17053]
[New LWP 17054]
[New LWP 17055]
[New LWP 17056]
[New LWP 17057]
[New LWP 17058]
[New LWP 17059]
[New LWP 17060]
[New LWP 17061]
[New LWP 17062]
[New LWP 17063]
[New LWP 17064]
[New LWP 17065]
[New LWP 17067]
[New LWP 17068]
[New LWP 17069]
[New LWP 17070]
[New LWP 17071]
[New LWP 17072]
[New LWP 17073]
[New LWP 17074]
[New LWP 17075]
[New LWP 17076]
[New LWP 17077]
[New LWP 17078]
[New LWP 17079]
[New LWP 17081]
[New LWP 17082]
[New LWP 17083]
[New LWP 17084]
[New LWP 17085]
[New LWP 17086]
[New LWP 17087]
[New LWP 17088]
[New LWP 17089]
[New LWP 17090]
[New LWP 17091]
[New LWP 17092]
[New LWP 17093]
[New LWP 17094]
[New LWP 17095]
[New LWP 17096]
[New LWP 17097]
[New LWP 17098]
[New LWP 17099]
[New LWP 17100]
[New LWP 17101]
[New LWP 17102]
[New LWP 17103]
[New LWP 17104]
[New LWP 17105]
[New LWP 17106]
[New LWP 17107]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f9dac861d50 in ?? ()
Id Target Id Frame
* 1 LWP 16972 "kudu" 0x00007f9dac861d50 in ?? ()
2 LWP 16976 "kudu" 0x00007f9dac85dfb9 in ?? ()
3 LWP 16977 "kudu" 0x00007f9dac85dfb9 in ?? ()
4 LWP 16978 "kudu" 0x00007f9dac85dfb9 in ?? ()
5 LWP 16979 "kernel-watcher-" 0x00007f9dac85dfb9 in ?? ()
6 LWP 16985 "ntp client-1698" 0x00007f9dac8619e2 in ?? ()
7 LWP 16986 "file cache-evic" 0x00007f9dac85dfb9 in ?? ()
8 LWP 16987 "sq_acceptor" 0x00007f9da55bacb9 in ?? ()
9 LWP 16991 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
10 LWP 16992 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
11 LWP 16993 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
12 LWP 16994 "rpc reactor-169" 0x00007f9da55c7a47 in ?? ()
13 LWP 16995 "MaintenanceMgr " 0x00007f9dac85dad3 in ?? ()
14 LWP 16996 "txn-status-mana" 0x00007f9dac85dfb9 in ?? ()
15 LWP 16997 "collect_and_rem" 0x00007f9dac85dfb9 in ?? ()
16 LWP 16998 "tc-session-exp-" 0x00007f9dac85dfb9 in ?? ()
17 LWP 16999 "rpc worker-1699" 0x00007f9dac85dad3 in ?? ()
18 LWP 17000 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
19 LWP 17001 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
20 LWP 17002 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
21 LWP 17003 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
22 LWP 17004 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
23 LWP 17005 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
24 LWP 17006 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
25 LWP 17007 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
26 LWP 17008 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
27 LWP 17009 "rpc worker-1700" 0x00007f9dac85dad3 in ?? ()
28 LWP 17010 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
29 LWP 17011 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
30 LWP 17012 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
31 LWP 17013 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
32 LWP 17014 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
33 LWP 17015 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
34 LWP 17016 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
35 LWP 17017 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
36 LWP 17019 "rpc worker-1701" 0x00007f9dac85dad3 in ?? ()
37 LWP 17020 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
38 LWP 17021 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
39 LWP 17022 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
40 LWP 17023 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
41 LWP 17024 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
42 LWP 17025 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
43 LWP 17026 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
44 LWP 17027 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
45 LWP 17028 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
46 LWP 17029 "rpc worker-1702" 0x00007f9dac85dad3 in ?? ()
47 LWP 17030 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
48 LWP 17031 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
49 LWP 17032 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
50 LWP 17033 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
51 LWP 17034 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
52 LWP 17035 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
53 LWP 17036 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
54 LWP 17037 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
55 LWP 17038 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
56 LWP 17039 "rpc worker-1703" 0x00007f9dac85dad3 in ?? ()
57 LWP 17040 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
58 LWP 17041 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
59 LWP 17042 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
60 LWP 17043 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
61 LWP 17044 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
62 LWP 17045 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
63 LWP 17046 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
64 LWP 17047 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
65 LWP 17048 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
66 LWP 17049 "rpc worker-1704" 0x00007f9dac85dad3 in ?? ()
67 LWP 17050 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
68 LWP 17051 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
69 LWP 17052 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
70 LWP 17053 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
71 LWP 17054 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
72 LWP 17055 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
73 LWP 17056 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
74 LWP 17057 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
75 LWP 17058 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
76 LWP 17059 "rpc worker-1705" 0x00007f9dac85dad3 in ?? ()
77 LWP 17060 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
78 LWP 17061 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
79 LWP 17062 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
80 LWP 17063 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
81 LWP 17064 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
82 LWP 17065 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
83 LWP 17067 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
84 LWP 17068 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
85 LWP 17069 "rpc worker-1706" 0x00007f9dac85dad3 in ?? ()
86 LWP 17070 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
87 LWP 17071 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
88 LWP 17072 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
89 LWP 17073 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
90 LWP 17074 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
91 LWP 17075 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
92 LWP 17076 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
93 LWP 17077 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
94 LWP 17078 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
95 LWP 17079 "rpc worker-1707" 0x00007f9dac85dad3 in ?? ()
96 LWP 17081 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
97 LWP 17082 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
98 LWP 17083 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
99 LWP 17084 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
100 LWP 17085 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
101 LWP 17086 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
102 LWP 17087 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
103 LWP 17088 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
104 LWP 17089 "rpc worker-1708" 0x00007f9dac85dad3 in ?? ()
105 LWP 17090 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
106 LWP 17091 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
107 LWP 17092 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
108 LWP 17093 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
109 LWP 17094 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
110 LWP 17095 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
111 LWP 17096 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
112 LWP 17097 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
113 LWP 17098 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
114 LWP 17099 "rpc worker-1709" 0x00007f9dac85dad3 in ?? ()
115 LWP 17100 "rpc worker-1710" 0x00007f9dac85dad3 in ?? ()
116 LWP 17101 "rpc worker-1710" 0x00007f9dac85dad3 in ?? ()
117 LWP 17102 "diag-logger-171" 0x00007f9dac85dfb9 in ?? ()
118 LWP 17103 "result-tracker-" 0x00007f9dac85dfb9 in ?? ()
119 LWP 17104 "excess-log-dele" 0x00007f9dac85dfb9 in ?? ()
120 LWP 17105 "acceptor-17105" 0x00007f9da55c90c7 in ?? ()
121 LWP 17106 "heartbeat-17106" 0x00007f9dac85dfb9 in ?? ()
122 LWP 17107 "maintenance_sch" 0x00007f9dac85dfb9 in ?? ()
Thread 122 (LWP 17107):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000318 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f9d62141550 in ?? ()
#6 0x0000000000000630 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 17106):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x00000000000000c5 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f9d629595b0 in ?? ()
#6 0x000000000000018a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 17105):
#0 0x00007f9da55c90c7 in ?? ()
#1 0x00006140000214a8 in ?? ()
#2 0x00007f9d63180d50 in ?? ()
#3 0x00007f9d63180d80 in ?? ()
#4 0x00007f9d63180e80 in ?? ()
#5 0x00007f9d63180d70 in ?? ()
#6 0x00007f9d63180de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x00007f9da9453620 in ?? ()
#10 0x0000000000000000 in ?? ()
Thread 119 (LWP 17104):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d6399af00 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffd8b8e6b70 in ?? ()
#5 0x00007f9d6399aef0 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 17103):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d641b30c0 in ?? ()
#2 0x00000000000000c6 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f9d641b30b0 in ?? ()
#6 0x000000000000018c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 17102):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 17101):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000023 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f9d651e2e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d651e2e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d651e2e70 in ?? ()
#11 0x00000ff3aca3c5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 17100):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000021 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f9d659fae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d659fae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d659fae70 in ?? ()
#11 0x00000ff3acb3f5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 17099):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 17098):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 17097):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 17096):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 17095):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 17094):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 17093):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 17092):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 17091):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 17090):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 17089):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 17088):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 17087):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 17086):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 17085):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 17084):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 17083):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 17082):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 17081):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 17079):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 17078):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 17077):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 17076):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 17075):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 17074):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 17073):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 17072):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 17071):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 17070):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 17069):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 17068):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 17067):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 17065):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 17064):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 17063):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 17062):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 17061):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 17060):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 17059):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f9d795b3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d795b3e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d795b3e70 in ?? ()
#11 0x00000ff3af2b67d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 17058):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 17057):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 17056):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 17055):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 17054):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 17053):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 17052):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 17051):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 17050):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 17049):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 17048):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 17047):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 17046):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 17045):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 17044):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 17043):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 17042):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 17041):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 17040):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 17039):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f9d83794e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d83794e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d83794e70 in ?? ()
#11 0x00000ff3b06f29d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 17038):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 17037):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 17036):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 17035):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 17034):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 17033):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 17032):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 17031):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 17030):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 17029):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 17028):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 17027):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 17026):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 17025):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 17024):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 17023):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 17022):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 17021):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 17020):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 17019):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f9d8d975e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f9d8d975e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f9dac85d770 in ?? ()
#10 0x00007f9d8d975e70 in ?? ()
#11 0x00000ff3b1b2ebd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 17017):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 17016):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 17015):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 17014):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 17013):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 17012):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 17011):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 17010):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 17009):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 17008):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 17007):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 17006):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 17005):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 17004):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 17003):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 17002):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 17001):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 17000):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 16999):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 16998):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d971c3c80 in ?? ()
#2 0x0000000000000013 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007f9d971c3c70 in ?? ()
#6 0x0000000000000026 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 16997):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 16996):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d981c51c0 in ?? ()
#2 0x0000000000000013 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f9d981c51b0 in ?? ()
#6 0x0000000000000026 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 16995):
#0 0x00007f9dac85dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 16994):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d991c82e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f9d991c82d0 in ?? ()
#4 0x00007f9d991c84e0 in ?? ()
#5 0x00007f9d991c8320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007f9d991c83a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb9589f42710000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x000000008e000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d21 in ?? ()
#16 0x00000ff433231074 in ?? ()
#17 0x00007f9d991c8380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d991c8350 in ?? ()
#20 0x3fb9589f42710000 in ?? ()
#21 0x00000000991c83a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb9589f42710000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 16993):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d999c92e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f9d999c92d0 in ?? ()
#4 0x00007f9d999c94e0 in ?? ()
#5 0x00007f9d999c9320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f9d999c93a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb952282feac000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000024000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d22 in ?? ()
#16 0x00000ff433331274 in ?? ()
#17 0x00007f9d999c9380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d999c9350 in ?? ()
#20 0x3fb952282feac000 in ?? ()
#21 0x00000000999c93a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb952282feac000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 16992):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d9a1ca2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f9d9a1ca2d0 in ?? ()
#4 0x00007f9d9a1ca4e0 in ?? ()
#5 0x00007f9d9a1ca320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f9d9a1ca3a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb96d1093970000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x000000001a000000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ff433431474 in ?? ()
#17 0x00007f9d9a1ca380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x00007f9d9a1ca350 in ?? ()
#20 0x3fb96d1093970000 in ?? ()
#21 0x000000009a1ca3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb96d1093970000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 16991):
#0 0x00007f9da55c7a47 in ?? ()
#1 0x00007f9d9bdb32e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f9d9bdb32d0 in ?? ()
#4 0x00007f9d9bdb34e0 in ?? ()
#5 0x00007f9d9bdb3320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f9d9bdb33a0 in ?? ()
#8 0x00007f9da7c2725d in ?? ()
#9 0x3fb96a165200a000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x000000002d800000 in ?? ()
#14 0x00007f9d00000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ff4337ae674 in ?? ()
#17 0x00007f9d9bdb3380 in ?? ()
#18 0x00007f9da7c2bba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 16987):
#0 0x00007f9da55bacb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f9d9ddb7788 in ?? ()
#3 0x0000602000016790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 16986):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d9d5b6080 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f9d9d5b6070 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 16985):
#0 0x00007f9dac8619e2 in ?? ()
#1 0x00007f9d9c5b2b60 in ?? ()
#2 0x00007f9d9c5b2c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f9d9c5b2bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f9d9c5b2c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f9d9c5b2c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f9db2185f0c in ?? ()
#12 0x00007f9db21757b9 in ?? ()
#13 0x00007f9db2179da6 in ?? ()
#14 0x00007f9db217e7e0 in ?? ()
#15 0x00007f9db217e609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f9da94740be in ?? ()
#18 0x00007f9dac8576db in ?? ()
#19 0x00000ff4338ae47c in ?? ()
#20 0x00007f9d9c5b2400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 16979):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x00007f9d9edb8c40 in ?? ()
#2 0x00000000000003e7 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f9d9edb8c30 in ?? ()
#6 0x00000000000007ce in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 16978):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 16977):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 16976):
#0 0x00007f9dac85dfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f9da05bc410 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 16972):
#0 0x00007f9dac861d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:09.879071 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 7 with UUID cc4301cbd2b84d22ab621b2c399022dc and pid 18787
************************ BEGIN STACKS **************************
[New LWP 18789]
[New LWP 18790]
[New LWP 18791]
[New LWP 18792]
[New LWP 18798]
[New LWP 18799]
[New LWP 18800]
[New LWP 18804]
[New LWP 18805]
[New LWP 18806]
[New LWP 18807]
[New LWP 18808]
[New LWP 18809]
[New LWP 18811]
[New LWP 18812]
[New LWP 18813]
[New LWP 18814]
[New LWP 18815]
[New LWP 18816]
[New LWP 18817]
[New LWP 18818]
[New LWP 18819]
[New LWP 18820]
[New LWP 18821]
[New LWP 18822]
[New LWP 18823]
[New LWP 18824]
[New LWP 18825]
[New LWP 18826]
[New LWP 18827]
[New LWP 18828]
[New LWP 18829]
[New LWP 18830]
[New LWP 18831]
[New LWP 18832]
[New LWP 18833]
[New LWP 18834]
[New LWP 18835]
[New LWP 18836]
[New LWP 18837]
[New LWP 18838]
[New LWP 18839]
[New LWP 18840]
[New LWP 18841]
[New LWP 18842]
[New LWP 18843]
[New LWP 18844]
[New LWP 18845]
[New LWP 18846]
[New LWP 18847]
[New LWP 18848]
[New LWP 18849]
[New LWP 18850]
[New LWP 18851]
[New LWP 18852]
[New LWP 18853]
[New LWP 18854]
[New LWP 18855]
[New LWP 18856]
[New LWP 18857]
[New LWP 18858]
[New LWP 18859]
[New LWP 18860]
[New LWP 18861]
[New LWP 18862]
[New LWP 18863]
[New LWP 18864]
[New LWP 18865]
[New LWP 18866]
[New LWP 18867]
[New LWP 18869]
[New LWP 18870]
[New LWP 18871]
[New LWP 18872]
[New LWP 18874]
[New LWP 18875]
[New LWP 18876]
[New LWP 18877]
[New LWP 18878]
[New LWP 18879]
[New LWP 18880]
[New LWP 18881]
[New LWP 18882]
[New LWP 18883]
[New LWP 18884]
[New LWP 18885]
[New LWP 18886]
[New LWP 18887]
[New LWP 18888]
[New LWP 18889]
[New LWP 18890]
[New LWP 18891]
[New LWP 18892]
[New LWP 18893]
[New LWP 18894]
[New LWP 18895]
[New LWP 18896]
[New LWP 18897]
[New LWP 18898]
[New LWP 18899]
[New LWP 18900]
[New LWP 18901]
[New LWP 18902]
[New LWP 18903]
[New LWP 18904]
[New LWP 18905]
[New LWP 18906]
[New LWP 18907]
[New LWP 18908]
[New LWP 18909]
[New LWP 18910]
[New LWP 18911]
[New LWP 18912]
[New LWP 18913]
[New LWP 18914]
[New LWP 18915]
[New LWP 18916]
[New LWP 18917]
[New LWP 18918]
[New LWP 18920]
[New LWP 18923]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007f1bd4396d50 in ?? ()
Id Target Id Frame
* 1 LWP 18787 "kudu" 0x00007f1bd4396d50 in ?? ()
2 LWP 18789 "kudu" 0x00007f1bd4392fb9 in ?? ()
3 LWP 18790 "kudu" 0x00007f1bd4392fb9 in ?? ()
4 LWP 18791 "kudu" 0x00007f1bd4392fb9 in ?? ()
5 LWP 18792 "kernel-watcher-" 0x00007f1bd4392fb9 in ?? ()
6 LWP 18798 "ntp client-1879" 0x00007f1bd43969e2 in ?? ()
7 LWP 18799 "file cache-evic" 0x00007f1bd4392fb9 in ?? ()
8 LWP 18800 "sq_acceptor" 0x00007f1bcd0efcb9 in ?? ()
9 LWP 18804 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
10 LWP 18805 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
11 LWP 18806 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
12 LWP 18807 "rpc reactor-188" 0x00007f1bcd0fca47 in ?? ()
13 LWP 18808 "MaintenanceMgr " 0x00007f1bd4392ad3 in ?? ()
14 LWP 18809 "txn-status-mana" 0x00007f1bd4392fb9 in ?? ()
15 LWP 18811 "collect_and_rem" 0x00007f1bd4392fb9 in ?? ()
16 LWP 18812 "tc-session-exp-" 0x00007f1bd4392fb9 in ?? ()
17 LWP 18813 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
18 LWP 18814 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
19 LWP 18815 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
20 LWP 18816 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
21 LWP 18817 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
22 LWP 18818 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
23 LWP 18819 "rpc worker-1881" 0x00007f1bd4392ad3 in ?? ()
24 LWP 18820 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
25 LWP 18821 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
26 LWP 18822 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
27 LWP 18823 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
28 LWP 18824 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
29 LWP 18825 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
30 LWP 18826 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
31 LWP 18827 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
32 LWP 18828 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
33 LWP 18829 "rpc worker-1882" 0x00007f1bd4392ad3 in ?? ()
34 LWP 18830 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
35 LWP 18831 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
36 LWP 18832 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
37 LWP 18833 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
38 LWP 18834 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
39 LWP 18835 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
40 LWP 18836 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
41 LWP 18837 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
42 LWP 18838 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
43 LWP 18839 "rpc worker-1883" 0x00007f1bd4392ad3 in ?? ()
44 LWP 18840 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
45 LWP 18841 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
46 LWP 18842 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
47 LWP 18843 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
48 LWP 18844 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
49 LWP 18845 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
50 LWP 18846 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
51 LWP 18847 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
52 LWP 18848 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
53 LWP 18849 "rpc worker-1884" 0x00007f1bd4392ad3 in ?? ()
54 LWP 18850 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
55 LWP 18851 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
56 LWP 18852 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
57 LWP 18853 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
58 LWP 18854 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
59 LWP 18855 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
60 LWP 18856 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
61 LWP 18857 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
62 LWP 18858 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
63 LWP 18859 "rpc worker-1885" 0x00007f1bd4392ad3 in ?? ()
64 LWP 18860 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
65 LWP 18861 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
66 LWP 18862 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
67 LWP 18863 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
68 LWP 18864 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
69 LWP 18865 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
70 LWP 18866 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
71 LWP 18867 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
72 LWP 18869 "rpc worker-1886" 0x00007f1bd4392ad3 in ?? ()
73 LWP 18870 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
74 LWP 18871 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
75 LWP 18872 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
76 LWP 18874 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
77 LWP 18875 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
78 LWP 18876 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
79 LWP 18877 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
80 LWP 18878 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
81 LWP 18879 "rpc worker-1887" 0x00007f1bd4392ad3 in ?? ()
82 LWP 18880 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
83 LWP 18881 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
84 LWP 18882 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
85 LWP 18883 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
86 LWP 18884 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
87 LWP 18885 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
88 LWP 18886 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
89 LWP 18887 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
90 LWP 18888 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
91 LWP 18889 "rpc worker-1888" 0x00007f1bd4392ad3 in ?? ()
92 LWP 18890 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
93 LWP 18891 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
94 LWP 18892 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
95 LWP 18893 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
96 LWP 18894 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
97 LWP 18895 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
98 LWP 18896 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
99 LWP 18897 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
100 LWP 18898 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
101 LWP 18899 "rpc worker-1889" 0x00007f1bd4392ad3 in ?? ()
102 LWP 18900 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
103 LWP 18901 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
104 LWP 18902 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
105 LWP 18903 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
106 LWP 18904 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
107 LWP 18905 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
108 LWP 18906 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
109 LWP 18907 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
110 LWP 18908 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
111 LWP 18909 "rpc worker-1890" 0x00007f1bd4392ad3 in ?? ()
112 LWP 18910 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
113 LWP 18911 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
114 LWP 18912 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
115 LWP 18913 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
116 LWP 18914 "rpc worker-1891" 0x00007f1bd4392ad3 in ?? ()
117 LWP 18915 "diag-logger-189" 0x00007f1bd4392fb9 in ?? ()
118 LWP 18916 "result-tracker-" 0x00007f1bd4392fb9 in ?? ()
119 LWP 18917 "excess-log-dele" 0x00007f1bd4392fb9 in ?? ()
120 LWP 18918 "acceptor-18918" 0x00007f1bcd0fe0c7 in ?? ()
121 LWP 18920 "heartbeat-18920" 0x00007f1bd4392fb9 in ?? ()
122 LWP 18923 "maintenance_sch" 0x00007f1bd4392fb9 in ?? ()
Thread 122 (LWP 18923):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000126 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f1b88b0d550 in ?? ()
#6 0x000000000000024c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 18920):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x000000000000004a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007f1b894275b0 in ?? ()
#6 0x0000000000000094 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 18918):
#0 0x00007f1bcd0fe0c7 in ?? ()
#1 0x00006140000410a8 in ?? ()
#2 0x00007f1b8a44dd50 in ?? ()
#3 0x00007f1b8a44dd80 in ?? ()
#4 0x00007f1b8a44de80 in ?? ()
#5 0x00007f1b8a44dd70 in ?? ()
#6 0x00007f1b8a44dde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 18917):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1b8ac66f00 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffca3bd0050 in ?? ()
#5 0x00007f1b8ac66ef0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 18916):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1b8b48f0c0 in ?? ()
#2 0x0000000000000049 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007f1b8b48f0b0 in ?? ()
#6 0x0000000000000092 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 18915):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 18914):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000017 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002e001c in ?? ()
#4 0x00007f1b8c4bfe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1b8c4bfe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002dffd0 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1b8c4bfe70 in ?? ()
#11 0x00000fe371897fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 18913):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000009 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002dd02c in ?? ()
#4 0x00007f1b8ccd7e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1b8ccd7e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002dcfe0 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1b8ccd7e70 in ?? ()
#11 0x00000fe37199afd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 18912):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 18911):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 18910):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 18909):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 18908):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 18907):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 18906):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 18905):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 18904):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 18903):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 18902):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 18901):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 18900):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 18899):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 18898):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 18897):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 18896):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 18895):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 18894):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 18893):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 18892):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 18891):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 18890):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 18889):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 18888):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 18887):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 18886):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 18885):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 18884):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 18883):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 18882):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 18881):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 18880):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 18879):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 18878):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 18877):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 18876):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 18875):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 18874):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000011 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001d005c in ?? ()
#4 0x00007f1ba0880e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba0880e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001d0010 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1ba0880e70 in ?? ()
#11 0x00000fe3741101d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 18872):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000285 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001cd06c in ?? ()
#4 0x00007f1ba1098e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba1098e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001cd020 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1ba1098e70 in ?? ()
#11 0x00000fe3742131d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 18871):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 18870):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 18869):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 18867):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 18866):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 18865):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 18864):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 18863):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 18862):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 18861):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 18860):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 18859):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 18858):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 18857):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 18856):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 18855):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 18854):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 18853):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 18852):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000146818 in ?? ()
#4 0x00007f1ba9a31e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1ba9a31e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 18851):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18850):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18849):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18848):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18847):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18846):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18845):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18844):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18843):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18842):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18841):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18840):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18839):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18838):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18837):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18836):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18835):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18834):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18833):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18832):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x000000000000000b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000bffcc in ?? ()
#4 0x00007f1bb3c12e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1bb3c12e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000bff80 in ?? ()
#9 0x00007f1bd4392770 in ?? ()
#10 0x00007f1bb3c12e70 in ?? ()
#11 0x00000fe3767825d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 18831):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18830):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18829):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18828):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18827):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18826):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18825):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18824):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18823):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18822):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18821):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18820):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18819):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18818):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18817):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18816):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18815):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18814):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18813):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18812):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bbde17c80 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001ba60 in ?? ()
#5 0x00007f1bbde17c70 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18811):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18809):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bbf6ae1c0 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007f1bbf6ae1b0 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18808):
#0 0x00007f1bd4392ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18807):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc06b12e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f1bc06b12d0 in ?? ()
#4 0x00007f1bc06b14e0 in ?? ()
#5 0x00007f1bc06b1320 in ?? ()
#6 0x0000614000041698 in ?? ()
#7 0x00007f1bc06b13a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb2178b43e2e000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x000000006b000000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000fe3f80ce274 in ?? ()
#17 0x00007f1bc06b1380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc06b1350 in ?? ()
#20 0x3fb2178b43e2e000 in ?? ()
#21 0x00000000c06b13a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000041698 in ?? ()
#24 0x3fb2178b43e2e000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18806):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc0eb22e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f1bc0eb22d0 in ?? ()
#4 0x00007f1bc0eb24e0 in ?? ()
#5 0x00007f1bc0eb2320 in ?? ()
#6 0x0000614000041498 in ?? ()
#7 0x00007f1bc0eb23a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb986d252eca000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000035000000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d20 in ?? ()
#16 0x00000fe3f81ce474 in ?? ()
#17 0x00007f1bc0eb2380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc0eb2350 in ?? ()
#20 0x3fb986d252eca000 in ?? ()
#21 0x00000000c0eb23a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000041498 in ?? ()
#24 0x3fb986d252eca000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18805):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc16b32e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f1bc16b32d0 in ?? ()
#4 0x00007f1bc16b34e0 in ?? ()
#5 0x00007f1bc16b3320 in ?? ()
#6 0x0000614000041298 in ?? ()
#7 0x00007f1bc16b33a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb4a3fc49150000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000008e00000 in ?? ()
#14 0x00007f1b00000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x00000fe3f82ce674 in ?? ()
#17 0x00007f1bc16b3380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x00007f1bc16b3350 in ?? ()
#20 0x3fb4a3fc49150000 in ?? ()
#21 0x00000000c16b33a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000041298 in ?? ()
#24 0x3fb4a3fc49150000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18804):
#0 0x00007f1bcd0fca47 in ?? ()
#1 0x00007f1bc36ac2e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007f1bc36ac2d0 in ?? ()
#4 0x00007f1bc36ac4e0 in ?? ()
#5 0x00007f1bc36ac320 in ?? ()
#6 0x0000614000041098 in ?? ()
#7 0x00007f1bc36ac3a0 in ?? ()
#8 0x00007f1bcf75c25d in ?? ()
#9 0x3fb98ae576cf8000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000cb000000 in ?? ()
#14 0x0000000300000000 in ?? ()
#15 0x41d9e0b657318d27 in ?? ()
#16 0x3eb0000000000000 in ?? ()
#17 0x00007f1bc36ac380 in ?? ()
#18 0x00007f1bcf760ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18800):
#0 0x00007f1bcd0efcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f1bc58b7788 in ?? ()
#3 0x00006020000179d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18799):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bc50b6080 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007f1bc50b6070 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18798):
#0 0x00007f1bd43969e2 in ?? ()
#1 0x00007f1bc48b3b60 in ?? ()
#2 0x00007f1bc48b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f1bc48b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f1bc48b3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007f1bc48b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f1bd9cbaf0c in ?? ()
#12 0x00007f1bd9caa7b9 in ?? ()
#13 0x00007f1bd9caeda6 in ?? ()
#14 0x00007f1bd9cb37e0 in ?? ()
#15 0x00007f1bd9cb3609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f1bd0fa90be in ?? ()
#18 0x00007f1bd438c6db in ?? ()
#19 0x00000fe3f890e67c in ?? ()
#20 0x00007f1bc48b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18792):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x00007f1bc68b8c40 in ?? ()
#2 0x0000000000000177 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f1bc68b8c30 in ?? ()
#6 0x00000000000002ee in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18791):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18790):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18789):
#0 0x00007f1bd4392fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f1bc80bc410 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18787):
#0 0x00007f1bd4396d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:10.675465 15240 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 8 with UUID 4c1123e925164e978604ac78771cb3ad and pid 18929
************************ BEGIN STACKS **************************
[New LWP 18933]
[New LWP 18934]
[New LWP 18935]
[New LWP 18936]
[New LWP 18942]
[New LWP 18943]
[New LWP 18944]
[New LWP 18947]
[New LWP 18948]
[New LWP 18949]
[New LWP 18950]
[New LWP 18951]
[New LWP 18952]
[New LWP 18953]
[New LWP 18954]
[New LWP 18955]
[New LWP 18956]
[New LWP 18957]
[New LWP 18958]
[New LWP 18959]
[New LWP 18960]
[New LWP 18961]
[New LWP 18962]
[New LWP 18963]
[New LWP 18964]
[New LWP 18965]
[New LWP 18966]
[New LWP 18967]
[New LWP 18968]
[New LWP 18974]
[New LWP 18975]
[New LWP 18976]
[New LWP 18977]
[New LWP 18978]
[New LWP 18979]
[New LWP 18980]
[New LWP 18981]
[New LWP 18982]
[New LWP 18983]
[New LWP 18984]
[New LWP 18985]
[New LWP 18987]
[New LWP 18988]
[New LWP 18989]
[New LWP 18990]
[New LWP 18991]
[New LWP 18992]
[New LWP 18993]
[New LWP 18994]
[New LWP 18995]
[New LWP 18996]
[New LWP 18997]
[New LWP 18998]
[New LWP 18999]
[New LWP 19000]
[New LWP 19001]
[New LWP 19002]
[New LWP 19003]
[New LWP 19004]
[New LWP 19005]
[New LWP 19006]
[New LWP 19007]
[New LWP 19008]
[New LWP 19009]
[New LWP 19010]
[New LWP 19011]
[New LWP 19012]
[New LWP 19013]
[New LWP 19014]
[New LWP 19015]
[New LWP 19016]
[New LWP 19017]
[New LWP 19018]
[New LWP 19019]
[New LWP 19020]
[New LWP 19021]
[New LWP 19022]
[New LWP 19023]
[New LWP 19024]
[New LWP 19025]
[New LWP 19026]
[New LWP 19027]
[New LWP 19028]
[New LWP 19029]
[New LWP 19030]
[New LWP 19031]
[New LWP 19032]
[New LWP 19033]
[New LWP 19034]
[New LWP 19035]
[New LWP 19036]
[New LWP 19037]
[New LWP 19038]
[New LWP 19039]
[New LWP 19040]
[New LWP 19041]
[New LWP 19042]
[New LWP 19043]
[New LWP 19044]
[New LWP 19045]
[New LWP 19046]
[New LWP 19047]
[New LWP 19048]
[New LWP 19049]
[New LWP 19050]
[New LWP 19051]
[New LWP 19052]
[New LWP 19053]
[New LWP 19054]
[New LWP 19055]
[New LWP 19056]
[New LWP 19057]
[New LWP 19058]
[New LWP 19059]
[New LWP 19060]
[New LWP 19061]
[New LWP 19062]
[New LWP 19063]
[New LWP 19064]
[New LWP 19065]
[New LWP 19068]
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7469
Cannot access memory at address 0x736e753c726f7461
0x00007fe5e8a63d50 in ?? ()
Id Target Id Frame
* 1 LWP 18929 "kudu" 0x00007fe5e8a63d50 in ?? ()
2 LWP 18933 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
3 LWP 18934 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
4 LWP 18935 "kudu" 0x00007fe5e8a5ffb9 in ?? ()
5 LWP 18936 "kernel-watcher-" 0x00007fe5e8a5ffb9 in ?? ()
6 LWP 18942 "ntp client-1894" 0x00007fe5e8a639e2 in ?? ()
7 LWP 18943 "file cache-evic" 0x00007fe5e8a5ffb9 in ?? ()
8 LWP 18944 "sq_acceptor" 0x00007fe5e17bccb9 in ?? ()
9 LWP 18947 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
10 LWP 18948 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
11 LWP 18949 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
12 LWP 18950 "rpc reactor-189" 0x00007fe5e17c9a47 in ?? ()
13 LWP 18951 "MaintenanceMgr " 0x00007fe5e8a5fad3 in ?? ()
14 LWP 18952 "txn-status-mana" 0x00007fe5e8a5ffb9 in ?? ()
15 LWP 18953 "collect_and_rem" 0x00007fe5e8a5ffb9 in ?? ()
16 LWP 18954 "tc-session-exp-" 0x00007fe5e8a5ffb9 in ?? ()
17 LWP 18955 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
18 LWP 18956 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
19 LWP 18957 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
20 LWP 18958 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
21 LWP 18959 "rpc worker-1895" 0x00007fe5e8a5fad3 in ?? ()
22 LWP 18960 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
23 LWP 18961 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
24 LWP 18962 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
25 LWP 18963 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
26 LWP 18964 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
27 LWP 18965 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
28 LWP 18966 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
29 LWP 18967 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
30 LWP 18968 "rpc worker-1896" 0x00007fe5e8a5fad3 in ?? ()
31 LWP 18974 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
32 LWP 18975 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
33 LWP 18976 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
34 LWP 18977 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
35 LWP 18978 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
36 LWP 18979 "rpc worker-1897" 0x00007fe5e8a5fad3 in ?? ()
37 LWP 18980 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
38 LWP 18981 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
39 LWP 18982 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
40 LWP 18983 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
41 LWP 18984 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
42 LWP 18985 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
43 LWP 18987 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
44 LWP 18988 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
45 LWP 18989 "rpc worker-1898" 0x00007fe5e8a5fad3 in ?? ()
46 LWP 18990 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
47 LWP 18991 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
48 LWP 18992 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
49 LWP 18993 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
50 LWP 18994 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
51 LWP 18995 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
52 LWP 18996 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
53 LWP 18997 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
54 LWP 18998 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
55 LWP 18999 "rpc worker-1899" 0x00007fe5e8a5fad3 in ?? ()
56 LWP 19000 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
57 LWP 19001 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
58 LWP 19002 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
59 LWP 19003 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
60 LWP 19004 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
61 LWP 19005 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
62 LWP 19006 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
63 LWP 19007 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
64 LWP 19008 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
65 LWP 19009 "rpc worker-1900" 0x00007fe5e8a5fad3 in ?? ()
66 LWP 19010 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
67 LWP 19011 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
68 LWP 19012 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
69 LWP 19013 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
70 LWP 19014 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
71 LWP 19015 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
72 LWP 19016 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
73 LWP 19017 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
74 LWP 19018 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
75 LWP 19019 "rpc worker-1901" 0x00007fe5e8a5fad3 in ?? ()
76 LWP 19020 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
77 LWP 19021 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
78 LWP 19022 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
79 LWP 19023 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
80 LWP 19024 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
81 LWP 19025 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
82 LWP 19026 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
83 LWP 19027 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
84 LWP 19028 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
85 LWP 19029 "rpc worker-1902" 0x00007fe5e8a5fad3 in ?? ()
86 LWP 19030 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
87 LWP 19031 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
88 LWP 19032 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
89 LWP 19033 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
90 LWP 19034 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
91 LWP 19035 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
92 LWP 19036 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
93 LWP 19037 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
94 LWP 19038 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
95 LWP 19039 "rpc worker-1903" 0x00007fe5e8a5fad3 in ?? ()
96 LWP 19040 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
97 LWP 19041 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
98 LWP 19042 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
99 LWP 19043 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
100 LWP 19044 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
101 LWP 19045 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
102 LWP 19046 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
103 LWP 19047 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
104 LWP 19048 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
105 LWP 19049 "rpc worker-1904" 0x00007fe5e8a5fad3 in ?? ()
106 LWP 19050 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
107 LWP 19051 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
108 LWP 19052 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
109 LWP 19053 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
110 LWP 19054 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
111 LWP 19055 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
112 LWP 19056 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
113 LWP 19057 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
114 LWP 19058 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
115 LWP 19059 "rpc worker-1905" 0x00007fe5e8a5fad3 in ?? ()
116 LWP 19060 "rpc worker-1906" 0x00007fe5e8a5fad3 in ?? ()
117 LWP 19061 "diag-logger-190" 0x00007fe5e8a5ffb9 in ?? ()
118 LWP 19062 "result-tracker-" 0x00007fe5e8a5ffb9 in ?? ()
119 LWP 19063 "excess-log-dele" 0x00007fe5e8a5ffb9 in ?? ()
120 LWP 19064 "acceptor-19064" 0x00007fe5e17cb0c7 in ?? ()
121 LWP 19065 "heartbeat-19065" 0x00007fe5e8a5ffb9 in ?? ()
122 LWP 19068 "maintenance_sch" 0x00007fe5e8a5ffb9 in ?? ()
Thread 122 (LWP 19068):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x000000000000011f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fe59db34550 in ?? ()
#6 0x000000000000023e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 19065):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0xfafa113d49b0d763 in ?? ()
#2 0x0000000000000047 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001aac0 in ?? ()
#5 0x00007fe59eb5b5b0 in ?? ()
#6 0x000000000000008e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 19064):
#0 0x00007fe5e17cb0c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007fe59f382d50 in ?? ()
#3 0x00007fe59f382d80 in ?? ()
#4 0x00007fe59f382e80 in ?? ()
#5 0x00007fe59f382d70 in ?? ()
#6 0x00007fe59f382de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000000000000 in ?? ()
Thread 119 (LWP 19063):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe59fb9cf00 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007ffc06fa12f0 in ?? ()
#5 0x00007fe59fb9cef0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 118 (LWP 19062):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5a03b50c0 in ?? ()
#2 0x0000000000000047 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000611000087c30 in ?? ()
#5 0x00007fe5a03b50b0 in ?? ()
#6 0x000000000000008e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 19061):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 19060):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b0048 in ?? ()
#4 0x00007fe5a13e4e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5a13e4e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 19059):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x000000000000000c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002a6858 in ?? ()
#4 0x00007fe5a1bfce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5a1bfce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 114 (LWP 19058):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 19057):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 19056):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 19055):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 19054):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 19053):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 19052):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 19051):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 19050):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 19049):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 19048):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 19047):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 19046):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 19045):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 19044):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 19043):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 19042):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 19041):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 19040):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 19039):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 19038):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 19037):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 19036):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 19035):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 19034):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 19033):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 19032):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 19031):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 19030):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 19029):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 19028):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 19027):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 19026):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 19025):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 19024):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 19023):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 19022):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 19021):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 19020):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007fe5b57b5e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5b57b5e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 19019):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 19018):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 19017):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 19016):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 19015):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 19014):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 19013):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 19012):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 19011):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 19010):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 19009):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 19008):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 19007):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 19006):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 19005):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 19004):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 19003):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 19002):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 19001):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 19000):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fe5bf996e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5bf996e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 18999):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 18998):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 18997):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 18996):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 18995):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 18994):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 18993):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 18992):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 18991):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 18990):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 18989):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 18988):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 18987):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 18985):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 18984):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 18983):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 18982):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 18981):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 18980):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 18979):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007fe5c9b77e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe5c9b77e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 18978):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 18977):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 18976):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 18975):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 18974):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 18968):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 18967):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 18966):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 18965):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 18964):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 18963):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 18962):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 18961):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 18960):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 18959):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 18958):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 18957):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 18956):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 18955):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 18954):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d3561c80 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b360 in ?? ()
#5 0x00007fe5d3561c70 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 18953):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 18952):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d459f1c0 in ?? ()
#2 0x0000000000000007 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000f9a8 in ?? ()
#5 0x00007fe5d459f1b0 in ?? ()
#6 0x000000000000000e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 18951):
#0 0x00007fe5e8a5fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 18950):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d55b22e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fe5d55b22d0 in ?? ()
#4 0x00007fe5d55b24e0 in ?? ()
#5 0x00007fe5d55b2320 in ?? ()
#6 0x0000614000021898 in ?? ()
#7 0x00007fe5d55b23a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb951f311ed4000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000038800000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ffd3aaae474 in ?? ()
#17 0x00007fe5d55b2380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d55b2350 in ?? ()
#20 0x3fb951f311ed4000 in ?? ()
#21 0x00000000d55b23a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000021898 in ?? ()
#24 0x3fb951f311ed4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 18949):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d5db32e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fe5d5db32d0 in ?? ()
#4 0x00007fe5d5db34e0 in ?? ()
#5 0x00007fe5d5db3320 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fe5d5db33a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb951a6b33c6000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x0000000069000000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d25 in ?? ()
#16 0x00000ffd3abae674 in ?? ()
#17 0x00007fe5d5db3380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d5db3350 in ?? ()
#20 0x3fb951a6b33c6000 in ?? ()
#21 0x00000000d5db33a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb951a6b33c6000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 18948):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d85ad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fe5d85ad2d0 in ?? ()
#4 0x00007fe5d85ad4e0 in ?? ()
#5 0x00007fe5d85ad320 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fe5d85ad3a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb985a5a0394000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000086400000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d23 in ?? ()
#16 0x00000ffd3b0ada74 in ?? ()
#17 0x00007fe5d85ad380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x00007fe5d85ad350 in ?? ()
#20 0x3fb985a5a0394000 in ?? ()
#21 0x00000000d85ad3a0 in ?? ()
#22 0x000061a00000e480 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb985a5a0394000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 18947):
#0 0x00007fe5e17c9a47 in ?? ()
#1 0x00007fe5d65b42e0 in ?? ()
#2 0x000061a00000de80 in ?? ()
#3 0x00007fe5d65b42d0 in ?? ()
#4 0x00007fe5d65b44e0 in ?? ()
#5 0x00007fe5d65b4320 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fe5d65b43a0 in ?? ()
#8 0x00007fe5e3e2925d in ?? ()
#9 0x3fb957cb2da2c000 in ?? ()
#10 0x000061a00000de80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000de80 in ?? ()
#13 0x00000000fb800000 in ?? ()
#14 0x00007fe500000000 in ?? ()
#15 0x41d9e0b657318d26 in ?? ()
#16 0x00000ffd3acae874 in ?? ()
#17 0x00007fe5d65b4380 in ?? ()
#18 0x00007fe5e3e2dba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 18944):
#0 0x00007fe5e17bccb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fe5d9fb7788 in ?? ()
#3 0x0000602000016c90 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 18943):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5d97b6080 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000e0c0 in ?? ()
#5 0x00007fe5d97b6070 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 18942):
#0 0x00007fe5e8a639e2 in ?? ()
#1 0x00007fe5d8fb3b60 in ?? ()
#2 0x00007fe5d8fb3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fe5d8fb3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fe5d8fb3c10 in ?? ()
#8 0x0000611000088bd0 in ?? ()
#9 0x00007fe5d8fb3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fe5ee387f0c in ?? ()
#12 0x00007fe5ee3777b9 in ?? ()
#13 0x00007fe5ee37bda6 in ?? ()
#14 0x00007fe5ee3807e0 in ?? ()
#15 0x00007fe5ee380609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fe5e56760be in ?? ()
#18 0x00007fe5e8a596db in ?? ()
#19 0x00000ffd3b1ee67c in ?? ()
#20 0x00007fe5d8fb3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 18936):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x00007fe5dafb8c40 in ?? ()
#2 0x000000000000016e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fe5dafb8c30 in ?? ()
#6 0x00000000000002dc in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 18935):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 18934):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 18933):
#0 0x00007fe5e8a5ffb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000004 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fe5dc7bc410 in ?? ()
#6 0x0000000000000008 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 18929):
#0 0x00007fe5e8a63d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250111 21:02:11.458729 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17622
I20250111 21:02:11.473263 17750 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:11.604350 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17622
I20250111 21:02:11.632748 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17338
I20250111 21:02:11.639816 17464 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:11.768431 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17338
I20250111 21:02:11.793288 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 17479
I20250111 21:02:11.800853 17605 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.022619 19663 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.022848 19663 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.023700 19663 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529), cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8:41133)
I20250111 21:02:12.037497 19667 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.037674 19667 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.038491 19667 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:12.038937 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee" is_pre_election: true
I20250111 21:02:12.039214 18177 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc4301cbd2b84d22ab621b2c399022dc in term 2.
I20250111 21:02:12.039685 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 21:02:12.039994 19667 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250111 21:02:12.040087 19667 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Starting leader election (detected failure of leader dc04179134174d03bd03d3db91b245f8)
I20250111 21:02:12.040182 19667 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 2 FOLLOWER]: Advancing to term 3
I20250111 21:02:12.042973 18872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "edf2bd29095149dd9e4592ae2870d4ee" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "cc4301cbd2b84d22ab621b2c399022dc" is_pre_election: true
I20250111 21:02:12.045189 19667 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.045614 18872 raft_consensus.cc:2388] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edf2bd29095149dd9e4592ae2870d4ee in current term 3: Already voted for candidate cc4301cbd2b84d22ab621b2c399022dc in this term.
I20250111 21:02:12.045905 19667 leader_election.cc:290] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: Requested vote from peers edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445), dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529)
I20250111 21:02:12.046509 18177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "edf2bd29095149dd9e4592ae2870d4ee"
I20250111 21:02:12.046746 18177 raft_consensus.cc:3054] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 2 FOLLOWER]: Advancing to term 3
I20250111 21:02:12.052289 18177 raft_consensus.cc:2463] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc4301cbd2b84d22ab621b2c399022dc in term 3.
I20250111 21:02:12.052911 18805 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc4301cbd2b84d22ab621b2c399022dc, edf2bd29095149dd9e4592ae2870d4ee; no voters:
I20250111 21:02:12.053246 19667 raft_consensus.cc:2798] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 FOLLOWER]: Leader election won for term 3
I20250111 21:02:12.053663 19667 raft_consensus.cc:695] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 LEADER]: Becoming Leader. State: Replica: cc4301cbd2b84d22ab621b2c399022dc, State: Running, Role: LEADER
I20250111 21:02:12.054176 19667 consensus_queue.cc:237] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15602, Committed index: 15602, Last appended: 2.15602, Last appended by leader: 15602, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:12.058341 15272 catalog_manager.cc:5526] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc reported cstate change: term changed from 2 to 3, leader changed from dc04179134174d03bd03d3db91b245f8 (127.14.226.3) to cc4301cbd2b84d22ab621b2c399022dc (127.14.226.8). New cstate: current_term: 3 leader_uuid: "cc4301cbd2b84d22ab621b2c399022dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } health_report { overall_health: HEALTHY } } }
I20250111 21:02:12.153136 18177 raft_consensus.cc:1270] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Refusing update from remote peer cc4301cbd2b84d22ab621b2c399022dc: Log matching property violated. Preceding OpId in replica: term: 2 index: 15602. Preceding OpId from leader: term: 3 index: 15603. (index mismatch)
I20250111 21:02:12.153901 19667 consensus_queue.cc:1035] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15603, Last known committed idx: 15602, Time since last communication: 0.000s
I20250111 21:02:12.176502 17565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a6628492fa41d593f0c59af04ad973" candidate_uuid: "cc4301cbd2b84d22ab621b2c399022dc" candidate_term: 3 candidate_status { last_received { term: 2 index: 15602 } } ignore_live_leader: false dest_uuid: "dc04179134174d03bd03d3db91b245f8" is_pre_election: true
I20250111 21:02:12.176983 18872 raft_consensus.cc:1235] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [term 3 LEADER]: Rejecting Update request from peer dc04179134174d03bd03d3db91b245f8 for earlier term 2. Current term is 3. Ops: []
I20250111 21:02:12.177083 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 17479
W20250111 21:02:12.237635 18805 connection.cc:537] client connection to 127.14.226.3:45529 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.237951 18805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250111 21:02:12.238274 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16832
W20250111 21:02:12.238996 19665 negotiation.cc:337] Failed RPC negotiation. Trace:
0111 21:02:12.023988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.226.3:45529 (local address 127.14.226.6:55827)
0111 21:02:12.035483 (+ 11495us) negotiation.cc:107] Waiting for socket to connect
0111 21:02:12.035512 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0111 21:02:12.035702 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0111 21:02:12.177013 (+141311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0111 21:02:12.177019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0111 21:02:12.177061 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0111 21:02:12.177595 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0111 21:02:12.177614 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0111 21:02:12.237845 (+ 60231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":11341,"thread_start_us":11691,"threads_started":1}
W20250111 21:02:12.239326 18112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.240811 18805 connection.cc:537] server connection from 127.14.226.3:51951 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250111 21:02:12.241119 18805 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
W20250111 21:02:12.241421 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250111 21:02:12.241533 18805 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
W20250111 21:02:12.242179 18112 leader_election.cc:336] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111)
I20250111 21:02:12.242411 18112 leader_election.cc:304] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf2bd29095149dd9e4592ae2870d4ee; no voters: cc4301cbd2b84d22ab621b2c399022dc, dc04179134174d03bd03d3db91b245f8
I20250111 21:02:12.242713 19663 raft_consensus.cc:2743] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250111 21:02:12.247170 16959 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.378327 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16832
I20250111 21:02:12.404733 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16694
I20250111 21:02:12.411810 16818 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:12.540218 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16694
I20250111 21:02:12.572142 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18094
I20250111 21:02:12.578969 18224 generic_service.cc:196] Checking for leaks (request via RPC)
W20250111 21:02:12.671893 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250111 21:02:13.046213 19663 raft_consensus.cc:491] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cc4301cbd2b84d22ab621b2c399022dc)
I20250111 21:02:13.046483 19663 raft_consensus.cc:513] T 54a6628492fa41d593f0c59af04ad973 P edf2bd29095149dd9e4592ae2870d4ee [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf2bd29095149dd9e4592ae2870d4ee" member_type: VOTER last_known_addr { host: "127.14.226.6" port: 42445 } } peers { permanent_uuid: "dc04179134174d03bd03d3db91b245f8" member_type: VOTER last_known_addr { host: "127.14.226.3" port: 45529 } } peers { permanent_uuid: "cc4301cbd2b84d22ab621b2c399022dc" member_type: VOTER last_known_addr { host: "127.14.226.8" port: 41133 } }
I20250111 21:02:13.046813 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18094
W20250111 21:02:13.117154 18805 connection.cc:537] client connection to 127.14.226.6:42445 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250111 21:02:13.117573 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 16972
W20250111 21:02:13.118556 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer edf2bd29095149dd9e4592ae2870d4ee (127.14.226.6:42445): Couldn't send request to peer edf2bd29095149dd9e4592ae2870d4ee. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.6:42445: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250111 21:02:13.124231 17101 generic_service.cc:196] Checking for leaks (request via RPC)
W20250111 21:02:13.146845 18805 consensus_peers.cc:487] T 54a6628492fa41d593f0c59af04ad973 P cc4301cbd2b84d22ab621b2c399022dc -> Peer dc04179134174d03bd03d3db91b245f8 (127.14.226.3:45529): Couldn't send request to peer dc04179134174d03bd03d3db91b245f8. Status: Network error: Client connection negotiation failed: client connection to 127.14.226.3:45529: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250111 21:02:13.254453 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 16972
I20250111 21:02:13.279937 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18787
I20250111 21:02:13.286808 18913 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:13.701277 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18787
I20250111 21:02:13.764771 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 18929
I20250111 21:02:13.771960 19060 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:13.876780 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 18929
I20250111 21:02:13.900236 15240 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskBASTb8/build/asan/bin/kudu pid 15248
I20250111 21:02:13.907297 15308 generic_service.cc:196] Checking for leaks (request via RPC)
I20250111 21:02:14.100737 15240 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskBASTb8/build/asan/bin/kudu with pid 15248
2025-01-11T21:02:14Z chronyd exiting
I20250111 21:02:14.158674 15240 test_util.cc:181] -----------------------------------------------
I20250111 21:02:14.158769 15240 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-taskBASTb8/test-tmp/maintenance_mode-itest.7.RollingRestartArgs_RollingRestartITest.TestWorkloads_3.1736629109910169-15240-0
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3, where GetParam() = 800-byte object <01-00 00-00 01-61 00-00 09-00 00-00 BE-BE BE-BE A0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-1F 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 BE-BE BE-BE BE-BE 03-00 00-00 03-00 00-00 00-BE BE-BE BE-BE BE-BE> (224178 ms)
[----------] 1 test from RollingRestartArgs/RollingRestartITest (224178 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (224178 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/3, where GetParam() = 800-byte object <01-00 00-00 01-61 00-00 09-00 00-00 BE-BE BE-BE A0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-1C 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-1F 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 BE-BE BE-BE BE-BE 03-00 00-00 03-00 00-00 00-BE BE-BE BE-BE BE-BE>
1 FAILED TEST