Diagnosed failure

UpdateScanDeltaCompactionTest.TestAll: Unrecognized error type. Please see the error log for more information.

Full log

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from UpdateScanDeltaCompactionTest
[ RUN      ] UpdateScanDeltaCompactionTest.TestAll
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240426 10:00:05.760174  1070 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.1.11.190:36195
I20240426 10:00:05.761554  1070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 10:00:05.762614  1070 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 10:00:05.772954  1076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 10:00:05.773384  1075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 10:00:05.773778  1078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 10:00:05.774411  1070 server_base.cc:1034] running on GCE node
I20240426 10:00:05.775101  1070 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 10:00:05.775278  1070 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 10:00:05.775349  1070 hybrid_clock.cc:648] HybridClock initialized: now 1714125605775333 us; error 0 us; skew 500 ppm
I20240426 10:00:05.775663  1070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240426 10:00:05.778028  1070 webserver.cc:460] Webserver started at http://127.1.11.190:36025/ using document root <none> and password file <none>
I20240426 10:00:05.778746  1070 fs_manager.cc:359] Metadata directory not provided
I20240426 10:00:05.778848  1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 10:00:05.779178  1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 10:00:05.782114  1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/master-0-root/instance:
uuid: "920f5f1415c349039150eecf9671a7bc"
format_stamp: "Formatted at 2024-04-26 10:00:05 on dist-test-slave-b435"
I20240426 10:00:05.787163  1070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20240426 10:00:05.790032  1083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 10:00:05.791250  1070 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20240426 10:00:05.791412  1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/master-0-root
uuid: "920f5f1415c349039150eecf9671a7bc"
format_stamp: "Formatted at 2024-04-26 10:00:05 on dist-test-slave-b435"
I20240426 10:00:05.791571  1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 10:00:05.812737  1070 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 10:00:05.813459  1070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 10:00:05.813709  1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 10:00:05.830427  1070 rpc_server.cc:307] RPC server started. Bound to: 127.1.11.190:36195
I20240426 10:00:05.830457  1134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.11.190:36195 every 8 connection(s)
I20240426 10:00:05.833611  1135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 10:00:05.844813  1135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc: Bootstrap starting.
I20240426 10:00:05.847883  1135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc: Neither blocks nor log segments found. Creating new log.
I20240426 10:00:05.848850  1135 log.cc:824] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc: Log is configured to *not* fsync() on all Append() calls
I20240426 10:00:05.851011  1135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc: No bootstrap required, opened a new log
I20240426 10:00:05.857124  1135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER }
I20240426 10:00:05.857386  1135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 10:00:05.857501  1135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 920f5f1415c349039150eecf9671a7bc, State: Initialized, Role: FOLLOWER
I20240426 10:00:05.858178  1135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER }
I20240426 10:00:05.858388  1135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 10:00:05.858513  1135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 10:00:05.858673  1135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 0 FOLLOWER]: Advancing to term 1
I20240426 10:00:05.860391  1135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER }
I20240426 10:00:05.860888  1135 leader_election.cc:302] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 920f5f1415c349039150eecf9671a7bc; no voters: 
I20240426 10:00:05.861470  1135 leader_election.cc:288] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 10:00:05.861603  1138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 1 FOLLOWER]: Leader election won for term 1
I20240426 10:00:05.862221  1138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [term 1 LEADER]: Becoming Leader. State: Replica: 920f5f1415c349039150eecf9671a7bc, State: Running, Role: LEADER
I20240426 10:00:05.862685  1138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER }
I20240426 10:00:05.863261  1135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [sys.catalog]: configured and running, proceeding with master startup.
I20240426 10:00:05.865517  1139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "920f5f1415c349039150eecf9671a7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER } }
I20240426 10:00:05.865705  1140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 920f5f1415c349039150eecf9671a7bc. Latest consensus state: current_term: 1 leader_uuid: "920f5f1415c349039150eecf9671a7bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920f5f1415c349039150eecf9671a7bc" member_type: VOTER } }
I20240426 10:00:05.865859  1139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [sys.catalog]: This master's current role is: LEADER
I20240426 10:00:05.865942  1140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc [sys.catalog]: This master's current role is: LEADER
I20240426 10:00:05.867893  1147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 10:00:05.871856  1147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 10:00:05.872498  1070 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 10:00:05.879194  1147 catalog_manager.cc:1347] Generated new cluster ID: cb44da495640422397d4243275d1930b
I20240426 10:00:05.879297  1147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 10:00:05.902592  1147 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 10:00:05.904039  1147 catalog_manager.cc:1504] Loading token signing keys...
I20240426 10:00:05.914445  1147 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 920f5f1415c349039150eecf9671a7bc: Generated new TSK 0
I20240426 10:00:05.915326  1147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 10:00:05.937615  1070 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 10:00:05.941743  1157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 10:00:05.942162  1158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 10:00:05.942543  1160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 10:00:05.942956  1070 server_base.cc:1034] running on GCE node
I20240426 10:00:05.943220  1070 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 10:00:05.943310  1070 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 10:00:05.943367  1070 hybrid_clock.cc:648] HybridClock initialized: now 1714125605943349 us; error 0 us; skew 500 ppm
I20240426 10:00:05.943539  1070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240426 10:00:05.944931  1070 webserver.cc:460] Webserver started at http://127.1.11.129:41179/ using document root <none> and password file <none>
I20240426 10:00:05.945163  1070 fs_manager.cc:359] Metadata directory not provided
I20240426 10:00:05.945233  1070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 10:00:05.945386  1070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 10:00:05.946031  1070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/ts-0-root/instance:
uuid: "556c15a7f6b54b1da959a30f940b0aa1"
format_stamp: "Formatted at 2024-04-26 10:00:05 on dist-test-slave-b435"
I20240426 10:00:05.948612  1070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s	user 0.001s	sys 0.003s
I20240426 10:00:05.950337  1165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 10:00:05.950717  1070 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 10:00:05.950840  1070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/ts-0-root
uuid: "556c15a7f6b54b1da959a30f940b0aa1"
format_stamp: "Formatted at 2024-04-26 10:00:05 on dist-test-slave-b435"
I20240426 10:00:05.950965  1070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskhSsTI7/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125605698364-1070-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 10:00:05.966184  1070 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 10:00:05.966910  1070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 10:00:05.967911  1070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 10:00:05.969575  1070 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 10:00:05.969718  1070 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 10:00:05.969833  1070 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 10:00:05.969919  1070 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 10:00:05.987716  1070 rpc_server.cc:307] RPC server started. Bound to: 127.1.11.129:42991
I20240426 10:00:05.987746  1227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.11.129:42991 every 8 connection(s)
I20240426 10:00:06.002681  1228 heartbeater.cc:347] Connected to a master server at 127.1.11.190:36195
I20240426 10:00:06.002959  1228 heartbeater.cc:464] Registering TS with master...
I20240426 10:00:06.003625  1228 heartbeater.cc:511] Master 127.1.11.190:36195 requested a full tablet report, sending...
I20240426 10:00:06.005569  1100 ts_manager.cc:194] Registered new tserver with Master: 556c15a7f6b54b1da959a30f940b0aa1 (127.1.11.129:42991)
I20240426 10:00:06.005847  1070 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016825444s
I20240426 10:00:06.007525  1100 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49064
I20240426 10:00:06.028859  1100 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49076:
name: "update-scan-delta-compact-tbl"
schema {
  columns {
    name: "key"
    type: INT64
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int64"
    type: INT64
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20240426 10:00:06.052242  1193 tablet_service.cc:1467] Processing CreateTablet for tablet b9de15633da24ce8ae0aa241b21b28e4 (DEFAULT_TABLE table=update-scan-delta-compact-tbl [id=7513ad3826624ec7811b445aebb3a889]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 10:00:06.052963  1193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9de15633da24ce8ae0aa241b21b28e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 10:00:06.058284  1240 tablet_bootstrap.cc:492] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1: Bootstrap starting.
I20240426 10:00:06.059864  1240 tablet_bootstrap.cc:654] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1: Neither blocks nor log segments found. Creating new log.
I20240426 10:00:06.061820  1240 tablet_bootstrap.cc:492] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1: No bootstrap required, opened a new log
I20240426 10:00:06.061998  1240 ts_tablet_manager.cc:1367] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20240426 10:00:06.062620  1240 raft_consensus.cc:348] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c15a7f6b54b1da959a30f940b0aa1" member_type: VOTER last_known_addr { host: "127.1.11.129" port: 42991 } }
I20240426 10:00:06.062804  1240 raft_consensus.cc:374] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 10:00:06.062959  1240 raft_consensus.cc:729] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556c15a7f6b54b1da959a30f940b0aa1, State: Initialized, Role: FOLLOWER
I20240426 10:00:06.063156  1240 consensus_queue.cc:260] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c15a7f6b54b1da959a30f940b0aa1" member_type: VOTER last_known_addr { host: "127.1.11.129" port: 42991 } }
I20240426 10:00:06.063390  1240 raft_consensus.cc:388] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 10:00:06.063524  1240 raft_consensus.cc:482] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 10:00:06.063661  1240 raft_consensus.cc:3037] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 0 FOLLOWER]: Advancing to term 1
I20240426 10:00:06.065490  1240 raft_consensus.cc:504] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c15a7f6b54b1da959a30f940b0aa1" member_type: VOTER last_known_addr { host: "127.1.11.129" port: 42991 } }
I20240426 10:00:06.065704  1240 leader_election.cc:302] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 556c15a7f6b54b1da959a30f940b0aa1; no voters: 
I20240426 10:00:06.066563  1240 leader_election.cc:288] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 10:00:06.066912  1242 raft_consensus.cc:2781] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 10:00:06.067679  1228 heartbeater.cc:503] Master 127.1.11.190:36195 was elected leader, sending a full tablet report...
I20240426 10:00:06.068267  1240 ts_tablet_manager.cc:1398] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20240426 10:00:06.073241  1242 raft_consensus.cc:686] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [term 1 LEADER]: Becoming Leader. State: Replica: 556c15a7f6b54b1da959a30f940b0aa1, State: Running, Role: LEADER
I20240426 10:00:06.073557  1242 consensus_queue.cc:237] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c15a7f6b54b1da959a30f940b0aa1" member_type: VOTER last_known_addr { host: "127.1.11.129" port: 42991 } }
I20240426 10:00:06.077481  1100 catalog_manager.cc:5484] T b9de15633da24ce8ae0aa241b21b28e4 P 556c15a7f6b54b1da959a30f940b0aa1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 556c15a7f6b54b1da959a30f940b0aa1 (127.1.11.129). New cstate: current_term: 1 leader_uuid: "556c15a7f6b54b1da959a30f940b0aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c15a7f6b54b1da959a30f940b0aa1" member_type: VOTER last_known_addr { host: "127.1.11.129" port: 42991 } health_report { overall_health: HEALTHY } } }
I20240426 10:00:09.989759  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=2.874778
I20240426 10:00:12.502944  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.513s	user 2.196s	sys 0.304s Metrics: {"bytes_written":211201,"cfile_init":1,"compiler_manager_pool.queue_time_us":343,"compiler_manager_pool.run_cpu_time_us":493445,"compiler_manager_pool.run_wall_time_us":495130,"dirs.queue_time_us":357,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":956,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":5578,"lbm_writes_lt_1ms":107,"peak_mem_usage":0,"rows_written":76001,"thread_start_us":506,"threads_started":2}
I20240426 10:00:12.504448  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 510305 bytes on disk
I20240426 10:00:12.505102  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4}
I20240426 10:00:14.505982  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=4.859153
I20240426 10:00:16.814801  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.309s	user 2.224s	sys 0.068s Metrics: {"bytes_written":236407,"cfile_init":1,"dirs.queue_time_us":296,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":1792,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6103,"lbm_writes_lt_1ms":120,"peak_mem_usage":0,"rows_written":88000,"thread_start_us":201,"threads_started":1}
I20240426 10:00:16.816313  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 589011 bytes on disk
I20240426 10:00:16.816890  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":106,"lbm_reads_lt_1ms":4}
I20240426 10:00:16.817569  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=0.014585
I20240426 10:00:19.197710  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.380s	user 2.286s	sys 0.076s Metrics: {"bytes_written":451063,"cfile_cache_hit":10,"cfile_cache_hit_bytes":2104,"cfile_cache_miss":198,"cfile_cache_miss_bytes":2289948,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":1980,"dirs.run_cpu_time_us":202,"dirs.run_wall_time_us":1636,"drs_written":1,"lbm_read_time_us":5947,"lbm_reads_lt_1ms":218,"lbm_write_time_us":10053,"lbm_writes_lt_1ms":209,"num_input_rowsets":2,"peak_mem_usage":1359469,"rows_written":164001,"thread_start_us":1322,"threads_started":6}
I20240426 10:00:19.198570  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=3.866965
I20240426 10:00:21.485078  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.286s	user 2.205s	sys 0.072s Metrics: {"bytes_written":233030,"cfile_init":1,"dirs.queue_time_us":378,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":1010,"drs_written":1,"lbm_read_time_us":97,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6310,"lbm_writes_lt_1ms":118,"peak_mem_usage":0,"rows_written":87000,"thread_start_us":269,"threads_started":1}
I20240426 10:00:21.485845  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=0.014365
I20240426 10:00:24.806905  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 3.321s	user 3.248s	sys 0.064s Metrics: {"bytes_written":690740,"cfile_cache_hit":10,"cfile_cache_hit_bytes":2995,"cfile_cache_miss":298,"cfile_cache_miss_bytes":3520105,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":2211,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1641,"drs_written":1,"lbm_read_time_us":8728,"lbm_reads_lt_1ms":326,"lbm_write_time_us":14106,"lbm_writes_lt_1ms":308,"num_input_rowsets":2,"peak_mem_usage":2627180,"rows_written":251001,"thread_start_us":1692,"threads_started":7}
I20240426 10:00:24.807679  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 1684702 bytes on disk
I20240426 10:00:24.808470  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":325,"lbm_reads_lt_1ms":4}
I20240426 10:00:24.808956  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=6.843528
I20240426 10:00:27.411329  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.602s	user 2.506s	sys 0.092s Metrics: {"bytes_written":276539,"cfile_init":1,"dirs.queue_time_us":351,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":955,"drs_written":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4,"lbm_write_time_us":7554,"lbm_writes_lt_1ms":138,"peak_mem_usage":0,"rows_written":103000,"thread_start_us":244,"threads_started":1}
I20240426 10:00:27.412277  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=0.014104
I20240426 10:00:32.687916  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 5.275s	user 5.108s	sys 0.120s Metrics: {"bytes_written":977051,"cfile_cache_hit":10,"cfile_cache_hit_bytes":4146,"cfile_cache_miss":414,"cfile_cache_miss_bytes":4959936,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":1917,"dirs.run_cpu_time_us":226,"dirs.run_wall_time_us":108797,"drs_written":1,"lbm_read_time_us":13517,"lbm_reads_lt_1ms":438,"lbm_write_time_us":44559,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":427,"num_input_rowsets":2,"peak_mem_usage":4040832,"rows_written":354001,"thread_start_us":1358,"threads_started":7}
I20240426 10:00:32.688712  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 2376129 bytes on disk
I20240426 10:00:32.689280  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20240426 10:00:32.689834  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=11.804465
I20240426 10:00:36.408824  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 3.719s	user 3.499s	sys 0.216s Metrics: {"bytes_written":378576,"cfile_init":1,"dirs.queue_time_us":361,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":1015,"drs_written":1,"lbm_read_time_us":44,"lbm_reads_lt_1ms":4,"lbm_write_time_us":9642,"lbm_writes_lt_1ms":182,"peak_mem_usage":0,"rows_written":141000,"thread_start_us":267,"threads_started":1}
I20240426 10:00:36.409762  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=1.000000
I20240426 10:00:38.139083  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 1.729s	user 1.683s	sys 0.020s Metrics: {"bytes_written":177164,"cfile_init":1,"dirs.queue_time_us":326,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":935,"drs_written":1,"lbm_read_time_us":43,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4855,"lbm_writes_lt_1ms":93,"peak_mem_usage":0,"rows_written":65000,"thread_start_us":241,"threads_started":1}
I20240426 10:00:38.140102  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 1382865 bytes on disk
I20240426 10:00:38.140774  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":134,"lbm_reads_lt_1ms":8}
I20240426 10:00:38.141279  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=0.022581
I20240426 10:00:45.936120  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 7.795s	user 7.607s	sys 0.159s Metrics: {"bytes_written":1541730,"cfile_cache_hit":15,"cfile_cache_hit_bytes":6452,"cfile_cache_miss":653,"cfile_cache_miss_bytes":7852358,"cfile_init":7,"delta_iterators_relevant":6,"dirs.queue_time_us":378,"dirs.run_cpu_time_us":102,"dirs.run_wall_time_us":103,"drs_written":1,"lbm_read_time_us":19485,"lbm_reads_lt_1ms":681,"lbm_write_time_us":33748,"lbm_writes_lt_1ms":667,"num_input_rowsets":3,"peak_mem_usage":5747250,"rows_written":560001,"thread_start_us":246,"threads_started":1}
I20240426 10:00:45.938486  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 3761676 bytes on disk
I20240426 10:00:45.938969  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20240426 10:00:45.939455  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=17.757590
I20240426 10:00:50.774847  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 4.835s	user 4.664s	sys 0.144s Metrics: {"bytes_written":490243,"cfile_init":1,"dirs.queue_time_us":560,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":982,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":13179,"lbm_writes_lt_1ms":230,"peak_mem_usage":0,"rows_written":183000,"thread_start_us":459,"threads_started":1}
I20240426 10:00:50.775638  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4): 1230094 bytes on disk
I20240426 10:00:50.776157  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: UndoDeltaBlockGCOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4}
I20240426 10:00:50.776741  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=4.859153
I20240426 10:00:53.144346  1170 maintenance_manager.cc:643] P 556c15a7f6b54b1da959a30f940b0aa1: FlushMRSOp(b9de15633da24ce8ae0aa241b21b28e4) complete. Timing: real 2.367s	user 2.259s	sys 0.100s Metrics: {"bytes_written":242148,"cfile_init":1,"dirs.queue_time_us":323,"dirs.run_cpu_time_us":172,"dirs.run_wall_time_us":999,"drs_written":1,"lbm_read_time_us":43,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6392,"lbm_writes_lt_1ms":121,"peak_mem_usage":0,"rows_written":89000,"thread_start_us":235,"threads_started":1}
I20240426 10:00:53.145320  1229 maintenance_manager.cc:419] P 556c15a7f6b54b1da959a30f940b0aa1: Scheduling CompactRowSetsOp(b9de15633da24ce8ae0aa241b21b28e4): perf score=0.021891
I20240426 10:00:59.850507  1070 update_scan_delta_compact-test.cc:209] Time spent Insert: real 53.762s	user 49.035s	sys 4.130s
I20240426 10:00:59.896589  1286 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 10:00:59.900722  1286 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 10:00:59.903769  1286 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 10:00:59.906541  1286 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
F20240426 10:00:59.908025  1286 monotime.cc:342] Check failed: t_beg.Initialized() 
*** Check failure stack trace: ***
*** Aborted at 1714125659 (unix time) try "date -d @1714125659" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e80000042e) received by PID 1070 (TID 0x7f16b5166700) from PID 1070; stack trace: ***
    @     0x7f16e576e980 (unknown) at ??:0
    @     0x7f16de81efb7 gsignal at ??:0
    @     0x7f16de820921 abort at ??:0
    @     0x7f16e1546dbd google::LogMessage::Fail() at ??:0
    @     0x7f16e154ab83 google::LogMessage::SendToLog() at ??:0
    @     0x7f16e15467bc google::LogMessage::Flush() at ??:0
    @     0x7f16e1547f49 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f16e2aaa880 kudu::operator-() at ??:0
    @     0x7f16ee2bf39a kudu::tablet::WriteOp::ToString() at ??:0
    @     0x7f16ee28982e kudu::tablet::OpDriver::ToStringUnlocked() at ??:0
    @     0x7f16ee289586 kudu::tablet::OpDriver::ToString() at ??:0
    @     0x7f16ee236529 kudu::tablet::TabletReplica::GetInFlightOps() at ??:0
    @     0x7f16f50ae0a7 kudu::tserver::TabletServerPathHandlers::HandleTransactionsPage() at ??:0
    @     0x7f16f50bcc5c kudu::tserver::TabletServerPathHandlers::Register()::$_3::operator()() at ??:0
    @     0x7f16f50bcac2 std::_Function_handler<>::_M_invoke() at ??:0
    @     0x7f16eb4256aa std::function<>::operator()() at ??:0
    @     0x7f16eb41de3a kudu::Webserver::RunPathHandler() at ??:0
    @     0x7f16eb418e71 kudu::Webserver::BeginRequestCallback() at ??:0
    @     0x7f16eb416fa7 kudu::Webserver::BeginRequestCallbackStatic() at ??:0
    @     0x7f16eb454709 handle_request at ??:0
    @     0x7f16eb456deb worker_thread at ??:0
    @     0x7f16e57636db start_thread at ??:0
    @     0x7f16de90171f clone at ??:0