Diagnosed failure

UpdateScanDeltaCompactionTest.TestAll: Unrecognized error type. Please see the error log for more information.

Full log

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from UpdateScanDeltaCompactionTest
[ RUN      ] UpdateScanDeltaCompactionTest.TestAll
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 13:59:34.558430 28144 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.27.124.62:34509
I20240502 13:59:34.559211 28144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:34.559732 28144 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:34.566885 28149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:34.567163 28152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:34.567633 28150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:34.568092 28144 server_base.cc:1034] running on GCE node
I20240502 13:59:34.568511 28144 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 13:59:34.568604 28144 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 13:59:34.568642 28144 hybrid_clock.cc:648] HybridClock initialized: now 1714658374568640 us; error 0 us; skew 500 ppm
I20240502 13:59:34.570279 28144 webserver.cc:460] Webserver started at http://127.27.124.62:43559/ using document root <none> and password file <none>
I20240502 13:59:34.570709 28144 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:34.570760 28144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:34.570945 28144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:34.572475 28144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/master-0-root/instance:
uuid: "6b68e7f27d0940b5a07c4fce97d051e4"
format_stamp: "Formatted at 2024-05-02 13:59:34 on dist-test-slave-0cl8"
I20240502 13:59:34.575474 28144 fs_manager.cc:691] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.002s
I20240502 13:59:34.577364 28157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240502 13:59:34.578282 28144 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20240502 13:59:34.578384 28144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/master-0-root
uuid: "6b68e7f27d0940b5a07c4fce97d051e4"
format_stamp: "Formatted at 2024-05-02 13:59:34 on dist-test-slave-0cl8"
I20240502 13:59:34.578462 28144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:34.595124 28144 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240502 13:59:34.595597 28144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:34.595731 28144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:34.601792 28144 rpc_server.cc:307] RPC server started. Bound to: 127.27.124.62:34509
I20240502 13:59:34.601795 28209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.124.62:34509 every 8 connection(s)
I20240502 13:59:34.603802 28210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:34.609113 28210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4: Bootstrap starting.
I20240502 13:59:34.611161 28210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:34.611738 28210 log.cc:824] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:34.613050 28210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4: No bootstrap required, opened a new log
I20240502 13:59:34.615676 28210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER }
I20240502 13:59:34.615825 28210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:34.615891 28210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b68e7f27d0940b5a07c4fce97d051e4, State: Initialized, Role: FOLLOWER
I20240502 13:59:34.616394 28210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER }
I20240502 13:59:34.616564 28210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:34.616624 28210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:34.616734 28210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:34.617345 28210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER }
I20240502 13:59:34.617725 28210 leader_election.cc:302] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b68e7f27d0940b5a07c4fce97d051e4; no voters: 
I20240502 13:59:34.617974 28210 leader_election.cc:288] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240502 13:59:34.618067 28213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:34.618278 28213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [term 1 LEADER]: Becoming Leader. State: Replica: 6b68e7f27d0940b5a07c4fce97d051e4, State: Running, Role: LEADER
I20240502 13:59:34.618680 28213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER }
I20240502 13:59:34.618788 28210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:34.620114 28215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b68e7f27d0940b5a07c4fce97d051e4. Latest consensus state: current_term: 1 leader_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER } }
I20240502 13:59:34.620232 28215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:34.620549 28224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:34.620529 28214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b68e7f27d0940b5a07c4fce97d051e4" member_type: VOTER } }
I20240502 13:59:34.620608 28214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:34.620983 28144 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240502 13:59:34.622545 28224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:34.626519 28224 catalog_manager.cc:1347] Generated new cluster ID: 3ca506342175427c9e71e23486d535da
I20240502 13:59:34.626576 28224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:34.646392 28224 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:34.647101 28224 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:34.652014 28224 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 6b68e7f27d0940b5a07c4fce97d051e4: Generated new TSK 0
I20240502 13:59:34.652580 28224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:34.685549 28144 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:34.688225 28233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:34.688580 28232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:34.688927 28235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:34.689491 28144 server_base.cc:1034] running on GCE node
I20240502 13:59:34.689657 28144 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240502 13:59:34.689693 28144 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240502 13:59:34.689707 28144 hybrid_clock.cc:648] HybridClock initialized: now 1714658374689707 us; error 0 us; skew 500 ppm
I20240502 13:59:34.690608 28144 webserver.cc:460] Webserver started at http://127.27.124.1:35579/ using document root <none> and password file <none>
I20240502 13:59:34.690745 28144 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:34.690786 28144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:34.690852 28144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:34.691188 28144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root/instance:
uuid: "aed9691ece2047098365fb0a6f8ffdca"
format_stamp: "Formatted at 2024-05-02 13:59:34 on dist-test-slave-0cl8"
I20240502 13:59:34.692513 28144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240502 13:59:34.693384 28240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240502 13:59:34.693641 28144 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240502 13:59:34.693702 28144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root
uuid: "aed9691ece2047098365fb0a6f8ffdca"
format_stamp: "Formatted at 2024-05-02 13:59:34 on dist-test-slave-0cl8"
I20240502 13:59:34.693763 28144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:34.714882 28144 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240502 13:59:34.715173 28144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:34.715482 28144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:34.716132 28144 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:34.716181 28144 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240502 13:59:34.716214 28144 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:34.716243 28144 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240502 13:59:34.721554 28144 rpc_server.cc:307] RPC server started. Bound to: 127.27.124.1:38881
I20240502 13:59:34.721598 28303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.124.1:38881 every 8 connection(s)
I20240502 13:59:34.730860 28304 heartbeater.cc:347] Connected to a master server at 127.27.124.62:34509
I20240502 13:59:34.731077 28304 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:34.731475 28304 heartbeater.cc:511] Master 127.27.124.62:34509 requested a full tablet report, sending...
I20240502 13:59:34.732942 28174 ts_manager.cc:194] Registered new tserver with Master: aed9691ece2047098365fb0a6f8ffdca (127.27.124.1:38881)
I20240502 13:59:34.733150 28144 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.011116542s
I20240502 13:59:34.734386 28174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41384
I20240502 13:59:34.742252 28174 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41398:
name: "update-scan-delta-compact-tbl"
schema {
  columns {
    name: "key"
    type: INT64
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int64"
    type: INT64
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20240502 13:59:34.755977 28268 tablet_service.cc:1467] Processing CreateTablet for tablet 7d1c512d7ce24924b19841d706f5e126 (DEFAULT_TABLE table=update-scan-delta-compact-tbl [id=a6b4a408a71f4e1caa53492863a84ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:34.756302 28268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d1c512d7ce24924b19841d706f5e126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:34.758352 28316 tablet_bootstrap.cc:492] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: Bootstrap starting.
I20240502 13:59:34.759294 28316 tablet_bootstrap.cc:654] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:34.760198 28316 tablet_bootstrap.cc:492] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: No bootstrap required, opened a new log
I20240502 13:59:34.760350 28316 ts_tablet_manager.cc:1367] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240502 13:59:34.760779 28316 raft_consensus.cc:348] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9691ece2047098365fb0a6f8ffdca" member_type: VOTER last_known_addr { host: "127.27.124.1" port: 38881 } }
I20240502 13:59:34.760874 28316 raft_consensus.cc:374] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:34.760905 28316 raft_consensus.cc:729] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9691ece2047098365fb0a6f8ffdca, State: Initialized, Role: FOLLOWER
I20240502 13:59:34.761021 28316 consensus_queue.cc:260] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9691ece2047098365fb0a6f8ffdca" member_type: VOTER last_known_addr { host: "127.27.124.1" port: 38881 } }
I20240502 13:59:34.761099 28316 raft_consensus.cc:388] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:34.761142 28316 raft_consensus.cc:482] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:34.761188 28316 raft_consensus.cc:3037] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:34.761775 28316 raft_consensus.cc:504] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9691ece2047098365fb0a6f8ffdca" member_type: VOTER last_known_addr { host: "127.27.124.1" port: 38881 } }
I20240502 13:59:34.761890 28316 leader_election.cc:302] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aed9691ece2047098365fb0a6f8ffdca; no voters: 
I20240502 13:59:34.762046 28316 leader_election.cc:288] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240502 13:59:34.762156 28318 raft_consensus.cc:2781] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:34.762367 28318 raft_consensus.cc:686] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [term 1 LEADER]: Becoming Leader. State: Replica: aed9691ece2047098365fb0a6f8ffdca, State: Running, Role: LEADER
I20240502 13:59:34.762384 28316 ts_tablet_manager.cc:1398] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20240502 13:59:34.762513 28304 heartbeater.cc:503] Master 127.27.124.62:34509 was elected leader, sending a full tablet report...
I20240502 13:59:34.762531 28318 consensus_queue.cc:237] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9691ece2047098365fb0a6f8ffdca" member_type: VOTER last_known_addr { host: "127.27.124.1" port: 38881 } }
I20240502 13:59:34.765272 28173 catalog_manager.cc:5484] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca reported cstate change: term changed from 0 to 1, leader changed from <none> to aed9691ece2047098365fb0a6f8ffdca (127.27.124.1). New cstate: current_term: 1 leader_uuid: "aed9691ece2047098365fb0a6f8ffdca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed9691ece2047098365fb0a6f8ffdca" member_type: VOTER last_known_addr { host: "127.27.124.1" port: 38881 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:36.722420 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=1.000000
I20240502 13:59:37.565577 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.843s	user 0.822s	sys 0.016s Metrics: {"bytes_written":164229,"cfile_init":1,"compiler_manager_pool.queue_time_us":167,"compiler_manager_pool.run_cpu_time_us":218998,"compiler_manager_pool.run_wall_time_us":225207,"dirs.queue_time_us":180,"dirs.run_cpu_time_us":173,"dirs.run_wall_time_us":1070,"drs_written":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3334,"lbm_writes_lt_1ms":86,"peak_mem_usage":0,"rows_written":59001,"thread_start_us":171,"threads_started":2}
I20240502 13:59:37.566498 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 396221 bytes on disk
I20240502 13:59:37.567137 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20240502 13:59:39.567965 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=2.874778
I20240502 13:59:40.595379 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 1.027s	user 1.001s	sys 0.008s Metrics: {"bytes_written":208780,"cfile_init":1,"dirs.queue_time_us":179,"dirs.run_cpu_time_us":167,"dirs.run_wall_time_us":1636,"drs_written":1,"lbm_read_time_us":37,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4418,"lbm_writes_lt_1ms":109,"peak_mem_usage":0,"rows_written":78000,"thread_start_us":78,"threads_started":1}
I20240502 13:59:40.596251 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 523430 bytes on disk
I20240502 13:59:40.596719 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":73,"lbm_reads_lt_1ms":4}
I20240502 13:59:40.597126 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.014653
I20240502 13:59:41.869174 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 1.272s	user 1.264s	sys 0.008s Metrics: {"bytes_written":380035,"cfile_cache_hit":10,"cfile_cache_hit_bytes":1769,"cfile_cache_miss":166,"cfile_cache_miss_bytes":1913763,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":1009,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1722,"drs_written":1,"lbm_read_time_us":3739,"lbm_reads_lt_1ms":186,"lbm_write_time_us":6845,"lbm_writes_lt_1ms":178,"num_input_rowsets":2,"peak_mem_usage":1196984,"rows_written":137001,"spinlock_wait_cycles":5888,"thread_start_us":390,"threads_started":6}
I20240502 13:59:41.869910 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=1.000000
I20240502 13:59:42.811049 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.941s	user 0.899s	sys 0.008s Metrics: {"bytes_written":168319,"cfile_init":1,"dirs.queue_time_us":1037,"dirs.run_cpu_time_us":154,"dirs.run_wall_time_us":864,"drs_written":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3548,"lbm_writes_lt_1ms":89,"peak_mem_usage":0,"rows_written":62000,"thread_start_us":74,"threads_started":1}
I20240502 13:59:42.811592 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.014495
I20240502 13:59:44.682431 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 1.871s	user 1.848s	sys 0.004s Metrics: {"bytes_written":549525,"cfile_cache_hit":10,"cfile_cache_hit_bytes":2373,"cfile_cache_miss":236,"cfile_cache_miss_bytes":2790329,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":560,"dirs.run_cpu_time_us":155,"dirs.run_wall_time_us":1521,"drs_written":1,"lbm_read_time_us":5407,"lbm_reads_lt_1ms":264,"lbm_write_time_us":9001,"lbm_writes_lt_1ms":250,"num_input_rowsets":2,"peak_mem_usage":2188464,"rows_written":199001,"thread_start_us":385,"threads_started":6}
I20240502 13:59:44.683005 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 1336218 bytes on disk
I20240502 13:59:44.683400 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20240502 13:59:44.683748 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=1.882590
I20240502 13:59:45.671461 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.988s	user 0.984s	sys 0.000s Metrics: {"bytes_written":196112,"cfile_init":1,"dirs.queue_time_us":192,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":863,"drs_written":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3800,"lbm_writes_lt_1ms":103,"peak_mem_usage":0,"rows_written":72000,"thread_start_us":85,"threads_started":1}
I20240502 13:59:45.672020 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 483872 bytes on disk
I20240502 13:59:45.672817 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4}
I20240502 13:59:45.673302 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.014312
I20240502 13:59:48.175204 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 2.502s	user 2.489s	sys 0.012s Metrics: {"bytes_written":746824,"cfile_cache_hit":10,"cfile_cache_hit_bytes":3212,"cfile_cache_miss":324,"cfile_cache_miss_bytes":3806694,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":789,"dirs.run_cpu_time_us":157,"dirs.run_wall_time_us":1303,"drs_written":1,"lbm_read_time_us":6685,"lbm_reads_lt_1ms":344,"lbm_write_time_us":12075,"lbm_writes_lt_1ms":333,"num_input_rowsets":2,"peak_mem_usage":3195893,"rows_written":271001,"spinlock_wait_cycles":768,"thread_start_us":349,"threads_started":6}
I20240502 13:59:48.175793 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=5.851340
I20240502 13:59:49.429934 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 1.254s	user 1.240s	sys 0.012s Metrics: {"bytes_written":255947,"cfile_init":1,"dirs.queue_time_us":185,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":775,"drs_written":1,"lbm_read_time_us":35,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4922,"lbm_writes_lt_1ms":127,"peak_mem_usage":0,"rows_written":95000,"thread_start_us":89,"threads_started":1}
I20240502 13:59:49.430663 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.014072
I20240502 13:59:52.943513 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 3.513s	user 3.391s	sys 0.016s Metrics: {"bytes_written":1010115,"cfile_cache_hit":10,"cfile_cache_hit_bytes":4260,"cfile_cache_miss":428,"cfile_cache_miss_bytes":5137992,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":156,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":4386,"drs_written":1,"lbm_read_time_us":9130,"lbm_reads_lt_1ms":456,"lbm_write_time_us":17062,"lbm_writes_lt_1ms":443,"num_input_rowsets":2,"peak_mem_usage":4365812,"rows_written":366001,"thread_start_us":400,"threads_started":7}
I20240502 13:59:52.944113 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 2457562 bytes on disk
I20240502 13:59:52.944590 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":77,"lbm_reads_lt_1ms":4}
I20240502 13:59:52.945019 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=9.820090
I20240502 13:59:54.663923 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 1.719s	user 1.629s	sys 0.004s Metrics: {"bytes_written":350371,"cfile_init":1,"dirs.queue_time_us":186,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":28160,"drs_written":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6990,"lbm_writes_lt_1ms":168,"peak_mem_usage":0,"rows_written":130000,"thread_start_us":88,"threads_started":1}
I20240502 13:59:54.664680 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.013742
I20240502 13:59:59.329560 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 4.665s	user 4.627s	sys 0.034s Metrics: {"bytes_written":1367741,"cfile_cache_hit":10,"cfile_cache_hit_bytes":5623,"cfile_cache_miss":576,"cfile_cache_miss_bytes":6962905,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":960,"dirs.run_cpu_time_us":166,"dirs.run_wall_time_us":987,"drs_written":1,"lbm_read_time_us":11953,"lbm_reads_lt_1ms":600,"lbm_write_time_us":22031,"lbm_writes_lt_1ms":593,"mutex_wait_us":23,"num_input_rowsets":2,"peak_mem_usage":5909455,"rows_written":496001,"spinlock_wait_cycles":4096,"thread_start_us":385,"threads_started":7}
I20240502 13:59:59.330192 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=15.773215
I20240502 14:00:01.532200 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 2.202s	user 2.192s	sys 0.006s Metrics: {"bytes_written":454440,"cfile_init":1,"dirs.queue_time_us":224,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":25849,"drs_written":1,"lbm_read_time_us":39,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8293,"lbm_writes_lt_1ms":216,"peak_mem_usage":0,"rows_written":170000,"thread_start_us":76,"threads_started":1}
I20240502 14:00:01.532742 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 4474838 bytes on disk
I20240502 14:00:01.533224 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":78,"lbm_reads_lt_1ms":8}
I20240502 14:00:01.533622 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.013311
I20240502 14:00:07.687822 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 6.154s	user 6.127s	sys 0.016s Metrics: {"bytes_written":1833811,"cfile_cache_hit":10,"cfile_cache_hit_bytes":7574,"cfile_cache_miss":774,"cfile_cache_miss_bytes":9359210,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":1623,"dirs.run_cpu_time_us":164,"dirs.run_wall_time_us":755,"drs_written":1,"lbm_read_time_us":15781,"lbm_reads_lt_1ms":794,"lbm_write_time_us":29730,"lbm_writes_lt_1ms":790,"mutex_wait_us":254,"num_input_rowsets":2,"peak_mem_usage":8021811,"rows_written":666001,"thread_start_us":441,"threads_started":7}
I20240502 14:00:07.688548 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=23.710715
I20240502 14:00:10.636221 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 2.948s	user 2.927s	sys 0.012s Metrics: {"bytes_written":611230,"cfile_init":1,"dirs.queue_time_us":164,"dirs.run_cpu_time_us":152,"dirs.run_wall_time_us":721,"drs_written":1,"lbm_read_time_us":35,"lbm_reads_lt_1ms":4,"lbm_write_time_us":11246,"lbm_writes_lt_1ms":282,"peak_mem_usage":0,"rows_written":229000,"thread_start_us":75,"threads_started":1}
I20240502 14:00:10.636817 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling LogGCOp(7d1c512d7ce24924b19841d706f5e126): free 8349227 bytes of WAL
I20240502 14:00:10.637048 28245 log_reader.cc:386] T 7d1c512d7ce24924b19841d706f5e126: removed 1 log segments from log reader
I20240502 14:00:10.637118 28245 log.cc:1076] T 7d1c512d7ce24924b19841d706f5e126 P aed9691ece2047098365fb0a6f8ffdca: Deleting log segment in path: /tmp/dist-test-taskDli4OH/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714658374549254-28144-0/minicluster-data/ts-0-root/wals/7d1c512d7ce24924b19841d706f5e126/wal-000000001 (ops 1-863)
I20240502 14:00:10.638744 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: LogGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {}
I20240502 14:00:10.639061 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 6011982 bytes on disk
I20240502 14:00:10.639600 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":120,"lbm_reads_lt_1ms":8}
I20240502 14:00:10.640034 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling FlushMRSOp(7d1c512d7ce24924b19841d706f5e126): perf score=2.874778
I20240502 14:00:11.616075 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: FlushMRSOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.976s	user 0.960s	sys 0.004s Metrics: {"bytes_written":200345,"cfile_init":1,"dirs.queue_time_us":179,"dirs.run_cpu_time_us":162,"dirs.run_wall_time_us":780,"drs_written":1,"lbm_read_time_us":37,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3846,"lbm_writes_lt_1ms":106,"peak_mem_usage":0,"rows_written":75000,"thread_start_us":81,"threads_started":1}
I20240502 14:00:11.616699 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126): 503913 bytes on disk
I20240502 14:00:11.617079 28245 maintenance_manager.cc:643] P aed9691ece2047098365fb0a6f8ffdca: UndoDeltaBlockGCOp(7d1c512d7ce24924b19841d706f5e126) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20240502 14:00:11.617501 28305 maintenance_manager.cc:419] P aed9691ece2047098365fb0a6f8ffdca: Scheduling CompactRowSetsOp(7d1c512d7ce24924b19841d706f5e126): perf score=0.021543
I20240502 14:00:11.782109 28144 update_scan_delta_compact-test.cc:209] Time spent Insert: real 37.011s	user 35.148s	sys 1.495s
I20240502 14:00:11.784691 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.787355 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.788863 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.790349 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.792294 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.794183 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.795945 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.797706 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.799468 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.801265 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.803036 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.804813 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.806582 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.808337 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.810103 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.811861 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.813566 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.815297 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.817036 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.818807 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240502 14:00:11.820596 28378 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
F20240502 14:00:11.821511 28378 monotime.cc:342] Check failed: t_beg.Initialized() 
*** Check failure stack trace: ***
*** Aborted at 1714658411 (unix time) try "date -d @1714658411" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e800006df0) received by PID 28144 (TID 0x7fb8ff5b1700) from PID 28144; stack trace: ***
    @     0x7fb92baab980 (unknown) at ??:0
    @     0x7fb929b0bfb7 gsignal at ??:0
    @     0x7fb929b0d921 abort at ??:0
    @     0x7fb92a67adcd google::LogMessage::Fail() at ??:0
    @     0x7fb92a67eb93 google::LogMessage::SendToLog() at ??:0
    @     0x7fb92a67a7cc google::LogMessage::Flush() at ??:0
    @     0x7fb92a67bf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7fb92b36f5b1 kudu::operator-() at ??:0
    @     0x7fb92c21afd2 kudu::tablet::WriteOp::ToString() at ??:0
    @     0x7fb92c20405b kudu::tablet::OpDriver::ToStringUnlocked() at ??:0
    @     0x7fb92c203f90 kudu::tablet::OpDriver::ToString() at ??:0
    @     0x7fb92c1da9bd kudu::tablet::TabletReplica::GetInFlightOps() at ??:0
    @     0x7fb92ccba861 kudu::tserver::TabletServerPathHandlers::HandleTransactionsPage() at ??:0
    @     0x7fb92ccb993a _ZZN4kudu7tserver24TabletServerPathHandlers8RegisterEPNS_9WebserverEENKUlRKNS_19WebCallbackRegistry10WebRequestEPNS4_22PrerenderedWebResponseEE2_clES7_S9_ at ??:0
    @     0x7fb92ccc2149 _ZNSt17_Function_handlerIFvRKN4kudu19WebCallbackRegistry10WebRequestEPNS1_22PrerenderedWebResponseEEZNS0_7tserver24TabletServerPathHandlers8RegisterEPNS0_9WebserverEEUlS4_S6_E2_E9_M_invokeERKSt9_Any_dataS4_OS6_ at ??:0
    @     0x7fb929293d09 std::function<>::operator()() at ??:0
    @     0x7fb92928e143 kudu::Webserver::RunPathHandler() at ??:0
    @     0x7fb92928da38 kudu::Webserver::BeginRequestCallback() at ??:0
    @     0x7fb92928d123 kudu::Webserver::BeginRequestCallbackStatic() at ??:0
    @     0x7fb9292b5b89 handle_request at ??:0
    @     0x7fb9292b826b worker_thread at ??:0
    @     0x7fb92baa06db start_thread at ??:0
    @     0x7fb929bee71f clone at ??:0