Diagnosed failure

UpdateScanDeltaCompactionTest.TestAll: Unrecognized error type. Please see the error log for more information.

Full log

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from UpdateScanDeltaCompactionTest
[ RUN      ] UpdateScanDeltaCompactionTest.TestAll
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240426 09:58:46.685107 26913 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.72.126:34931
I20240426 09:58:46.686760 26913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:58:46.688171 26913 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:58:46.700559 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:58:46.700973 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:58:46.701617 26921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:58:46.702558 26913 server_base.cc:1034] running on GCE node
I20240426 09:58:46.703543 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 09:58:46.703727 26913 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 09:58:46.703827 26913 hybrid_clock.cc:648] HybridClock initialized: now 1714125526703803 us; error 0 us; skew 500 ppm
I20240426 09:58:46.704200 26913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240426 09:58:46.713240 26913 webserver.cc:460] Webserver started at http://127.26.72.126:42547/ using document root <none> and password file <none>
I20240426 09:58:46.714519 26913 fs_manager.cc:359] Metadata directory not provided
I20240426 09:58:46.714716 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:58:46.715292 26913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:58:46.720989 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/master-0-root/instance:
uuid: "15a4ffb7397b40a19649c2e00c5cf6dd"
format_stamp: "Formatted at 2024-04-26 09:58:46 on dist-test-slave-0k0w"
I20240426 09:58:46.730407 26913 fs_manager.cc:691] Time spent creating directory manager: real 0.008s	user 0.000s	sys 0.007s
I20240426 09:58:46.734681 26926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 09:58:46.736903 26913 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20240426 09:58:46.737200 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/master-0-root
uuid: "15a4ffb7397b40a19649c2e00c5cf6dd"
format_stamp: "Formatted at 2024-04-26 09:58:46 on dist-test-slave-0k0w"
I20240426 09:58:46.737424 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:58:46.763908 26913 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 09:58:46.764772 26913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:58:46.765048 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:58:46.785148 26913 rpc_server.cc:307] RPC server started. Bound to: 127.26.72.126:34931
I20240426 09:58:46.785168 26977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.72.126:34931 every 8 connection(s)
I20240426 09:58:46.789209 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:58:46.806344 26978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd: Bootstrap starting.
I20240426 09:58:46.810207 26978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd: Neither blocks nor log segments found. Creating new log.
I20240426 09:58:46.811414 26978 log.cc:824] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd: Log is configured to *not* fsync() on all Append() calls
I20240426 09:58:46.814117 26978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd: No bootstrap required, opened a new log
I20240426 09:58:46.821501 26978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER }
I20240426 09:58:46.821873 26978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:58:46.821988 26978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a4ffb7397b40a19649c2e00c5cf6dd, State: Initialized, Role: FOLLOWER
I20240426 09:58:46.822746 26978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER }
I20240426 09:58:46.822968 26978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:58:46.823099 26978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:58:46.823279 26978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:58:46.825448 26978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER }
I20240426 09:58:46.825986 26978 leader_election.cc:302] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a4ffb7397b40a19649c2e00c5cf6dd; no voters: 
I20240426 09:58:46.826651 26978 leader_election.cc:288] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 09:58:46.826792 26981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:58:46.827455 26981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [term 1 LEADER]: Becoming Leader. State: Replica: 15a4ffb7397b40a19649c2e00c5cf6dd, State: Running, Role: LEADER
I20240426 09:58:46.828040 26981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER }
I20240426 09:58:46.828831 26978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:58:46.831005 26982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER } }
I20240426 09:58:46.831089 26983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a4ffb7397b40a19649c2e00c5cf6dd. Latest consensus state: current_term: 1 leader_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a4ffb7397b40a19649c2e00c5cf6dd" member_type: VOTER } }
I20240426 09:58:46.831350 26982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [sys.catalog]: This master's current role is: LEADER
I20240426 09:58:46.831487 26983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd [sys.catalog]: This master's current role is: LEADER
I20240426 09:58:46.839358 26988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:58:46.845561 26988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:58:46.846222 26913 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 09:58:46.856329 26988 catalog_manager.cc:1347] Generated new cluster ID: b44a65bea5b94dd788cffcd8ea74fa99
I20240426 09:58:46.856482 26988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:58:46.879369 26988 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:58:46.880683 26988 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:58:46.892673 26988 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 15a4ffb7397b40a19649c2e00c5cf6dd: Generated new TSK 0
I20240426 09:58:46.893638 26988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:58:46.912128 26913 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:58:46.917197 27001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:58:46.917742 27000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:58:46.918431 27003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:58:46.918897 26913 server_base.cc:1034] running on GCE node
I20240426 09:58:46.919198 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 09:58:46.919315 26913 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 09:58:46.919397 26913 hybrid_clock.cc:648] HybridClock initialized: now 1714125526919363 us; error 0 us; skew 500 ppm
I20240426 09:58:46.919601 26913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240426 09:58:46.921079 26913 webserver.cc:460] Webserver started at http://127.26.72.65:41403/ using document root <none> and password file <none>
I20240426 09:58:46.921365 26913 fs_manager.cc:359] Metadata directory not provided
I20240426 09:58:46.921463 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:58:46.921618 26913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:58:46.922324 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root/instance:
uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd"
format_stamp: "Formatted at 2024-04-26 09:58:46 on dist-test-slave-0k0w"
I20240426 09:58:46.925252 26913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20240426 09:58:46.927151 27008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 09:58:46.927635 26913 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 09:58:46.927744 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root
uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd"
format_stamp: "Formatted at 2024-04-26 09:58:46 on dist-test-slave-0k0w"
I20240426 09:58:46.927894 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:58:46.950353 26913 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 09:58:46.951090 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:58:46.952142 26913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:58:46.953749 26913 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:58:46.953869 26913 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 09:58:46.954003 26913 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:58:46.954087 26913 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 09:58:46.975565 26913 rpc_server.cc:307] RPC server started. Bound to: 127.26.72.65:43755
I20240426 09:58:46.975597 27070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.72.65:43755 every 8 connection(s)
I20240426 09:58:46.991039 27071 heartbeater.cc:347] Connected to a master server at 127.26.72.126:34931
I20240426 09:58:46.991382 27071 heartbeater.cc:464] Registering TS with master...
I20240426 09:58:46.992141 27071 heartbeater.cc:511] Master 127.26.72.126:34931 requested a full tablet report, sending...
I20240426 09:58:46.994516 26943 ts_manager.cc:194] Registered new tserver with Master: 02ac1e8dbbc941a29e6445ecdc2e73cd (127.26.72.65:43755)
I20240426 09:58:46.994874 26913 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.017837902s
I20240426 09:58:46.997267 26943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59230
I20240426 09:58:47.016157 26943 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59246:
name: "update-scan-delta-compact-tbl"
schema {
  columns {
    name: "key"
    type: INT64
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int64"
    type: INT64
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20240426 09:58:47.049147 27036 tablet_service.cc:1467] Processing CreateTablet for tablet 9d8fcef8d3b7470dbe8b3c3063e55f65 (DEFAULT_TABLE table=update-scan-delta-compact-tbl [id=930e6326f53544619e61b283ad532dda]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:58:47.050097 27036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d8fcef8d3b7470dbe8b3c3063e55f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:58:47.058404 27083 tablet_bootstrap.cc:492] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: Bootstrap starting.
I20240426 09:58:47.060622 27083 tablet_bootstrap.cc:654] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: Neither blocks nor log segments found. Creating new log.
I20240426 09:58:47.062661 27083 tablet_bootstrap.cc:492] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: No bootstrap required, opened a new log
I20240426 09:58:47.062844 27083 ts_tablet_manager.cc:1367] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20240426 09:58:47.063669 27083 raft_consensus.cc:348] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" member_type: VOTER last_known_addr { host: "127.26.72.65" port: 43755 } }
I20240426 09:58:47.063894 27083 raft_consensus.cc:374] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:58:47.064013 27083 raft_consensus.cc:729] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ac1e8dbbc941a29e6445ecdc2e73cd, State: Initialized, Role: FOLLOWER
I20240426 09:58:47.064380 27083 consensus_queue.cc:260] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" member_type: VOTER last_known_addr { host: "127.26.72.65" port: 43755 } }
I20240426 09:58:47.064591 27083 raft_consensus.cc:388] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:58:47.064690 27083 raft_consensus.cc:482] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:58:47.064831 27083 raft_consensus.cc:3037] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:58:47.067335 27083 raft_consensus.cc:504] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" member_type: VOTER last_known_addr { host: "127.26.72.65" port: 43755 } }
I20240426 09:58:47.067596 27083 leader_election.cc:302] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02ac1e8dbbc941a29e6445ecdc2e73cd; no voters: 
I20240426 09:58:47.068142 27083 leader_election.cc:288] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 09:58:47.068334 27085 raft_consensus.cc:2781] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:58:47.069121 27083 ts_tablet_manager.cc:1398] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20240426 09:58:47.069370 27071 heartbeater.cc:503] Master 127.26.72.126:34931 was elected leader, sending a full tablet report...
I20240426 09:58:47.069856 27085 raft_consensus.cc:686] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [term 1 LEADER]: Becoming Leader. State: Replica: 02ac1e8dbbc941a29e6445ecdc2e73cd, State: Running, Role: LEADER
I20240426 09:58:47.070171 27085 consensus_queue.cc:237] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" member_type: VOTER last_known_addr { host: "127.26.72.65" port: 43755 } }
I20240426 09:58:47.074476 26943 catalog_manager.cc:5484] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 02ac1e8dbbc941a29e6445ecdc2e73cd (127.26.72.65). New cstate: current_term: 1 leader_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ac1e8dbbc941a29e6445ecdc2e73cd" member_type: VOTER last_known_addr { host: "127.26.72.65" port: 43755 } health_report { overall_health: HEALTHY } } }
I20240426 09:58:50.978092 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=1.882590
I20240426 09:58:53.646416 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.668s	user 2.330s	sys 0.297s Metrics: {"bytes_written":187587,"cfile_init":1,"compiler_manager_pool.queue_time_us":358,"compiler_manager_pool.run_cpu_time_us":469718,"compiler_manager_pool.run_wall_time_us":470497,"dirs.queue_time_us":409,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1190,"drs_written":1,"lbm_read_time_us":95,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8056,"lbm_writes_lt_1ms":98,"peak_mem_usage":0,"rows_written":67001,"thread_start_us":539,"threads_started":2}
I20240426 09:58:53.648869 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 450427 bytes on disk
I20240426 09:58:53.649909 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":135,"lbm_reads_lt_1ms":4}
I20240426 09:58:55.651005 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=2.874778
I20240426 09:58:57.872298 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.221s	user 2.131s	sys 0.088s Metrics: {"bytes_written":208808,"cfile_init":1,"dirs.queue_time_us":431,"dirs.run_cpu_time_us":274,"dirs.run_wall_time_us":1590,"drs_written":1,"lbm_read_time_us":82,"lbm_reads_lt_1ms":4,"lbm_write_time_us":9022,"lbm_writes_lt_1ms":109,"peak_mem_usage":0,"rows_written":78000,"thread_start_us":295,"threads_started":1}
I20240426 09:58:57.874018 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 523410 bytes on disk
I20240426 09:58:57.874672 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20240426 09:58:57.875491 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=0.014632
I20240426 09:59:00.182520 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.307s	user 2.216s	sys 0.076s Metrics: {"bytes_written":399217,"cfile_cache_hit":10,"cfile_cache_hit_bytes":1895,"cfile_cache_miss":180,"cfile_cache_miss_bytes":2027505,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":2207,"dirs.run_cpu_time_us":254,"dirs.run_wall_time_us":2165,"drs_written":1,"lbm_read_time_us":9601,"lbm_reads_lt_1ms":200,"lbm_write_time_us":14106,"lbm_writes_lt_1ms":186,"num_input_rowsets":2,"peak_mem_usage":1196984,"rows_written":145001,"thread_start_us":1469,"threads_started":6}
I20240426 09:59:00.183487 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 973141 bytes on disk
I20240426 09:59:00.184113 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4}
I20240426 09:59:00.184605 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=1.882590
I20240426 09:59:02.574764 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.390s	user 2.226s	sys 0.112s Metrics: {"bytes_written":196030,"cfile_init":1,"dirs.queue_time_us":427,"dirs.run_cpu_time_us":171,"dirs.run_wall_time_us":860,"drs_written":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8792,"lbm_writes_lt_1ms":103,"peak_mem_usage":0,"rows_written":72000,"thread_start_us":296,"threads_started":1}
I20240426 09:59:02.575774 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 483894 bytes on disk
I20240426 09:59:02.576313 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":81,"lbm_reads_lt_1ms":4}
I20240426 09:59:02.576851 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=0.014450
I20240426 09:59:05.953507 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 3.376s	user 3.259s	sys 0.108s Metrics: {"bytes_written":596751,"cfile_cache_hit":10,"cfile_cache_hit_bytes":2604,"cfile_cache_miss":260,"cfile_cache_miss_bytes":3044414,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":2632,"dirs.run_cpu_time_us":240,"dirs.run_wall_time_us":1905,"drs_written":1,"lbm_read_time_us":12520,"lbm_reads_lt_1ms":280,"lbm_write_time_us":21290,"lbm_writes_lt_1ms":269,"num_input_rowsets":2,"peak_mem_usage":2318455,"rows_written":217001,"thread_start_us":1792,"threads_started":7}
I20240426 09:59:05.954386 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=4.859153
I20240426 09:59:08.541504 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.587s	user 2.484s	sys 0.060s Metrics: {"bytes_written":240646,"cfile_init":1,"dirs.queue_time_us":425,"dirs.run_cpu_time_us":174,"dirs.run_wall_time_us":792,"drs_written":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10223,"lbm_writes_lt_1ms":121,"peak_mem_usage":0,"rows_written":89000,"spinlock_wait_cycles":1280,"thread_start_us":306,"threads_started":1}
I20240426 09:59:08.542486 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 2050670 bytes on disk
I20240426 09:59:08.543501 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.001s	user 0.001s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":226,"lbm_reads_lt_1ms":8}
I20240426 09:59:08.544175 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=0.014225
I20240426 09:59:13.403959 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 4.860s	user 4.671s	sys 0.188s Metrics: {"bytes_written":841110,"cfile_cache_hit":10,"cfile_cache_hit_bytes":3585,"cfile_cache_miss":358,"cfile_cache_miss_bytes":4286037,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":2952,"dirs.run_cpu_time_us":263,"dirs.run_wall_time_us":1914,"drs_written":1,"lbm_read_time_us":16885,"lbm_reads_lt_1ms":378,"lbm_write_time_us":28346,"lbm_writes_lt_1ms":372,"num_input_rowsets":2,"peak_mem_usage":3488374,"rows_written":306001,"thread_start_us":2328,"threads_started":7}
I20240426 09:59:13.404903 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 2056537 bytes on disk
I20240426 09:59:13.405689 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":267,"lbm_reads_lt_1ms":4}
I20240426 09:59:13.406345 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=8.827903
I20240426 09:59:16.982146 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 3.576s	user 3.377s	sys 0.176s Metrics: {"bytes_written":318988,"cfile_init":1,"dirs.queue_time_us":405,"dirs.run_cpu_time_us":159,"dirs.run_wall_time_us":855,"drs_written":1,"lbm_read_time_us":79,"lbm_reads_lt_1ms":4,"lbm_write_time_us":13802,"lbm_writes_lt_1ms":155,"peak_mem_usage":0,"rows_written":118000,"thread_start_us":278,"threads_started":1}
I20240426 09:59:16.983155 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 790184 bytes on disk
I20240426 09:59:16.983752 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4}
I20240426 09:59:16.984302 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=0.013925
I20240426 09:59:23.723470 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 6.739s	user 6.553s	sys 0.160s Metrics: {"bytes_written":1165911,"cfile_cache_hit":10,"cfile_cache_hit_bytes":4939,"cfile_cache_miss":494,"cfile_cache_miss_bytes":5952933,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":3756,"dirs.run_cpu_time_us":220,"dirs.run_wall_time_us":1600,"drs_written":1,"lbm_read_time_us":23856,"lbm_reads_lt_1ms":514,"lbm_write_time_us":39969,"lbm_writes_lt_1ms":508,"num_input_rowsets":2,"peak_mem_usage":4934525,"rows_written":424001,"spinlock_wait_cycles":2304,"thread_start_us":3021,"threads_started":7}
I20240426 09:59:23.724363 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 2850027 bytes on disk
I20240426 09:59:23.724853 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 09:59:23.725364 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=14.781028
I20240426 09:59:28.504303 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 4.779s	user 4.527s	sys 0.202s Metrics: {"bytes_written":436480,"cfile_init":1,"dirs.queue_time_us":389,"dirs.run_cpu_time_us":76,"dirs.run_wall_time_us":77,"drs_written":1,"lbm_read_time_us":86,"lbm_reads_lt_1ms":4,"lbm_write_time_us":18218,"lbm_writes_lt_1ms":206,"peak_mem_usage":0,"rows_written":163000,"thread_start_us":273,"threads_started":1}
I20240426 09:59:28.505195 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 1095173 bytes on disk
I20240426 09:59:28.505720 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":76,"lbm_reads_lt_1ms":4}
I20240426 09:59:28.506300 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=2.874778
I20240426 09:59:30.738346 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 2.232s	user 2.176s	sys 0.032s Metrics: {"bytes_written":200264,"cfile_init":1,"dirs.queue_time_us":398,"dirs.run_cpu_time_us":179,"dirs.run_wall_time_us":970,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8720,"lbm_writes_lt_1ms":106,"peak_mem_usage":0,"rows_written":75000,"thread_start_us":275,"threads_started":1}
I20240426 09:59:30.739646 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=0.022323
I20240426 09:59:41.230736 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: CompactRowSetsOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 10.491s	user 10.225s	sys 0.220s Metrics: {"bytes_written":1824211,"cfile_cache_hit":15,"cfile_cache_hit_bytes":7621,"cfile_cache_miss":771,"cfile_cache_miss_bytes":9292619,"cfile_init":8,"delta_iterators_relevant":6,"dirs.queue_time_us":4542,"dirs.run_cpu_time_us":1162,"dirs.run_wall_time_us":6464,"drs_written":1,"lbm_read_time_us":36827,"lbm_reads_lt_1ms":803,"lbm_write_time_us":65724,"lbm_writes_lt_1ms":786,"num_input_rowsets":3,"peak_mem_usage":6884670,"rows_written":662001,"spinlock_wait_cycles":1408,"thread_start_us":4853,"threads_started":7}
I20240426 09:59:41.231725 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=20.734153
I20240426 09:59:47.274950 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 6.043s	user 5.864s	sys 0.148s Metrics: {"bytes_written":554263,"cfile_init":1,"dirs.queue_time_us":423,"dirs.run_cpu_time_us":177,"dirs.run_wall_time_us":1194,"drs_written":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4,"lbm_write_time_us":24027,"lbm_writes_lt_1ms":259,"peak_mem_usage":0,"rows_written":207000,"thread_start_us":300,"threads_started":1}
I20240426 09:59:47.276058 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling LogGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): free 8349123 bytes of WAL
I20240426 09:59:47.276446 27013 log_reader.cc:386] T 9d8fcef8d3b7470dbe8b3c3063e55f65: removed 1 log segments from log reader
I20240426 09:59:47.276569 27013 log.cc:1076] T 9d8fcef8d3b7470dbe8b3c3063e55f65 P 02ac1e8dbbc941a29e6445ecdc2e73cd: Deleting log segment in path: /tmp/dist-test-taskdul5gL/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714125526610473-26913-0/minicluster-data/ts-0-root/wals/9d8fcef8d3b7470dbe8b3c3063e55f65/wal-000000001 (ops 1-863)
I20240426 09:59:47.278930 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: LogGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.003s	user 0.000s	sys 0.000s Metrics: {}
I20240426 09:59:47.279408 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65): 5838363 bytes on disk
I20240426 09:59:47.280190 27013 maintenance_manager.cc:643] P 02ac1e8dbbc941a29e6445ecdc2e73cd: UndoDeltaBlockGCOp(9d8fcef8d3b7470dbe8b3c3063e55f65) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":174,"lbm_reads_lt_1ms":8}
I20240426 09:59:47.280921 27072 maintenance_manager.cc:419] P 02ac1e8dbbc941a29e6445ecdc2e73cd: Scheduling FlushMRSOp(9d8fcef8d3b7470dbe8b3c3063e55f65): perf score=4.859153
I20240426 09:59:49.882169 26913 update_scan_delta_compact-test.cc:209] Time spent Insert: real 62.797s	user 57.674s	sys 4.426s
I20240426 09:59:49.925643 27140 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 09:59:49.930961 27140 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 09:59:49.934541 27140 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240426 09:59:49.940758 27140 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
F20240426 09:59:49.942409 27140 monotime.cc:342] Check failed: t_beg.Initialized() 
*** Check failure stack trace: ***
*** Aborted at 1714125589 (unix time) try "date -d @1714125589" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e800006921) received by PID 26913 (TID 0x7fc29bd58700) from PID 26913; stack trace: ***
    @     0x7fc2cbb47980 (unknown) at ??:0
    @     0x7fc2c4bf7fb7 gsignal at ??:0
    @     0x7fc2c4bf9921 abort at ??:0
    @     0x7fc2c791fdbd google::LogMessage::Fail() at ??:0
    @     0x7fc2c7923b83 google::LogMessage::SendToLog() at ??:0
    @     0x7fc2c791f7bc google::LogMessage::Flush() at ??:0
    @     0x7fc2c7920f49 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7fc2c8e83880 kudu::operator-() at ??:0
    @     0x7fc2d469839a kudu::tablet::WriteOp::ToString() at ??:0
    @     0x7fc2d466282e kudu::tablet::OpDriver::ToStringUnlocked() at ??:0
    @     0x7fc2d4662586 kudu::tablet::OpDriver::ToString() at ??:0
    @     0x7fc2d460f529 kudu::tablet::TabletReplica::GetInFlightOps() at ??:0
    @     0x7fc2db4870a7 kudu::tserver::TabletServerPathHandlers::HandleTransactionsPage() at ??:0
    @     0x7fc2db495c5c kudu::tserver::TabletServerPathHandlers::Register()::$_3::operator()() at ??:0
    @     0x7fc2db495ac2 std::_Function_handler<>::_M_invoke() at ??:0
    @     0x7fc2d17fe6aa std::function<>::operator()() at ??:0
    @     0x7fc2d17f6e3a kudu::Webserver::RunPathHandler() at ??:0
    @     0x7fc2d17f1e71 kudu::Webserver::BeginRequestCallback() at ??:0
    @     0x7fc2d17effa7 kudu::Webserver::BeginRequestCallbackStatic() at ??:0
    @     0x7fc2d182d709 handle_request at ??:0
    @     0x7fc2d182fdeb worker_thread at ??:0
    @     0x7fc2cbb3c6db start_thread at ??:0
    @     0x7fc2c4cda71f clone at ??:0