Diagnosed failure

UpdateScanDeltaCompactionTest.TestAll: Unrecognized error type. Please see the error log for more information.

Full log

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from UpdateScanDeltaCompactionTest
[ RUN      ] UpdateScanDeltaCompactionTest.TestAll
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240430 20:15:17.627529 17037 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.16.163.126:46649
I20240430 20:15:17.629329 17037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240430 20:15:17.630748 17037 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240430 20:15:17.643529 17045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240430 20:15:17.644497 17043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240430 20:15:17.645046 17042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240430 20:15:17.645834 17037 server_base.cc:1034] running on GCE node
I20240430 20:15:17.646765 17037 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240430 20:15:17.646958 17037 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240430 20:15:17.647050 17037 hybrid_clock.cc:648] HybridClock initialized: now 1714508117647026 us; error 0 us; skew 500 ppm
I20240430 20:15:17.647449 17037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240430 20:15:17.650440 17037 webserver.cc:460] Webserver started at http://127.16.163.126:41089/ using document root <none> and password file <none>
I20240430 20:15:17.651296 17037 fs_manager.cc:359] Metadata directory not provided
I20240430 20:15:17.651422 17037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240430 20:15:17.651808 17037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240430 20:15:17.655391 17037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/master-0-root/instance:
uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991"
format_stamp: "Formatted at 2024-04-30 20:15:17 on dist-test-slave-8dnq"
I20240430 20:15:17.661856 17037 fs_manager.cc:691] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.002s
I20240430 20:15:17.665637 17050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240430 20:15:17.667522 17037 fs_manager.cc:725] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20240430 20:15:17.667735 17037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/master-0-root
uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991"
format_stamp: "Formatted at 2024-04-30 20:15:17 on dist-test-slave-8dnq"
I20240430 20:15:17.667954 17037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240430 20:15:17.691080 17037 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240430 20:15:17.692387 17037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240430 20:15:17.692746 17037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240430 20:15:17.718017 17037 rpc_server.cc:307] RPC server started. Bound to: 127.16.163.126:46649
I20240430 20:15:17.718032 17101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.163.126:46649 every 8 connection(s)
I20240430 20:15:17.721905 17102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240430 20:15:17.734985 17102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991: Bootstrap starting.
I20240430 20:15:17.738720 17102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991: Neither blocks nor log segments found. Creating new log.
I20240430 20:15:17.739825 17102 log.cc:824] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991: Log is configured to *not* fsync() on all Append() calls
I20240430 20:15:17.742599 17102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991: No bootstrap required, opened a new log
I20240430 20:15:17.749744 17102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER }
I20240430 20:15:17.750088 17102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240430 20:15:17.750177 17102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ab3a0b70b8848f298c0cdd3fe0c1991, State: Initialized, Role: FOLLOWER
I20240430 20:15:17.750882 17102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER }
I20240430 20:15:17.751091 17102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240430 20:15:17.751197 17102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240430 20:15:17.751340 17102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 0 FOLLOWER]: Advancing to term 1
I20240430 20:15:17.753393 17102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER }
I20240430 20:15:17.753926 17102 leader_election.cc:302] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ab3a0b70b8848f298c0cdd3fe0c1991; no voters: 
I20240430 20:15:17.754578 17102 leader_election.cc:288] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240430 20:15:17.754730 17105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 1 FOLLOWER]: Leader election won for term 1
I20240430 20:15:17.755549 17105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [term 1 LEADER]: Becoming Leader. State: Replica: 6ab3a0b70b8848f298c0cdd3fe0c1991, State: Running, Role: LEADER
I20240430 20:15:17.756182 17105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER }
I20240430 20:15:17.757089 17102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [sys.catalog]: configured and running, proceeding with master startup.
I20240430 20:15:17.759444 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER } }
I20240430 20:15:17.759707 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [sys.catalog]: This master's current role is: LEADER
I20240430 20:15:17.760425 17107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ab3a0b70b8848f298c0cdd3fe0c1991. Latest consensus state: current_term: 1 leader_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ab3a0b70b8848f298c0cdd3fe0c1991" member_type: VOTER } }
I20240430 20:15:17.760641 17112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240430 20:15:17.760717 17107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991 [sys.catalog]: This master's current role is: LEADER
I20240430 20:15:17.767041 17112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240430 20:15:17.769167 17037 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240430 20:15:17.777529 17112 catalog_manager.cc:1347] Generated new cluster ID: 82369d27ae0d4f63948bfce4a1e569b1
I20240430 20:15:17.777673 17112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240430 20:15:17.805507 17112 catalog_manager.cc:1370] Generated new certificate authority record
I20240430 20:15:17.807451 17112 catalog_manager.cc:1504] Loading token signing keys...
I20240430 20:15:17.820083 17112 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 6ab3a0b70b8848f298c0cdd3fe0c1991: Generated new TSK 0
I20240430 20:15:17.821445 17112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240430 20:15:17.834867 17037 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240430 20:15:17.841305 17124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240430 20:15:17.842203 17125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240430 20:15:17.843170 17127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240430 20:15:17.845211 17037 server_base.cc:1034] running on GCE node
I20240430 20:15:17.845633 17037 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240430 20:15:17.845749 17037 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240430 20:15:17.845817 17037 hybrid_clock.cc:648] HybridClock initialized: now 1714508117845794 us; error 0 us; skew 500 ppm
I20240430 20:15:17.846046 17037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240430 20:15:17.848062 17037 webserver.cc:460] Webserver started at http://127.16.163.65:37537/ using document root <none> and password file <none>
I20240430 20:15:17.848441 17037 fs_manager.cc:359] Metadata directory not provided
I20240430 20:15:17.848562 17037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240430 20:15:17.848734 17037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240430 20:15:17.849704 17037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root/instance:
uuid: "d9e2129f8e784c35a4ebe88f16148a6a"
format_stamp: "Formatted at 2024-04-30 20:15:17 on dist-test-slave-8dnq"
I20240430 20:15:17.853605 17037 fs_manager.cc:691] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20240430 20:15:17.856122 17132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240430 20:15:17.856631 17037 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.002s	sys 0.001s
I20240430 20:15:17.856786 17037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root
uuid: "d9e2129f8e784c35a4ebe88f16148a6a"
format_stamp: "Formatted at 2024-04-30 20:15:17 on dist-test-slave-8dnq"
I20240430 20:15:17.856948 17037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240430 20:15:17.881654 17037 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240430 20:15:17.882596 17037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240430 20:15:17.883667 17037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240430 20:15:17.885843 17037 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240430 20:15:17.886013 17037 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240430 20:15:17.886135 17037 ts_tablet_manager.cc:590] Registered 0 tablets
I20240430 20:15:17.886226 17037 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240430 20:15:17.908054 17194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.163.65:40551 every 8 connection(s)
I20240430 20:15:17.908056 17037 rpc_server.cc:307] RPC server started. Bound to: 127.16.163.65:40551
I20240430 20:15:17.928131 17195 heartbeater.cc:347] Connected to a master server at 127.16.163.126:46649
I20240430 20:15:17.928555 17195 heartbeater.cc:464] Registering TS with master...
I20240430 20:15:17.929397 17195 heartbeater.cc:511] Master 127.16.163.126:46649 requested a full tablet report, sending...
I20240430 20:15:17.932049 17067 ts_manager.cc:194] Registered new tserver with Master: d9e2129f8e784c35a4ebe88f16148a6a (127.16.163.65:40551)
I20240430 20:15:17.932144 17037 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.022261676s
I20240430 20:15:17.934739 17067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36980
I20240430 20:15:17.952641 17067 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36986:
name: "update-scan-delta-compact-tbl"
schema {
  columns {
    name: "key"
    type: INT64
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string"
    type: STRING
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int64"
    type: INT64
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20240430 20:15:17.980474 17160 tablet_service.cc:1467] Processing CreateTablet for tablet f3c12dbe3ede41b98a313d3b6a126494 (DEFAULT_TABLE table=update-scan-delta-compact-tbl [id=94331b7ee2ac4d158784b41de2a1ba5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20240430 20:15:17.981221 17160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3c12dbe3ede41b98a313d3b6a126494. 1 dirs total, 0 dirs full, 0 dirs failed
I20240430 20:15:17.987708 17207 tablet_bootstrap.cc:492] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: Bootstrap starting.
I20240430 20:15:17.990444 17207 tablet_bootstrap.cc:654] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: Neither blocks nor log segments found. Creating new log.
I20240430 20:15:17.992978 17207 tablet_bootstrap.cc:492] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: No bootstrap required, opened a new log
I20240430 20:15:17.993187 17207 ts_tablet_manager.cc:1367] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20240430 20:15:17.994159 17207 raft_consensus.cc:348] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" member_type: VOTER last_known_addr { host: "127.16.163.65" port: 40551 } }
I20240430 20:15:17.994467 17207 raft_consensus.cc:374] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240430 20:15:17.994578 17207 raft_consensus.cc:729] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e2129f8e784c35a4ebe88f16148a6a, State: Initialized, Role: FOLLOWER
I20240430 20:15:17.994853 17207 consensus_queue.cc:260] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" member_type: VOTER last_known_addr { host: "127.16.163.65" port: 40551 } }
I20240430 20:15:17.995085 17207 raft_consensus.cc:388] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240430 20:15:17.995195 17207 raft_consensus.cc:482] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240430 20:15:17.995327 17207 raft_consensus.cc:3037] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 0 FOLLOWER]: Advancing to term 1
I20240430 20:15:17.998580 17207 raft_consensus.cc:504] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" member_type: VOTER last_known_addr { host: "127.16.163.65" port: 40551 } }
I20240430 20:15:17.998920 17207 leader_election.cc:302] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e2129f8e784c35a4ebe88f16148a6a; no voters: 
I20240430 20:15:17.999500 17207 leader_election.cc:288] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240430 20:15:17.999713 17209 raft_consensus.cc:2781] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 1 FOLLOWER]: Leader election won for term 1
I20240430 20:15:18.000380 17207 ts_tablet_manager.cc:1398] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20240430 20:15:18.000756 17195 heartbeater.cc:503] Master 127.16.163.126:46649 was elected leader, sending a full tablet report...
I20240430 20:15:18.000944 17209 raft_consensus.cc:686] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [term 1 LEADER]: Becoming Leader. State: Replica: d9e2129f8e784c35a4ebe88f16148a6a, State: Running, Role: LEADER
I20240430 20:15:18.001511 17209 consensus_queue.cc:237] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" member_type: VOTER last_known_addr { host: "127.16.163.65" port: 40551 } }
I20240430 20:15:18.005821 17066 catalog_manager.cc:5484] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a reported cstate change: term changed from 0 to 1, leader changed from <none> to d9e2129f8e784c35a4ebe88f16148a6a (127.16.163.65). New cstate: current_term: 1 leader_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e2129f8e784c35a4ebe88f16148a6a" member_type: VOTER last_known_addr { host: "127.16.163.65" port: 40551 } health_report { overall_health: HEALTHY } } }
I20240430 20:15:21.910970 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=1.000000
I20240430 20:15:24.428494 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.517s	user 2.125s	sys 0.369s Metrics: {"bytes_written":168682,"cfile_init":1,"compiler_manager_pool.queue_time_us":446,"compiler_manager_pool.run_cpu_time_us":475415,"compiler_manager_pool.run_wall_time_us":482115,"dirs.queue_time_us":533,"dirs.run_cpu_time_us":237,"dirs.run_wall_time_us":5864,"drs_written":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8421,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":86,"peak_mem_usage":0,"rows_written":60001,"thread_start_us":669,"threads_started":2}
I20240430 20:15:24.430449 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 403040 bytes on disk
I20240430 20:15:24.431331 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":87,"lbm_reads_lt_1ms":4}
I20240430 20:15:26.432587 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=2.874778
I20240430 20:15:28.568243 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.135s	user 2.051s	sys 0.084s Metrics: {"bytes_written":200339,"cfile_init":1,"dirs.queue_time_us":419,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":1710,"drs_written":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8628,"lbm_writes_lt_1ms":105,"peak_mem_usage":0,"rows_written":74000,"thread_start_us":305,"threads_started":1}
I20240430 20:15:28.569926 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.014660
I20240430 20:15:30.789822 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.220s	user 2.140s	sys 0.068s Metrics: {"bytes_written":370783,"cfile_cache_hit":10,"cfile_cache_hit_bytes":1723,"cfile_cache_miss":162,"cfile_cache_miss_bytes":1875141,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":2073,"dirs.run_cpu_time_us":709,"dirs.run_wall_time_us":4296,"drs_written":1,"lbm_read_time_us":8163,"lbm_reads_lt_1ms":186,"lbm_write_time_us":13732,"lbm_writes_lt_1ms":175,"num_input_rowsets":2,"peak_mem_usage":1131986,"rows_written":134001,"thread_start_us":1825,"threads_started":4}
I20240430 20:15:30.791062 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=1.882590
I20240430 20:15:32.858572 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.067s	user 2.020s	sys 0.044s Metrics: {"bytes_written":186292,"cfile_init":1,"dirs.queue_time_us":394,"dirs.run_cpu_time_us":196,"dirs.run_wall_time_us":1144,"drs_written":1,"lbm_read_time_us":101,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8573,"lbm_writes_lt_1ms":100,"peak_mem_usage":0,"rows_written":69000,"thread_start_us":274,"threads_started":1}
I20240430 20:15:32.859594 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 1362966 bytes on disk
I20240430 20:15:32.860383 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":191,"lbm_reads_lt_1ms":8}
I20240430 20:15:32.861034 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.014485
I20240430 20:15:36.112388 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 3.251s	user 3.177s	sys 0.073s Metrics: {"bytes_written":559107,"cfile_cache_hit":10,"cfile_cache_hit_bytes":2481,"cfile_cache_miss":248,"cfile_cache_miss_bytes":2846039,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":3230,"dirs.run_cpu_time_us":796,"dirs.run_wall_time_us":5205,"drs_written":1,"lbm_read_time_us":12205,"lbm_reads_lt_1ms":268,"lbm_write_time_us":19095,"lbm_writes_lt_1ms":254,"num_input_rowsets":2,"peak_mem_usage":2139715,"rows_written":203001,"thread_start_us":3265,"threads_started":6}
I20240430 20:15:36.113389 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=3.866965
I20240430 20:15:38.607048 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.493s	user 2.410s	sys 0.071s Metrics: {"bytes_written":217789,"cfile_init":1,"dirs.queue_time_us":368,"dirs.run_cpu_time_us":183,"dirs.run_wall_time_us":968,"drs_written":1,"lbm_read_time_us":94,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10031,"lbm_writes_lt_1ms":113,"peak_mem_usage":0,"rows_written":82000,"thread_start_us":250,"threads_started":1}
I20240430 20:15:38.608050 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 1912197 bytes on disk
I20240430 20:15:38.608995 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":197,"lbm_reads_lt_1ms":8}
I20240430 20:15:38.609645 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.014279
I20240430 20:15:43.339002 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 4.729s	user 4.563s	sys 0.140s Metrics: {"bytes_written":784582,"cfile_cache_hit":10,"cfile_cache_hit_bytes":3371,"cfile_cache_miss":338,"cfile_cache_miss_bytes":3995531,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":2207,"dirs.run_cpu_time_us":370,"dirs.run_wall_time_us":2464,"drs_written":1,"lbm_read_time_us":15350,"lbm_reads_lt_1ms":358,"lbm_write_time_us":26327,"lbm_writes_lt_1ms":348,"num_input_rowsets":2,"peak_mem_usage":3260886,"rows_written":285001,"thread_start_us":2236,"threads_started":6}
I20240430 20:15:43.339867 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=7.835715
I20240430 20:15:47.041313 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 3.701s	user 3.506s	sys 0.152s Metrics: {"bytes_written":305984,"cfile_init":1,"dirs.queue_time_us":10793,"dirs.run_cpu_time_us":206,"dirs.run_wall_time_us":1891,"drs_written":1,"lbm_read_time_us":119,"lbm_reads_lt_1ms":4,"lbm_write_time_us":13080,"lbm_writes_lt_1ms":149,"peak_mem_usage":0,"rows_written":113000,"thread_start_us":10634,"threads_started":1}
I20240430 20:15:47.042100 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 2672000 bytes on disk
I20240430 20:15:47.042719 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":115,"lbm_reads_lt_1ms":8}
I20240430 20:15:47.043257 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.013991
I20240430 20:15:53.412679 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 6.369s	user 6.127s	sys 0.231s Metrics: {"bytes_written":1095231,"cfile_cache_hit":10,"cfile_cache_hit_bytes":4634,"cfile_cache_miss":464,"cfile_cache_miss_bytes":5588486,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":4214,"dirs.run_cpu_time_us":1622,"dirs.run_wall_time_us":3566,"drs_written":1,"lbm_read_time_us":20382,"lbm_reads_lt_1ms":484,"lbm_write_time_us":36129,"lbm_writes_lt_1ms":480,"num_input_rowsets":2,"peak_mem_usage":4593295,"rows_written":398001,"thread_start_us":4795,"threads_started":4}
I20240430 20:15:53.413501 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 2674951 bytes on disk
I20240430 20:15:53.413993 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4}
I20240430 20:15:53.414438 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=13.788840
I20240430 20:15:58.194386 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 4.780s	user 4.621s	sys 0.124s Metrics: {"bytes_written":420152,"cfile_init":1,"dirs.queue_time_us":449,"dirs.run_cpu_time_us":166,"dirs.run_wall_time_us":1862,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":17227,"lbm_writes_lt_1ms":199,"peak_mem_usage":0,"rows_written":156000,"thread_start_us":335,"threads_started":1}
I20240430 20:15:58.195235 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=2.874778
I20240430 20:16:00.356712 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.161s	user 2.111s	sys 0.044s Metrics: {"bytes_written":200664,"cfile_init":1,"dirs.queue_time_us":437,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":992,"drs_written":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8866,"lbm_writes_lt_1ms":106,"peak_mem_usage":0,"rows_written":75000,"thread_start_us":298,"threads_started":1}
I20240430 20:16:00.357844 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.022406
I20240430 20:16:10.435806 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 10.078s	user 9.788s	sys 0.264s Metrics: {"bytes_written":1730410,"cfile_cache_hit":15,"cfile_cache_hit_bytes":7291,"cfile_cache_miss":739,"cfile_cache_miss_bytes":8831871,"cfile_init":9,"delta_iterators_relevant":6,"dirs.queue_time_us":3282,"dirs.run_cpu_time_us":225,"dirs.run_wall_time_us":1799,"drs_written":1,"lbm_read_time_us":31055,"lbm_reads_lt_1ms":775,"lbm_write_time_us":57103,"lbm_writes_lt_1ms":746,"num_input_rowsets":3,"peak_mem_usage":6462198,"rows_written":629001,"thread_start_us":2394,"threads_started":8}
I20240430 20:16:10.436692 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 4227149 bytes on disk
I20240430 20:16:10.437194 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":58,"lbm_reads_lt_1ms":4}
I20240430 20:16:10.437623 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=19.741965
I20240430 20:16:16.425158 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 5.987s	user 5.755s	sys 0.220s Metrics: {"bytes_written":529169,"cfile_init":1,"dirs.queue_time_us":509,"dirs.run_cpu_time_us":184,"dirs.run_wall_time_us":1131,"drs_written":1,"lbm_read_time_us":104,"lbm_reads_lt_1ms":4,"lbm_write_time_us":22331,"lbm_writes_lt_1ms":249,"peak_mem_usage":0,"rows_written":198000,"thread_start_us":263,"threads_started":1}
I20240430 20:16:16.426366 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 1330016 bytes on disk
I20240430 20:16:16.426999 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4}
I20240430 20:16:16.427604 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=4.859153
I20240430 20:16:19.138832 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: FlushMRSOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 2.711s	user 2.627s	sys 0.064s Metrics: {"bytes_written":250302,"cfile_init":1,"dirs.queue_time_us":349,"dirs.run_cpu_time_us":147,"dirs.run_wall_time_us":1016,"drs_written":1,"lbm_read_time_us":91,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10287,"lbm_writes_lt_1ms":125,"peak_mem_usage":0,"rows_written":93000,"thread_start_us":235,"threads_started":1}
I20240430 20:16:19.139828 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling LogGCOp(f3c12dbe3ede41b98a313d3b6a126494): free 8349015 bytes of WAL
I20240430 20:16:19.140260 17137 log_reader.cc:386] T f3c12dbe3ede41b98a313d3b6a126494: removed 1 log segments from log reader
I20240430 20:16:19.140393 17137 log.cc:1076] T f3c12dbe3ede41b98a313d3b6a126494 P d9e2129f8e784c35a4ebe88f16148a6a: Deleting log segment in path: /tmp/dist-test-taskezig1B/test-tmp/update_scan_delta_compact-test.0.UpdateScanDeltaCompactionTest.TestAll.1714508117553665-17037-0/minicluster-data/ts-0-root/wals/f3c12dbe3ede41b98a313d3b6a126494/wal-000000001 (ops 1-863)
I20240430 20:16:19.142735 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: LogGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.003s	user 0.000s	sys 0.000s Metrics: {}
I20240430 20:16:19.143138 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494): 622247 bytes on disk
I20240430 20:16:19.143677 17137 maintenance_manager.cc:643] P d9e2129f8e784c35a4ebe88f16148a6a: UndoDeltaBlockGCOp(f3c12dbe3ede41b98a313d3b6a126494) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20240430 20:16:19.144327 17196 maintenance_manager.cc:419] P d9e2129f8e784c35a4ebe88f16148a6a: Scheduling CompactRowSetsOp(f3c12dbe3ede41b98a313d3b6a126494): perf score=0.021669
I20240430 20:16:21.512777 17037 update_scan_delta_compact-test.cc:209] Time spent Insert: real 63.495s	user 57.967s	sys 4.871s
I20240430 20:16:21.561918 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240430 20:16:21.565903 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240430 20:16:21.569208 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240430 20:16:21.572607 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240430 20:16:21.575937 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
I20240430 20:16:21.579388 17259 scanners.cc:360] Slow scans show is disabled. Set --show_slow_scans to enable it.
F20240430 20:16:21.581166 17259 monotime.cc:342] Check failed: t_beg.Initialized() 
*** Check failure stack trace: ***
*** Aborted at 1714508181 (unix time) try "date -d @1714508181" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e80000428d) received by PID 17037 (TID 0x7f7d6f189700) from PID 17037; stack trace: ***
    @     0x7f7d9df56980 (unknown) at ??:0
    @     0x7f7d97001fb7 gsignal at ??:0
    @     0x7f7d97003921 abort at ??:0
    @     0x7f7d99d2adcd google::LogMessage::Fail() at ??:0
    @     0x7f7d99d2eb93 google::LogMessage::SendToLog() at ??:0
    @     0x7f7d99d2a7cc google::LogMessage::Flush() at ??:0
    @     0x7f7d99d2bf59 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f7d9b28fc80 kudu::operator-() at ??:0
    @     0x7f7da6ab014a kudu::tablet::WriteOp::ToString() at ??:0
    @     0x7f7da6a7a5de kudu::tablet::OpDriver::ToStringUnlocked() at ??:0
    @     0x7f7da6a7a336 kudu::tablet::OpDriver::ToString() at ??:0
    @     0x7f7da6a272d9 kudu::tablet::TabletReplica::GetInFlightOps() at ??:0
    @     0x7f7dad8a5d87 kudu::tserver::TabletServerPathHandlers::HandleTransactionsPage() at ??:0
    @     0x7f7dad8b493c kudu::tserver::TabletServerPathHandlers::Register()::$_3::operator()() at ??:0
    @     0x7f7dad8b47a2 std::_Function_handler<>::_M_invoke() at ??:0
    @     0x7f7da3c1202a std::function<>::operator()() at ??:0
    @     0x7f7da3c0a7ba kudu::Webserver::RunPathHandler() at ??:0
    @     0x7f7da3c057f1 kudu::Webserver::BeginRequestCallback() at ??:0
    @     0x7f7da3c03927 kudu::Webserver::BeginRequestCallbackStatic() at ??:0
    @     0x7f7da3c41089 handle_request at ??:0
    @     0x7f7da3c4376b worker_thread at ??:0
    @     0x7f7d9df4b6db start_thread at ??:0
    @     0x7f7d970e471f clone at ??:0