Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-20552

Repair - Reconcile component database from blobstore task dry run option can block download of hosted assets which do not have an owning component


    • Story Points:
    • Sprint:
      NXRM Trinity Sprint 11


      Problem Summary

      Some repository formats allow storing asset records without an owning component. When the Dry Run option of the Repair - Reconcile component database from blobstore task is used, logic is triggered that incorrectly deletes such asset records from the component database. The related blobs are seemingly not deleted from the blobstore ( ie. they are recoverable ) but the assets are no longer downloadable after being deleted from the database.

      Recover the incorrectly deleted asset records by running a Repair - Reconcile component database from blobstore task, without the Dry Run option enabled.

      Sample Reproduce for Yum Hosted repo

      1. Create a yum-hosted repo at zero depth
      2. Deploy an rpm file into it. I used http://mirror.centos.org/centos/7/os/x86_64/Packages/389-ds-base- and this command:

      curl -v -u admin:admin123 "http://localhost:8081/repository/yum-hosted/os/x86_64/Packages/PackageKit-glib-1.1.10-1.el7.centos.x86_64.rpm" --upload-file PackageKit-glib-1.1.10-1.el7.centos.x86_64.rpm --request PUT

      3. Attempt to download the RPM file at the same path you uploaded it. This should work with 200 response.
      4. After 60 seconds)expected), the Browse and Search views show the repodata/* files are generated. Attempt to download these metadata files - it works.
      5. Create a Repair - Reconcile component database from blobstore task. Select the Dry Run option. Run the task.
      6. Bug 1: Logging reports that an RPM file would have been restored by the task:

      2019-07-23 10:24:03,879-0300 INFO  [quartz-3-thread-2] *SYSTEM org.sonatype.nexus.blobstore.restore.RestoreMetadataTask - ::DRY RUN:: Actions will be logged, but no changes will be made.
      2019-07-23 10:24:03,923-0300 INFO  [quartz-3-thread-2] *SYSTEM com.sonatype.nexus.blobstore.restore.internal.YumRestoreBlobStrategy - ::DRY RUN:: Restored asset, blob store: default, repository: yum-hosted, path: os/x86_64/Packages/PackageKit-glib-1.1.10-1.el7.centos.x86_64.rpm, blob name: os/x86_64/Packages/PackageKit-glib-1.1.10-1.el7.centos.x86_64.rpm, blob id: 7f3dc2bf-c00b-4d8c-bafb-4e5670ac9bcc

      Why would it restore that file? The file downloads fine, is a valid RPM and there are no inconsistencies between blobstore and database.
      7. 60 seconds after the reconcile task finishes, the YUM metadata is rebuilt automatically - supposedly from the manipulation which did not happen? ( ie dry run ):

      2019-07-23 10:24:03,949-0300 INFO  [quartz-3-thread-2] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'reconcile' [blobstore.rebuildComponentDB] state change RUNNING -> WAITING (OK)
      2019-07-23 10:25:03,929-0300 INFO  [event-6-thread-17] *SYSTEM org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl - Rebuilding yum metadata for repository yum-hosted
      2019-07-23 10:25:03,954-0300 INFO  [event-6-thread-17] *SYSTEM org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl - Finished rebuilding yum metadata for repository yum-hosted

      8. Now you are in the following state:

      • the RPM blob file is still in the blob store, not marked soft-deleted or changed in any way from when it was first uploaded
      • the Browse view only shows os/x86_64/Packages empty node ( normal according to docs after running reconcile )
      • the Search view for yum-hosted is empty ( normal according to docs after running reconcile )
      • Bug 2: attempts to download yum-hosted metadata files or the rpm file all fail with 404

      9. Run the Reconcile component database from blobstore task again, this time without the Dry Run option selected. After 60 seconds yum metadata is rebuilt and repodata files and the rpm is downloadable.

      The dry run option is essentially acting like it is manipulating the database in the inverse of reconciling.


      • dry run should not trigger any other tasks, or act like it actually did anything other than log what it would do
      • reconcile task had no business manipulating a perfectly good and working RPM, even if this was not dry run
      • rpms that are not soft-deleted or manipulated visibly should still be downloadable from the yum-hosted repo, regardless of the presence of metadata




            isverdlov Ivan Sverdlov [X] (Inactive)
            plynch Peter Lynch
            Last Updated By:
            Michael Prescott Michael Prescott
            Original Nexus - Trinity - not for new tickets
            0 Vote for this issue
            5 Start watching this issue


              Date of First Response: