Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-17853

Restore npm database from blobstore could miss packages that are cached locally but removed from the remote


    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.14.0
    • Fix Version/s: None
    • Component/s: NPM
    • Labels:
    • Story Points:


      We changed the behaviour of NXRM to continue to serve up locally cached npm packages even if those packages are removed from the remote (see https://issues.sonatype.org/browse/NEXUS-15714). However this leaves a potential gap with the CMA restore task because there can be packages in NXRM that don't exist in the remote and the restore task is order dependent. The package root is only restored if it doesn't already exist, therefore if it has been refetched the package root containing the deleted package will not be restored.

      1. Create an npm-hosted repository.
      2. Upload two versions of the same package.
      3. Create an npm-proxy repository that proxies the hosted repository.
      4. Pull one version through the proxy so that it is cached.
      5. Run database backup task
      6. Pull the metadata through the proxy so that it is cached.
      7. Pull the second version through the proxy so now you should have both versions and the metadata cached and in the blobstore.
      8. Delete one of the versions from the hosted repository (simulating it being removed from the "remote"
      9. Shutdown NXRM and follow the documented steps to restore from database backup. You will now have a database for npm-proxy that contains a single package, but the blobstore will contain two packages.
      10. Pull the metadata through the proxy a second time so that it is cached again, now it should only list the one package that was remaining in the npm-hosted repository
      11. Run the "Repair - reconcile component database from blobstore" task.
      12. You should now see both versions in repository browse, however if you look at the metadata there is a chance only one package is listed.

      Technical Direction

      • Possible solution 1: Go to remote, get most up to date package.json and restore from the remote then merge in packages from the blob restore.
      • Possible solution 2: Loop through the blobstore, find the package.json that matches and restore them. Then go through the blobstore and restore the packages afterwards.
      • May be other solutions, are leaving to developer who grabs this to evaluate/decide best course.




            Unassigned Unassigned
            jstephens Joseph Stephens
            Last Updated By:
            Peter Lynch Peter Lynch
            0 Vote for this issue
            2 Start watching this issue


              Date of First Response: