Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-14860

Rebuild maven repository metadata task bloats blobstore


    • 3



      1. Start up Nexus with a clean blob store
      2. Deploy a Maven project into a hosted release repository
      3. Scan the blobstore with "find . -name "*.properties"|xargs grep deleted=true", observe there are no deleted blobs
      4. Schedule and run a rebuild repository metadata task against the repository you deployed into
      5. Scan the blobstore again, observer there are now 3 deleted blobs for each maven-metadata.xml file
      6. Run the task again
      7. Scan the blobstore again, observe there are now an additional 3 deleted blobs for each maven-metadata.xml file


      Rebuilt maven-metadata.xml files should only be written out if they have changed from the previous version.  This means that it is expected that the first run of the task will delete all the existing maven-metadata.xml files and their checksums, since Nexus does not build the files in the same format as Maven.

      The consequence of this behavior is that blob storage gets unnecessarily full of deleted files, consuming way more space than is needed, and running a compact blob store task can take a very long time to complete.

      The hope is two things:

      • avoid replacing the blob with an identical blob - if the hashes match, leave the original in place (and see if there's a way to purge the rejected candidate replacement)
      • another thing to consider would be hard-deleting maven-metadata files immediately instead of soft-deleting them (as long as that's not a weird special case down in generic asset code); this would stop blobstores getting bloated with soft-deleted derived content




            mbucher Michael Bucher
            rseddon Rich Seddon
            Peter Lynch Peter Lynch
            NXRM - Morpheus
            0 Vote for this issue
            4 Start watching this issue