Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28078

Docker - Delete unused manifests and images task may delete referenced layers if the database query to select components encounters limits


    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.30.0, 3.30.1
    • Fix Version/s: 3.31.1
    • Component/s: Docker


      The Docker - Delete unused manifests and images task will perform a database query to find all components ( docker manifests) that reference a given asset. If this query will select a large number of records, and exceeds 10000 records, there is a possibility that some manifests that do reference a layer asset will not be found. In this case NXRM will think a layer is not referenced and may incorrectly delete it.

      Example of what one may see in the nexus.log indicating this could have happened:

       org.sonatype.nexus.repository.docker.internal.orient.DockerGCFacetImpl - Unable to read V2 Manifest for asset Asset{...} v1779}, name=v2/-/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4}, manifest invalid
      2021-05-13 02:04:26,217+0000 INFO  [Thread-6913708 <command>sql.select from asset where (attributes.docker.asset_kind = :p0) and (bucket=#9:1330)</command>]  *SYSTEM com.orientechnologies.common.profiler.OProfilerStub - $ANSI{green {db=component}} [TIP] Query 'SELECT FROM asset WHERE (attributes.docker.asset_kind = "MANIFEST" ) AND (bucket = #9:1330 )' returned a result set with more than 10000 records. Check if you really need all these records, or reduce the resultset by using a LIMIT to improve both performance and used RAM

      docker pulls for previously working images may fail to find certain layers. Docker pull may show an error message to the user containing the message:

      error pulling image configuration: unknown blob

      If you have the Audit Log feature enabled, then the audit log should be able to prove what task actually deleted the affected layers. Find the entry in the audit log by grepping the audit log for the layer hash that is being reported missing.


      Make the query this task performs more robust such that it will not accidentally delete referenced layers.


      The issue was introduced in version 3.30.0.

      If you have recently upgraded to an affected version with this bug, and are running the Docker - Delete unused manifests and images task, Sonatype recommends the task be disabled until you can upgrade to a version with the fix for this issue. Go to the task Settings page and uncheck the Task enabled checkbox and Save.


          Issue Links



              Unassigned Unassigned
              aornatovskyy Anatoliy Ornatovskyy [X] (Inactive)
              Last Updated By:
              Rich Seddon Rich Seddon
              NXRM - Trinity
              0 Vote for this issue
              7 Start watching this issue


                Date of First Response: