Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28078

Docker - Delete unused manifests and images task may delete referenced layers if the database query to select components encounters limits

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.30.0, 3.30.1
    • Fix Version/s: 3.31.1
    • Component/s: Docker

      Description

      The Docker - Delete unused manifests and images task will perform a database query to find all components ( docker manifests) that reference a given asset. If this query will select a large number of records, and exceeds 10000 records, there is a possibility that some manifests that do reference a layer asset will not be found. In this case NXRM will think a layer is not referenced and may incorrectly delete it.

      Example of what one may see in the nexus.log indicating this could have happened:

       org.sonatype.nexus.repository.docker.internal.orient.DockerGCFacetImpl - Unable to read V2 Manifest for asset Asset{...} v1779}, name=v2/-/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4}, manifest invalid
      
      2021-05-13 02:04:26,217+0000 INFO  [Thread-6913708 <command>sql.select from asset where (attributes.docker.asset_kind = :p0) and (bucket=#9:1330)</command>]  *SYSTEM com.orientechnologies.common.profiler.OProfilerStub - $ANSI{green {db=component}} [TIP] Query 'SELECT FROM asset WHERE (attributes.docker.asset_kind = "MANIFEST" ) AND (bucket = #9:1330 )' returned a result set with more than 10000 records. Check if you really need all these records, or reduce the resultset by using a LIMIT to improve both performance and used RAM
      

      docker pulls for previously working images may fail to find certain layers. Docker pull may show an error message to the user containing the message:

      error pulling image configuration: unknown blob

      If you have the Audit Log feature enabled, then the audit log should be able to prove what task actually deleted the affected layers. Find the entry in the audit log by grepping the audit log for the layer hash that is being reported missing.

      Expected

      Make the query this task performs more robust such that it will not accidentally delete referenced layers.

      Mitigation

      The issue was introduced in version 3.30.0.

      If you have recently upgraded to an affected version with this bug, and are running the Docker - Delete unused manifests and images task, Sonatype recommends the task be disabled until you can upgrade to a version with the fix for this issue. Go to the task Settings page and uncheck the Task enabled checkbox and Save.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              aornatovskyy Anatoliy Ornatovskyy
              Last Updated By:
              Joe Tom Joe Tom
              Team:
              NXRM - Trinity
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title