Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-30534

Docker Proxy S3 - Exception pulling pulling an image which has the same checksum as an existing image

    Details

    • Story Points:
      1
    • Release Note:
      Yes
    • Sprint:
      NXRM Sentinels Sprint 23, NXRM Sentinels Sprint 24, NXRM Sentinels Sprint 25, NXRM Sentinels Sprint 26
    • InvestmentLayer:
      maintenance

      Description

      While investigating NEXUS-30168 we noticed that a Docker proxy repository backed by an S3 blob store generates an exception when you pull an image which has the same checksum as an existing image in that repository. Specifically, in https://issues.sonatype.org/browse/NEXUS-29390 we updated the S3 blob store implementation to no longer create duplicate blobs during upload. As a result, the logic in the https://github.com/sonatype/nexus-internal/blob/1cf2968e6d64c1a3b0dcfdcfdd9407d7fb3ab276/private/plugins/nexus-repository-docker/src/main/java/org/sonatype/nexus/repository/docker/internal/datastore/recipe/DockerContentFacetImpl.java#L368-L369 and https://github.com/sonatype/nexus-internal/blob/1cf2968e6d64c1a3b0dcfdcfdd9407d7fb3ab276/private/plugins/nexus-repository-docker/src/main/java/org/sonatype/nexus/repository/docker/internal/orient/DockerProxyFacetImpl.java#L499-L500  classes causes a docker pull to inadvertently delete the blob associated with that image if another image already exists in the repository with the same checksum. 

       

      The steps below use two Docker images namely  debian:unstable-slim and debian:unstable-20211201-slim which at the time of writing both have the same checksum according to the information on the Docker hub page.

       

      To reproduce

      • Create a S3 blob store
      • Create a docker proxy repository an associate it with that S3 Blob store
      • Do a docker pull of an image such as debian e.g.
        docker pull <your_nxrm_ip_address:docker_proxy_port>/debian:unstable-slim
      • Do another docker pull: 
        docker pull 192.168.1.125:9091/debian:unstable-20211201-slim

      Outcome

      A 500 error occurs with a stack trace similar to:

       

      com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: EABPAY6PRJF9RXKM; S3 Extended Request ID: SbYtHRRZYjJp/hyRDpUa60GYhSVEeFrit1T7hn/SEfvaXq23UlFEXHzdM4thNZmNBJfGia7inRQ=; Proxy: null)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1811)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1395)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1371)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
       at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
       at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
       at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
       at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
       at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
       at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1490)
       at com.amazonaws.services.s3.EncryptingAmazonS3Client.getObject(EncryptingAmazonS3Client.java:164)
       at com.amazonaws.services.s3.EncryptingAmazonS3Client.getObject(EncryptingAmazonS3Client.java:171)
       at org.sonatype.nexus.blobstore.s3.internal.S3BlobStore$S3Blob.doGetInputStream(S3BlobStore.java:692)
       at org.sonatype.nexus.blobstore.BlobSupport.getInputStream(BlobSupport.java:87)
       at org.sonatype.nexus.repository.view.payloads.TempBlob.get(TempBlob.java:110)
       at org.sonatype.nexus.common.io.InputStreamSupplier$get.call(Unknown Source)
       at org.sonatype.nexus.repository.docker.internal.orient.V2ManifestUtilImpl.readManifest(V2ManifestUtilImpl.groovy:107)
       at org.sonatype.nexus.repository.docker.internal.DockerContentValidator.v2DetermineContentType(DockerContentValidator.java:168)
       at org.sonatype.nexus.repository.docker.internal.DockerContentValidator.determineContentType(DockerContentValidator.java:97)
       at org.sonatype.nexus.repository.storage.StorageTxImpl.determineContentType(StorageTxImpl.java:1071)
       at org.sonatype.nexus.repository.storage.StorageTxImpl.buildStorageHeaders(StorageTxImpl.java:779)
       at org.sonatype.nexus.repository.storage.StorageTxImpl.createBlob(StorageTxImpl.java:739)
       at org.sonatype.nexus.repository.storage.StorageTxImpl.setBlob(StorageTxImpl.java:1036)
       at sun.reflect.GeneratedMethodAccessor298.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53)
       at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:69)
       at com.sun.proxy.$Proxy240.setBlob(Unknown Source)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerFacetDatabaseUtils.saveAsset(DockerFacetDatabaseUtils.java:347)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl.doPutManifestByTag(DockerProxyFacetImpl.java:468)
       at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:58)
       at org.sonatype.nexus.transaction.TransactionInterceptor.proceedWithTransaction(TransactionInterceptor.java:66)
       at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl.putManifestByTag(DockerProxyFacetImpl.java:449)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl.putManifest(DockerProxyFacetImpl.java:432)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl.storeByAssetKind(DockerProxyFacetImpl.java:172)
       at org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl.store(DockerProxyFacetImpl.java:141)
       at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.doGet(ProxyFacetSupport.java:286)
       at org.sonatype.nexus.repository.docker.internal.DockerProxyFacetSupport.doGet(DockerProxyFacetSupport.java:247)
       at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.lambda$1(ProxyFacetSupport.java:260)
       at org.sonatype.nexus.common.io.CooperatingFuture.performCall(CooperatingFuture.java:122)
       at org.sonatype.nexus.common.io.CooperatingFuture.call(CooperatingFuture.java:64)
       at org.sonatype.nexus.common.io.ScopedCooperationFactorySupport$ScopedCooperation.cooperate(ScopedCooperationFactorySupport.java:99)
       at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.get(ProxyFacetSupport.java:251)
       at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.get(ProxyFacetSupport.java:240)
       at org.sonatype.nexus.repository.proxy.ProxyHandler.handle(ProxyHandler.java:52)
       at org.sonatype.nexus.repository.view.Context.proceed(Context.java:88)
       at org.sonatype.nexus.repository.storage.LastDownloadedHandler.handle(LastDownloadedHandler.java:59)
       at org.sonatype.nexus.repository.view.Context.proceed(Context.java:88)
       at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
       at org.sonatype.nexus.repository.view.Context.proceed(Context.java:88)
       at org.sonatype.nexus.repository.view.Context$proceed.call(Unknown Source)
       at org.sonatype.nexus.repository.docker.internal.V2Handlers$_closure16.doCall(V2Handlers.groovy:280)
       at sun.reflect.GeneratedMethodAccessor295.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
       at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
       at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
       at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
       at groovy.lang.Closure.call(Closure.java:420)
       at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
       at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
       at com.sun.proxy.$Proxy292.handle(Unknown Source)
      
      

       

       

      Expected

      • Pulling a docker image which has a checksum as an existing docker image should not generate such an exception. That is, the docker pull should be successful.

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              moliverio Michael Oliverio
              Reporter:
              oshiyanbade Olu Shiyanbade
              Last Updated By:
              Michael Oliverio Michael Oliverio
              Team:
              NXRM - Sentinels
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title