Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-19052

Support for Nexus 3 Blob Storage S3 Hard Delete (Minio)

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Automatically Closed
    • Affects Version/s: 3.15.2, 3.16.1
    • Fix Version/s: None
    • Component/s: Blobstore, S3
    • Labels:
      None
    • Environment:
      Nexus deployed on Kubernetes

      Description

      Hello !

      I know other S3 provider as not officially supported by Sonatype but we really would like to have Minio as S3 provider full working (specially because we want to use it as Azure Blob Gateway)

      We are able to configure our blobstores successfully, but the deletion of blobs doesn't work. It uses the S3 LIfecycle (object tagging) not suported by Minio.

      So is it possible to have some kind of configuration per blobstore to support "hard" deletion rather than soft-delete ?

      Minio configured similar to this : https://gist.github.com/ataylor284/7b15c276441906d16d43f58cf8e3ea94
      Possible issue here : https://github.com/sonatype/nexus-public/blob/master/plugins/nexus-blobstore-s3/src/main/java/org/sonatype/nexus/blobstore/s3/internal/S3BlobStore.java#L375

      Running Nexus 3.15

      Here is the full stackstrace :
       

      org.sonatype.nexus.blobstore.api.BlobStoreException: BlobId: path$nexus-repository-docker/160145ae-4e0b-416b-baaf-2a7be0d44aa0/0, com.amazonaws.services.s3.model.AmazonS3Exception: A header you provided implies functionality that is not implemented (Service: Amazon S3; Status Code: 501; Error Code: NotImplemented; Request ID: /nexus-01-mino-local-dev-docker/content/directpath/nexus-repository-docker/160145ae-4e0b-416b-baaf-2a7be0d44aa0/0.bytes; S3 Extended Request ID: 3L137), S3 Extended Request ID: 3L137, Cause: A header you provided implies functionality that is not implemented (Service: Amazon S3; Status Code: 501; Error Code: NotImplemented; Request ID: /nexus-01-mino-local-dev-docker/content/directpath/nexus-repository-docker/160145ae-4e0b-416b-baaf-2a7be0d44aa0/0.bytes; S3 Extended Request ID: 3L137)
      
      at org.sonatype.nexus.blobstore.s3.internal.S3BlobStore.doDelete(S3BlobStore.java:383)
      
      at org.sonatype.nexus.blobstore.BlobStoreSupport.delete(BlobStoreSupport.java:102)
      
      at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
      
      at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
      
      at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
      
      at org.sonatype.nexus.repository.docker.internal.Upload.lambda$0(Upload.java:153)
      
      at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
      
      at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
      
      at org.sonatype.nexus.repository.docker.internal.Upload.cleanup(Upload.java:153)
      
      at org.sonatype.nexus.repository.docker.internal.UploadManagerImpl.cancel(UploadManagerImpl.java:120)
      
      at org.sonatype.nexus.repository.docker.internal.DockerHostedFacetImpl.completeBlobUpload(DockerHostedFacetImpl.java:607)
      
      at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
      
      at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
      
      at org.sonatype.nexus.repository.docker.internal.DockerHostedFacet$completeBlobUpload$2.call(Unknown Source)
      
      at org.sonatype.nexus.repository.docker.internal.V2Handlers$_closure5.doCall(V2Handlers.groovy:146)
      
      at sun.reflect.GeneratedMethodAccessor362.invoke(Unknown Source)
      
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      
      at java.lang.reflect.Method.invoke(Method.java:498)
      
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
      
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
      
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
      
      at groovy.lang.Closure.call(Closure.java:418)
      
      at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
      
      at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
      
      at com.sun.proxy.$Proxy226.handle(Unknown Source)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.view.Context$proceed.call(Unknown Source)
      
      at org.sonatype.nexus.repository.docker.internal.V2Handlers$_closure18.doCall(V2Handlers.groovy:294)
      
      at sun.reflect.GeneratedMethodAccessor262.invoke(Unknown Source)
      
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      
      at java.lang.reflect.Method.invoke(Method.java:498)
      
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
      
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
      
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
      
      at groovy.lang.Closure.call(Closure.java:418)
      
      at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
      
      at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
      
      at com.sun.proxy.$Proxy226.handle(Unknown Source)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.view.Context$proceed.call(Unknown Source)
      
      at org.sonatype.nexus.repository.docker.internal.V2Handlers$_closure1.doCall(V2Handlers.groovy:88)
      
      at sun.reflect.GeneratedMethodAccessor261.invoke(Unknown Source)
      
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      
      at java.lang.reflect.Method.invoke(Method.java:498)
      
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
      
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
      
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264)
      
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
      
      at groovy.lang.Closure.call(Closure.java:418)
      
      at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:54)
      
      at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:124)
      
      at com.sun.proxy.$Proxy226.handle(Unknown Source)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
      
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
      
      at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
      
      at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:64)
      
      at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
      
      at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
      
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:210)
      
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:172)
      
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
      
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      
      at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
      
      at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
      
      at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
      
      at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
      
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      
      at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
      
      at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
      
      at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
      
      at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
      
      at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
      
      at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
      
      at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
      
      at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
      
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:80)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
      
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      
      at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
      
      at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
      
      at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
      
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
      
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
      
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      
      at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
      
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      
      at org.eclipse.jetty.server.Server.handle(Server.java:531)
      
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
      
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
      
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      
      at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
      
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
      
      	at java.lang.Thread.run(Thread.java:748)
      

       
      Thanks in advance.

      Valentin

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            jonesbusy@gmail.com Valentin Delaye
            Last Updated By:
            Joe Tom Joe Tom
            Votes:
            14 Vote for this issue
            Watchers:
            15 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title