Details
Description
Several upload handlers currently follow this anti-pattern:
- start transaction
- do some validation
- create temporary blobs
- create components/assets
- commit transaction
The problem with this approach is that it leaves the transaction open for a long time. If the transaction needs to be retried then it will repeat the same checks and create more temporary blobs.
EDIT: The other issue with starting the storage transaction early on is that it breaks the later call to check content selector permissions (but only if content selector permissions apply to the upload). This is because the current active transaction is for the component DB, but the content selector check wants to query the config DB. We don't support nested active transactions, so the content selector basically ends up querying the current DB (ie. component) for config related information - this leads to schema problems and other exceptions.
Ideally the transaction should only be started just before it's needed, ie. calling into the facet to create the component/assets, and closed as soon as possible. This also solves the content selector issue because the call to check permissions is now made outside of an active transaction, before the storage transaction is started.
This was recently done for the Maven upload handler:
here's the original which followed the anti-pattern:
For simple handlers you may not even need to specify any transaction because the facet will do that for you when you call their "put" method. But to do this you may need to setup a unit-of-work which the facet can then use - because I don't believe the resource endpoint exposed for upload has an equivalent to UnitOfWorkHandler, which sets it up for incoming repository requests. (See the updated Maven upload handler for an example.)
I've listed the formats that need their upload handlers checking/reworking in the components field.
You've hit this problem if on upload from a non-admin user account, you see this error in nexus.log:
2018-08-02 10:40:20,631+0200 WARN [qtp2023704419-47590] *UNKNOWN org.sonatype.nexus.siesta.internal.UnexpectedExceptionMapper - (ID 6366c419-bc9e-44ff-9259-3feda5834aa9) Unexpected exception: java.lang.IllegalArgumentException: Class 'selector_selector' not found in current database java.lang.IllegalArgumentException: Class 'selector_selector' not found in current database at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.browseClass(ODatabaseDocumentTx.java:2589) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.browseClass(ODatabaseDocumentTx.java:2581) at org.sonatype.nexus.orient.entity.EntityAdapter.browseDocuments(EntityAdapter.java:244) at org.sonatype.nexus.orient.entity.action.BrowseEntitiesAction.execute(BrowseEntitiesAction.java:40) at org.sonatype.nexus.orient.entity.IterableEntityAdapter.browse(IterableEntityAdapter.java:96) at org.sonatype.nexus.internal.selector.SelectorConfigurationStoreImpl.lambda$0(SelectorConfigurationStoreImpl.java:73) at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$1(OrientOperations.java:56) at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64) at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:196) at org.sonatype.nexus.transaction.Operations.call(Operations.java:146) at org.sonatype.nexus.orient.transaction.OrientOperations.call(OrientOperations.java:56) at org.sonatype.nexus.internal.selector.SelectorConfigurationStoreImpl.browse(SelectorConfigurationStoreImpl.java:73) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:270) at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53) at org.sonatype.nexus.internal.selector.SelectorManagerImpl.browse(SelectorManagerImpl.java:108) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:270) at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53) at org.sonatype.nexus.repository.security.internal.ContentPermissionCheckerImpl.isPermitted(ContentPermissionCheckerImpl.java:98) at org.sonatype.nexus.repository.upload.UploadHandler.ensurePermitted(UploadHandler.java:74) at org.sonatype.nexus.repository.maven.MavenUploadHandler.createAssets(MavenUploadHandler.java:215) at org.sonatype.nexus.repository.maven.MavenUploadHandler.lambda$0(MavenUploadHandler.java:168) at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56) at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:200) at org.sonatype.nexus.transaction.Operations.call(Operations.java:146) at org.sonatype.nexus.repository.maven.MavenUploadHandler.doUpload(MavenUploadHandler.java:151) at org.sonatype.nexus.repository.maven.MavenUploadHandler.handle(MavenUploadHandler.java:139) at org.sonatype.nexus.repository.upload.internal.UploadManagerImpl.handle(UploadManagerImpl.java:71) at org.sonatype.nexus.repository.rest.internal.resources.ComponentsResource.uploadComponent(ComponentsResource.java:255) at sun.reflect.GeneratedMethodAccessor3050.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:294) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:248) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:235) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at org.sonatype.nexus.siesta.internal.resteasy.ComponentContainerImpl.service(ComponentContainerImpl.java:106) at org.sonatype.nexus.siesta.SiestaServlet.service(SiestaServlet.java:137) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181) at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) at com.sonatype.nexus.analytics.internal.RestRequestCollector.doFilter(RestRequestCollector.java:86) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135) at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 2018-08-02 10:40:20,631+0200 WARN [qtp2023704419-47590] *UNKNOWN org.sonatype.nexus.siesta.internal.UnexpectedExceptionMapper - (ID 6366c419-bc9e-44ff-9259-3feda5834aa9) Response: [500] 'ERROR: (ID 6366c419-bc9e-44ff-9259-3feda5834aa9) java.lang.IllegalArgumentException: Class 'selector_selector' not found in current database'; mapped from: java.lang.IllegalArgumentException: Class 'selector_selector' not found in current database
Resolution
This issue describes a general component upload problem which affected all repository formats.
For Maven, the problem was fixed by NEXUS-16740 changes and shipped in 3.12.0.
Other formats have linked issues on this ticket and were fixed as of 3.14.0.
NEXUS-17735- NPM - fixed 3.14.0NEXUS-17736- NuGet - fixed 3.14.0NEXUS-17737- RubyGems - fixed 3.14.0NEXUS-17738- RAW - fixed 3.14.0- NEXUS-16943 - PyPI - fixed 3.14.0
Attachments
Issue Links
- is superceded by
-
NEXUS-16740 Nexus Upload Interface doesnt update or create metadata after upload file
-
- Closed
-
-
NEXUS-17735 NPM Upload Transaction Too Long
-
- Closed
-
-
NEXUS-17736 NuGet Upload Transaction Too Long
-
- Closed
-
-
NEXUS-17737 Ruby Gems Transaction Too Long
-
- Closed
-
-
NEXUS-17738 Raw Upload Transaction Too Long
-
- Closed
-
-
NEXUS-18481 PyPI Upload Transaction Too Long
-
- Closed
-