Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-14439

/service/siesta/rest/beta/search IllegalStateException current database instance is not active on the current thread while under stress

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.2
    • Fix Version/s: 3.10.0
    • Component/s: Database, REST
    • Labels:
    • Story Points:
      3

      Description

      Under stress ( 700-900 requests per second of various types ) the /service/siesta/rest/beta/search resource may trigger the following message in the nexus.log:

      2017-09-26 00:12:11,238-0700 ERROR [qtp297617943-16913] nxadmin com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPooled - Error after transaction rollback
      java.lang.IllegalStateException: The current database instance (com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPooled@394fa658) is not active on the current thread (Thread[qtp297617943-16913,5,main]). Current active database is: null
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.checkIfActive(ODatabaseDocumentTx.java:3391)
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.getConfiguration(ODatabaseDocumentTx.java:1203)
      	at com.orientechnologies.orient.core.query.live.OLiveQueryHook.onAfterTxRollback(OLiveQueryHook.java:159)
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.rollback(ODatabaseDocumentTx.java:2972)
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.rollback(ODatabaseDocumentTx.java:2946)
      	at org.sonatype.nexus.repository.storage.StorageTxImpl.rollback(StorageTxImpl.java:181)
      	at org.sonatype.nexus.repository.storage.StorageTxImpl.close(StorageTxImpl.java:224)
      	at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53)
      	at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
      	at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191)
      	at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
      	at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:66)
      	at com.sun.proxy.$Proxy225.close(Unknown Source)
      	at org.sonatype.nexus.repository.browse.internal.BrowseServiceImpl.browseComponentAssets(BrowseServiceImpl.java:173)
      	at org.sonatype.nexus.repository.browse.internal.resources.SearchResource.toComponent(SearchResource.java:169)
      	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
      	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
      	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
      	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
      	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
      	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
      	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
      	at org.sonatype.nexus.repository.browse.internal.resources.SearchResource.search(SearchResource.java:157)
      	at sun.reflect.GeneratedMethodAccessor280.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
      	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:294)
      	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:248)
      	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:235)
      	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402)
      	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209)
      	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227)
      	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
      	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
      	at org.sonatype.nexus.siesta.internal.resteasy.ComponentContainerImpl.service(ComponentContainerImpl.java:106)
      	at org.sonatype.nexus.siesta.SiestaServlet.service(SiestaServlet.java:137)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
      	at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
      	at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
      	at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
      	at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
      	at com.sonatype.nexus.analytics.internal.RestRequestCollector.doFilter(RestRequestCollector.java:80)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
      	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      	at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
      	at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
      	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      	at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
      	at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
      	at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
      	at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
      	at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
      	at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
      	at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
      	at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
      	at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
      	at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:102)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
      	at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
      	at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
      	at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
      	at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
      	at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
      	at org.eclipse.jetty.server.Server.handle(Server.java:517)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
      	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
      	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
      	at java.lang.Thread.run(Thread.java:748)
      

      *Note*: When this exception happens, the request.log will not show any 4xx or 5xx status codes for this request, only 200 - this seems wrong as it sounds like a pretty severe error and may suggest the REST resource needs better error handling. Unclear what the response payload did contain ( and error payload with message and status code 200???? )

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              plynch Peter Lynch
              Last Updated By:
              Peter Lynch
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title