[NEXUS-19535] failing to obtain a repository configuration lock in 60 seconds can prevent content from being served from that repository - Sonatype JIRA

XML

Word

Printable

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.15.2, 3.21.1
Fix Version/s: None
Component/s: Configuration, Repository, Search
Labels:

Notability:
3

Description

When a user tries to edit a repository configuration, internally this operation is given 60 seconds to obtain a lock on the repository configuration to make changes. A long running search operation against that repository content may not relinquish that lock in 60 seconds.

When the lock attempt fails, the repository is left in a state where no content can be served from it until Nexus is restarted.

Symptoms

Example log messages when a user tried to edit an npm proxy repo:

2019-03-05 11:22:37,058+0000 INFO  [qtp725662768-40079]  admin org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl - Updating repository: npmjs-registry -> Configuration{repositoryName='npmjs-registry', recipeName='npm-proxy', attributes={proxy={remoteUrl=https://registry.npmjs.org, contentMaxAge=60.0, metadataMaxAge=60.0}, httpclient={blocked=false, autoBlock=true, connection={useTrustStore=false}}, storage={blobStoreName=default, strictContentTypeValidation=true}, negativeCache={enabled=true, timeToLive=60.0}, cleanup={policyName=None}}}
2019-03-05 11:23:39,632+0000 ERROR [qtp725662768-40079]  admin org.sonatype.nexus.repository.manager.internal.RepositoryImpl - Failed to stop facet: org.sonatype.nexus.repository.search.SearchFacetImpl$$EnhancerByGuice$$85e78f00@38b7b7f3
java.lang.RuntimeException: Failed to obtain lock after 60 seconds
 at org.sonatype.goodies.common.Locks.lock(Locks.java:43)
 at org.sonatype.goodies.common.Locks.write(Locks.java:65)
 at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:184)
 at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
 at org.sonatype.nexus.repository.manager.internal.RepositoryImpl.stop(RepositoryImpl.java:248)
 at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
 at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:193)
 at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
 at org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl.update(RepositoryManagerImpl.java:369)
 at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
 at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
 at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
 at org.sonatype.nexus.repository.manager.RepositoryManager$update$4.call(Unknown Source)
 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
 at org.sonatype.nexus.coreui.RepositoryComponent.update(RepositoryComponent.groovy:238)
 at com.palominolabs.metrics.guice.ExceptionMeteredInterceptor.invoke(ExceptionMeteredInterceptor.java:49)
 at com.palominolabs.metrics.guice.TimedInterceptor.invoke(TimedInterceptor.java:47)
 at org.sonatype.nexus.validation.internal.ValidationInterceptor.invoke(ValidationInterceptor.java:53)
 at org.apache.shiro.guice.aop.AopAllianceMethodInvocationAdapter.proceed(AopAllianceMethodInvocationAdapter.java:49)
 at org.apache.shiro.authz.aop.AuthorizingAnnotationMethodInterceptor.invoke(AuthorizingAnnotationMethodInterceptor.java:68)
 at org.apache.shiro.guice.aop.AopAllianceMethodInterceptorAdapter.invoke(AopAllianceMethodInterceptorAdapter.java:36)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeJavaMethod(DispatcherBase.java:142)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeMethod(DispatcherBase.java:133)
 at org.sonatype.nexus.extdirect.internal.ExtDirectDispatcher.invokeMethod(ExtDirectDispatcher.java:82)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.dispatch(DispatcherBase.java:63)
 at com.softwarementors.extjs.djn.router.processor.standard.StandardRequestProcessorBase.dispatchStandardMethod(StandardRequestProcessorBase.java:73)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.processIndividualRequest(JsonRequestProcessor.java:502)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.processIndividualRequestsInThisThread(JsonRequestProcessor.java:150)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.process(JsonRequestProcessor.java:133)
 at com.softwarementors.extjs.djn.router.RequestRouter.processJsonRequest(RequestRouter.java:83)
 at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.processRequest(DirectJNgineServlet.java:632)
 at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.doPost(DirectJNgineServlet.java:595)
 at org.sonatype.nexus.extdirect.internal.ExtDirectServlet.doPost(ExtDirectServlet.java:130)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
 at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
 at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
 at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
 at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
 at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
 at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
 at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
 at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
 at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
 at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
 at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
 at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
 at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
 at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
 at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
 at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.Server.handle(Server.java:531)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
 at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
 at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
 at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
 at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
 at java.lang.Thread.run(Thread.java:748)
 
2019-03-05 11:23:39,642+0000 ERROR [qtp725662768-40079]  admin org.sonatype.nexus.repository.manager.internal.RepositoryImpl - Failed transition: STARTED -> STOPPED
org.sonatype.goodies.common.MultipleFailures$MultipleFailuresException: Failed to stop facets; 1 failure
 at org.sonatype.goodies.common.MultipleFailures.maybePropagate(MultipleFailures.java:95)
 at org.sonatype.nexus.repository.manager.internal.RepositoryImpl.stop(RepositoryImpl.java:255)
 at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
 at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:193)
 at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
 at org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl.update(RepositoryManagerImpl.java:369)
 at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
 at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
 at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
 at org.sonatype.nexus.repository.manager.RepositoryManager$update$4.call(Unknown Source)
 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
 at org.sonatype.nexus.coreui.RepositoryComponent.update(RepositoryComponent.groovy:238)
 at com.palominolabs.metrics.guice.ExceptionMeteredInterceptor.invoke(ExceptionMeteredInterceptor.java:49)
 at com.palominolabs.metrics.guice.TimedInterceptor.invoke(TimedInterceptor.java:47)
 at org.sonatype.nexus.validation.internal.ValidationInterceptor.invoke(ValidationInterceptor.java:53)
 at org.apache.shiro.guice.aop.AopAllianceMethodInvocationAdapter.proceed(AopAllianceMethodInvocationAdapter.java:49)
 at org.apache.shiro.authz.aop.AuthorizingAnnotationMethodInterceptor.invoke(AuthorizingAnnotationMethodInterceptor.java:68)
 at org.apache.shiro.guice.aop.AopAllianceMethodInterceptorAdapter.invoke(AopAllianceMethodInterceptorAdapter.java:36)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeJavaMethod(DispatcherBase.java:142)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.invokeMethod(DispatcherBase.java:133)
 at org.sonatype.nexus.extdirect.internal.ExtDirectDispatcher.invokeMethod(ExtDirectDispatcher.java:82)
 at com.softwarementors.extjs.djn.router.dispatcher.DispatcherBase.dispatch(DispatcherBase.java:63)
 at com.softwarementors.extjs.djn.router.processor.standard.StandardRequestProcessorBase.dispatchStandardMethod(StandardRequestProcessorBase.java:73)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.processIndividualRequest(JsonRequestProcessor.java:502)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.processIndividualRequestsInThisThread(JsonRequestProcessor.java:150)
 at com.softwarementors.extjs.djn.router.processor.standard.json.JsonRequestProcessor.process(JsonRequestProcessor.java:133)
 at com.softwarementors.extjs.djn.router.RequestRouter.processJsonRequest(RequestRouter.java:83)
 at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.processRequest(DirectJNgineServlet.java:632)
 at com.softwarementors.extjs.djn.servlet.DirectJNgineServlet.doPost(DirectJNgineServlet.java:595)
 at org.sonatype.nexus.extdirect.internal.ExtDirectServlet.doPost(ExtDirectServlet.java:130)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
 at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
 at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
 at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
 at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
 at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
 at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
 at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
 at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
 at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
 at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
 at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
 at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
 at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
 at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
 at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
 at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
 at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
 at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
 at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
 at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
 at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at org.eclipse.jetty.server.Server.handle(Server.java:531)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
 at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
 at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
 at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
 at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
 at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
 at java.lang.Thread.run(Thread.java:748)
 Suppressed: java.lang.RuntimeException: Failed to obtain lock after 60 seconds
  at org.sonatype.goodies.common.Locks.lock(Locks.java:43)
  at org.sonatype.goodies.common.Locks.write(Locks.java:65)
  at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:184)
  at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
  at org.sonatype.nexus.repository.manager.internal.RepositoryImpl.stop(RepositoryImpl.java:248)
  ... 112 common frames omitted

Example subsequent errors that will continue trying to serve content:

2019-03-05 11:23:40,199+0000 WARN  [qtp725662768-39517]  *UNKNOWN org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: GET /nexus3/repository/npmjs-group/mkdirp/-/mkdirp-0.5.1.tgz
org.sonatype.nexus.common.stateguard.InvalidStateException: Invalid state: STOPPED; allowed: [STARTED]
 at org.sonatype.nexus.common.stateguard.StateGuard._ensure(StateGuard.java:115)
 at org.sonatype.nexus.common.stateguard.StateGuard.access$1(StateGuard.java:108)
 at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:271)
 at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
 at org.sonatype.nexus.repository.cache.NegativeCacheHandler.handle(NegativeCacheHandler.java:54)

Mitigation

A restart of repository manager should clear the invalid state of the repository affected.

Expected

Such a lock timeout should not completely disable serving content from the repository and not require a restart to escape from.

Attachments

Issue Links

is related to

NEXUS-13322 Lock timeout when removing docker connector leaves repository in invalid state

Closed

Activity

People

Assignee:: Unassigned

Reporter:: Peter Lynch

Last Updated By:: Joe Tom

Votes:: 4 Vote for this issue

Watchers:: 8 Start watching this issue

Dates

Created:: 03/28/19 02:25 PM

Updated:: 12/16/20 04:38 PM

Date of First Response:: 28/Mar/19 5:40 PM