Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.14.3, 2.14.8
-
Fix Version/s: 2.14.9
-
Component/s: Repository
-
Labels:
-
Story Points:1
Description
org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap is a synchronized method allowing only one thread access at a time.
There are several triggers to get a repositories map per HTTP request into Nexus.
Example thread stacks which confluence around this method are:
qtp162721995-13491 id=13491 state=BLOCKED - waiting to lock <0x0e70f442> (a org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry) owned by qtp162721995-5951 id=5951 at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap(DefaultRepositoryRegistry.java:207) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepository(DefaultRepositoryRegistry.java:118) at org.sonatype.nexus.proxy.repository.AbstractGroupRepository.getMemberRepositories(AbstractGroupRepository.java:390) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getGroupsOfRepository(DefaultRepositoryRegistry.java:175) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:69) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.index.DefaultIndexArtifactFilter.filterArtifactInfo(DefaultIndexArtifactFilter.java:85) at org.sonatype.nexus.index.DefaultIndexerManager$18.accepts(DefaultIndexerManager.java:1854) at org.apache.maven.index.AndMultiArtifactInfoFilter.accepts(AndMultiArtifactInfoFilter.java:46) at org.apache.maven.index.AbstractMultiArtifactInfoFilter.accepts(AbstractMultiArtifactInfoFilter.java:80) at org.apache.maven.index.DefaultIteratorResultSet.createNextAi(DefaultIteratorResultSet.java:245) at org.apache.maven.index.DefaultIteratorResultSet.next(DefaultIteratorResultSet.java:153) at org.apache.maven.index.DefaultIteratorResultSet.next(DefaultIteratorResultSet.java:51) at org.sonatype.nexus.rest.index.AbstractIndexerNexusPlexusResource.ai2NaColl(AbstractIndexerNexusPlexusResource.java:67) at org.sonatype.nexus.rest.index.AbstractIndexPlexusResource.get(AbstractIndexPlexusResource.java:165) at org.sonatype.nexus.rest.index.DefaultIndexPlexusResource.get(DefaultIndexPlexusResource.java:86) at org.sonatype.plexus.rest.resource.RestletResource.represent(RestletResource.java:233)
qtp162721995-414 id=414 state=BLOCKED - waiting to lock <0x0e70f442> (a org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry) owned by qtp162721995-5951 id=5951 at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap(DefaultRepositoryRegistry.java:207) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepository(DefaultRepositoryRegistry.java:118) at org.sonatype.nexus.proxy.repository.AbstractGroupRepository.getMemberRepositories(AbstractGroupRepository.java:390) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getGroupsOfRepository(DefaultRepositoryRegistry.java:175) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:69) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.index.DefaultIndexArtifactFilter.filterArtifactInfo(DefaultIndexArtifactFilter.java:85) at org.sonatype.nexus.index.DefaultIndexerManager$18.accepts(DefaultIndexerManager.java:1854) at org.apache.maven.index.AndMultiArtifactInfoFilter.accepts(AndMultiArtifactInfoFilter.java:46) at org.apache.maven.index.AbstractMultiArtifactInfoFilter.accepts(AbstractMultiArtifactInfoFilter.java:80) at org.apache.maven.index.DefaultIteratorResultSet.createNextAi(DefaultIteratorResultSet.java:245) at org.apache.maven.index.DefaultIteratorResultSet.next(DefaultIteratorResultSet.java:153) at org.apache.maven.index.DefaultIteratorResultSet.next(DefaultIteratorResultSet.java:51) at org.sonatype.nexus.index.LockingIteratorResultSet.next(LockingIteratorResultSet.java:46) at org.sonatype.nexus.index.LockingIteratorResultSet.next(LockingIteratorResultSet.java:25) at org.sonatype.nexus.rest.indexng.SearchNGIndexPlexusResource.packSearchNGResponse(SearchNGIndexPlexusResource.java:338) at org.sonatype.nexus.rest.indexng.SearchNGIndexPlexusResource.get(SearchNGIndexPlexusResource.java:256) at org.sonatype.nexus.rest.indexng.SearchNGIndexPlexusResource.get(SearchNGIndexPlexusResource.java:66) at org.sonatype.plexus.rest.resource.RestletResource.represent(RestletResource.java:233) at org.sonatype.nexus.rest.NexusRestletResource.represent(NexusRestletResource.java:39)
qtp162721995-408 id=408 state=BLOCKED - waiting to lock <0x0e70f442> (a org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry) owned by qtp162721995-5951 id=5951 at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap(DefaultRepositoryRegistry.java:207) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepository(DefaultRepositoryRegistry.java:118) at org.sonatype.nexus.proxy.repository.AbstractGroupRepository.getMemberRepositories(AbstractGroupRepository.java:390) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getGroupsOfRepository(DefaultRepositoryRegistry.java:175) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:69) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.router.DefaultRepositoryRouter.authorizePath(DefaultRepositoryRouter.java:619) at org.sonatype.nexus.security.filter.authz.NexusTargetMappingAuthorizationFilter.isAccessAllowed(NexusTargetMappingAuthorizationFilter.java:160) at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162) at org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203) at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) at org.sonatype.nexus.web.internal.SecurityFilter.executeChain(SecurityFilter.java:90)
qtp162721995-1056 id=1056 state=BLOCKED - waiting to lock <0x0e70f442> (a org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry) owned by qtp162721995-5951 id=5951 at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap(DefaultRepositoryRegistry.java:207) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepository(DefaultRepositoryRegistry.java:118) at org.sonatype.nexus.proxy.repository.AbstractGroupRepository.getMemberRepositories(AbstractGroupRepository.java:390) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getGroupsOfRepository(DefaultRepositoryRegistry.java:175) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:69) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePathCascade(DefaultNexusItemAuthorizer.java:71) at org.sonatype.nexus.proxy.access.DefaultNexusItemAuthorizer.authorizePath(DefaultNexusItemAuthorizer.java:63) at org.sonatype.nexus.proxy.access.DefaultAccessManager.decide(DefaultAccessManager.java:46) at org.sonatype.nexus.proxy.repository.AbstractRepository.checkConditions(AbstractRepository.java:1194) at org.sonatype.nexus.proxy.repository.AbstractRepository.retrieveItem(AbstractRepository.java:590) at org.sonatype.nexus.proxy.router.DefaultRepositoryRouter.retrieveItem(DefaultRepositoryRouter.java:155) at org.sonatype.nexus.content.internal.ContentServlet.doGet(ContentServlet.java:386)
qtp162721995-1015 id=1015 state=BLOCKED - waiting to lock <0x0e70f442> (a org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry) owned by qtp162721995-5951 id=5951 at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepositoriesMap(DefaultRepositoryRegistry.java:207) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getRepository(DefaultRepositoryRegistry.java:118) at org.sonatype.nexus.proxy.repository.AbstractGroupRepository.getMemberRepositories(AbstractGroupRepository.java:390) at org.sonatype.nexus.proxy.registry.DefaultRepositoryRegistry.getGroupsOfRepository(DefaultRepositoryRegistry.java:175) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.includeRecipients(PublishEventInspector.java:268) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.includeRecipients(PublishEventInspector.java:271) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.includeRecipients(PublishEventInspector.java:271) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.includeRecipients(PublishEventInspector.java:271) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.recipients(PublishEventInspector.java:253) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.maybeFireItemUpdated(PublishEventInspector.java:354) at com.sonatype.nexus.plugins.smartproxy.event.internal.PublishEventInspector.on(PublishEventInspector.java:182) at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.sonatype.sisu.goodies.eventbus.internal.guava.EventHandler.handleEvent(EventHandler.java:80) at org.sonatype.sisu.goodies.eventbus.internal.guava.EventBus.dispatch(EventBus.java:329) at org.sonatype.sisu.goodies.eventbus.internal.DefaultGuavaEventBus.dispatch(DefaultGuavaEventBus.java:34) at org.sonatype.sisu.goodies.eventbus.internal.ReentrantGuavaEventBus.dispatchQueuedEvents(ReentrantGuavaEventBus.java:57) at org.sonatype.sisu.goodies.eventbus.internal.guava.EventBus.post(EventBus.java:281) at org.sonatype.sisu.goodies.eventbus.internal.DefaultEventBus.post(DefaultEventBus.java:78) at org.sonatype.nexus.proxy.repository.AbstractRepository.storeItem(AbstractRepository.java:1032) at org.sonatype.nexus.proxy.maven.AbstractMavenRepository.storeItem(AbstractMavenRepository.java:466) at org.sonatype.nexus.proxy.repository.AbstractRepository.storeItem(AbstractRepository.java:661) at org.sonatype.nexus.proxy.router.DefaultRepositoryRouter.storeItem(DefaultRepositoryRouter.java:179) at org.sonatype.nexus.content.internal.ContentServlet.doPut(ContentServlet.java:602) at org.sonatype.nexus.content.internal.ContentServlet.service(ContentServlet.java:357)
Since the method is synchronized and this method copies the entire list of repository members which may be large, in a heavily loaded system receiving combinations of the mentioned code paths, threads may backup to the point where Nexus may appear non-responsive due to a high number of blocked threads on the subject method.
This performance problem is most likely to surface under two primary conditions:
1) nested group repo hierarchy
2) group repos containing thousands of repositories