-
Type:
Bug
-
Status: Closed
-
Priority:
Critical
-
Resolution: Incomplete
-
Affects Version/s: 3.3.0, 3.3.2
-
Fix Version/s: None
-
Labels:
When there are large numbers of Maven repositories in a Nexus 3 instance (~500+) processing of GET requests to group repositories becomes unusably slow.
Thread dump shows that this is not a deadlock, but it causes request threads to be serialized.
For example, this running thread:
qtp1995359069-302 id=302 state=RUNNABLE at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:285) at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWALPageChangesPortion.readData(OWALPageChangesPortion.java:227) at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWALPageChangesPortion.getIntValue(OWALPageChangesPortion.java:85) at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.getIntValue(ODurablePage.java:116) at com.orientechnologies.orient.core.storage.impl.local.paginated.OClusterPage.doDefragmentation(OClusterPage.java:434) at com.orientechnologies.orient.core.storage.impl.local.paginated.OClusterPage.appendRecord(OClusterPage.java:89) at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.addEntry(OPaginatedCluster.java:2005) at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.createSinglePageRecord(OPaginatedCluster.java:600) at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.createDataRecord(OPaginatedCluster.java:486) at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.createRecord(OPaginatedCluster.java:451) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doCreateRecord(OAbstractPaginatedStorage.java:3032) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitEntry(OAbstractPaginatedStorage.java:3652) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1439) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:539) at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:104) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2840) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2809) at org.sonatype.nexus.repository.storage.StorageTxImpl.commit(StorageTxImpl.java:171) at sun.reflect.GeneratedMethodAccessor336.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191) at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56) at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:66) at com.sun.proxy.$Proxy238.commit(Unknown Source) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:67) at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl$$EnhancerByGuice$$a9e791b8.doPut(<generated>) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl.put(MavenFacetImpl.java:200) at org.sonatype.nexus.repository.maven.internal.MavenFacetUtils.putWithHashes(MavenFacetUtils.java:213) at org.sonatype.nexus.repository.maven.internal.group.MavenGroupFacet.cache(MavenGroupFacet.java:158) at org.sonatype.nexus.repository.maven.internal.group.MavenGroupFacet.mergeAndCache(MavenGroupFacet.java:135) at org.sonatype.nexus.repository.maven.internal.group.MergingGroupHandler.doGet(MergingGroupHandler.java:98) at org.sonatype.nexus.repository.group.GroupHandler.handle(GroupHandler.java:79)
Is blocking 2 other threads with this stack:
qtp1995359069-325 id=325 state=WAITING - waiting on <0x3e092391> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) - locked <0x3e092391> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) owned by qtp1995359069-302 id=302 at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) at com.orientechnologies.common.concur.lock.OOneEntryPerKeyLockManager.acquireLock(OOneEntryPerKeyLockManager.java:169) at com.orientechnologies.common.concur.lock.OOneEntryPerKeyLockManager.acquireLock(OOneEntryPerKeyLockManager.java:101) at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.acquireReadLock(OAtomicOperationsManager.java:519) at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.iterateEntriesBetween(OSBTree.java:765) at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.iterateEntriesBetween(OSBTreeIndexEngine.java:185) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doIterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2064) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.iterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2052) at com.orientechnologies.orient.core.index.OIndexMultiValues.iterateEntriesBetween(OIndexMultiValues.java:275) at com.orientechnologies.orient.core.index.OIndexAbstractDelegate.iterateEntriesBetween(OIndexAbstractDelegate.java:104) at com.orientechnologies.orient.core.index.OIndexTxAwareMultiValue.iterateEntriesBetween(OIndexTxAwareMultiValue.java:339) at com.orientechnologies.orient.core.sql.operator.OQueryOperatorEquals.executeIndexQuery(OQueryOperatorEquals.java:173) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2178) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:997) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:209) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:526) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:508) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:484) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:74) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:2615) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:2561) at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:69) at org.sonatype.nexus.repository.storage.MetadataNodeEntityAdapter.findByProperty(MetadataNodeEntityAdapter.java:148) at org.sonatype.nexus.repository.storage.StorageTxImpl.findAssetWithProperty(StorageTxImpl.java:307) at sun.reflect.GeneratedMethodAccessor340.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:270) at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53) at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:63) at com.sun.proxy.$Proxy238.findAssetWithProperty(Unknown Source) at org.sonatype.nexus.repository.maven.internal.MavenFacetUtils.findAsset(MavenFacetUtils.java:132) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl.get(MavenFacetImpl.java:174) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl$$EnhancerByGuice$$a9e791b8.CGLIB$get$10(<generated>) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl$$EnhancerByGuice$$a9e791b8$$FastClassByGuice$$b1b9dd3a.invoke(<generated>) at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56) at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl$$EnhancerByGuice$$a9e791b8.get(<generated>) at org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet.getCachedContent(MavenProxyFacet.java:58) at org.sonatype.nexus.repository.proxy.ProxyFacetSupport.get(ProxyFacetSupport.java:166) at org.sonatype.nexus.repository.proxy.ProxyHandler.handle(ProxyHandler.java:49)
Full thread dump is attached.
- mentioned in
-
Page Loading...