-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.15.2, 3.16.1
-
Fix Version/s: 3.18.0
-
Component/s: Browse Storage
-
Labels:
When NXRM contains over 100 content selectors matching similar paths, it has been reported that Browse operations can become a bottleneck for non-admin users, often having UI operations taking longer than the default of 60 seconds before the UI Standard request timeout is reached and the UI reports a timeout.
The bottleneck seems to be associated with database query performance and code rooted in org.sonatype.nexus.repository.storage.BrowseNodeEntityAdapter.getByPath and org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.getByPath.
Example thread activity that may be seen during the processing of these slow requests:
Example 3.16.1 thread processing Browse request with NXRM containing 418 "@class": "selector_selector"
qtp1419760197-754 <command>sql.select from browse_node where (repository_name=:repository_name and parent_path>:base_path and parent_path<:base_boundary) and (asset_id is not null and ((asset_id.format = :s0p0 and asset_id.name matches :s0p1) or (asset_id.format = :s1p0 and asset_id.name matches :s1p1))) limit 1 timeout 59000 return</command>" #754 prio=5 os_prio=0 tid=0x00007f96dc02d000 nid=0x6f84 runnable [0x00007f95f7bdd000] java.lang.Thread.State: RUNNABLE at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.isClosed(OAbstractPaginatedStorage.java:536) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.isClosed(ODatabaseDocumentTx.java:1371) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.checkOpeness(ODatabaseDocumentTx.java:3175) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.getMetadata(ODatabaseDocumentTx.java:838) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.getMetadata(ODatabaseDocumentTx.java:103) at com.orientechnologies.orient.core.record.impl.ODocument.getSchemaClass(ODocument.java:1979) at com.orientechnologies.orient.core.sql.filter.OSQLFilterItemField.getBinaryField(OSQLFilterItemField.java:152) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:373) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:88) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:384) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:88) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:384) at com.orientechnologies.orient.core.sql.filter.OSQLFilterCondition.evaluate(OSQLFilterCondition.java:88) at com.orientechnologies.orient.core.sql.filter.OSQLFilter.evaluate(OSQLFilter.java:105) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.evaluateRecord(OCommandExecutorSQLResultsetAbstract.java:414) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.filter(OCommandExecutorSQLResultsetAbstract.java:404) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearchRecord(OCommandExecutorSQLSelect.java:609) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.serialIterator(OCommandExecutorSQLSelect.java:1638) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:1585) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchValuesFromIndexCursor(OCommandExecutorSQLSelect.java:2466) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2280) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:1017) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:203) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:527) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:509) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:485) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:70) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3400) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:3318) at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:69) at org.sonatype.nexus.repository.storage.BrowseNodeEntityAdapter.getByPath(BrowseNodeEntityAdapter.java:384) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.lambda$10(BrowseNodeStoreImpl.java:243) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$Lambda$456/78361960.apply(Unknown Source) at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$1(OrientOperations.java:56) at org.sonatype.nexus.orient.transaction.OrientOperations$$Lambda$166/1468844470.call(Unknown Source) at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56) at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:200) at org.sonatype.nexus.transaction.Operations.call(Operations.java:146) at org.sonatype.nexus.orient.transaction.OrientOperations.call(OrientOperations.java:56) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.getByPath(BrowseNodeStoreImpl.java:242) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.getByPath(BrowseNodeStoreImpl.java:206) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$3a9f08f.CGLIB$getByPath$3(<generated>) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$3a9f08f$$FastClassByGuice$$4a93b4c9.invoke(<generated>) at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272) at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$3a9f08f.getByPath(<generated>) at org.sonatype.nexus.repository.storage.BrowseNodeStore$getByPath.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47) at org.sonatype.nexus.repository.storage.BrowseNodeStore$getByPath.call(Unknown Source) at org.sonatype.nexus.coreui.BrowseComponent.read(BrowseComponent.groovy:75) at org.sonatype.nexus.coreui.BrowseComponent$$EnhancerByGuice$$9336241c.CGLIB$read$13(<generated>) at org.sonatype.nexus.coreui.BrowseComponent$$EnhancerByGuice$$9336241c$$FastClassByGuice$$69e1b711.invoke(<generated>) at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76) at com.palominolabs.metrics.guice.ExceptionMeteredInterceptor.invoke(ExceptionMeteredInterceptor.java:23) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.palominolabs.metrics.guice.TimedInterceptor.invoke(TimedInterceptor.java:26) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.coreui.BrowseComponent$$EnhancerByGuice$$9336241c.read(<generated>)
Example 3.15.2-01 thread processing browse request with 299 "@class": "selector_selector"
qtp2141801567-227581 <command>sql.select from browse_node where (repository_name=:repository_name and parent_path>=:subtree_boundary and parent_path<:base_boundary) and (asset_id is not null and ((asset_id.name matches :s0p0) or (asset_id.name matches :s1p0) or (asset_id.name matches :s2p0) or (asset_id.name matches :s3p0) or (asset_id.name matches :s4p0) or (asset_id.name matches :s5p0) or (asset_id.name matches :s6p0) or (asset_id.name matches :s7p0) or (asset_id.name matches :s8p0) or (asset_id.name matches :s9p0) or (asset_id.name matches :s10p0) or (asset_id.name matches :s11p0) or (asset_id.name matches :s12p0) or (asset_id.name matches :s13p0) or (asset_id.name matches :s14p0) or (asset_id.name matches :s15p0) or (asset_id.name matches :s16p0) or (asset_id.name matches :s17p0) or (asset_id.name matches :s18p0) or (asset_id.name matches :s19p0) or (asset_id.name matches :s20p0) or (asset_id.name matches :s21p0) or (asset_id.name matches :s22p0) or (asset_id.name matches :s23p0) or (asset_id.name matches :s24p0) or (asset_id.name matches :s25p0) or (asset_id.name matches :s26p0) or (asset_id.name matches :s27p0) or (asset_id.name matches :s28p0) or (asset_id.name matches :s29p0) or (asset_id.name matches :s30p0) or (asset_id.name matches :s31p0) or (asset_id.name matches :s32p0) or (asset_id.name matches :s33p0) or (asset_id.name matches :s34p0) or (asset_id.name matches :s35p0) or (asset_id.name matches :s36p0) or (asset_id.name matches :s37p0) or (asset_id.name matches :s38p0) or (asset_id.name matches :s39p0) or (asset_id.name matches :s40p0) or (asset_id.name matches :s41p0) or (asset_id.name matches :s42p0) or (asset_id.name matches :s43p0) or (asset_id.name matches :s44p0) or (asset_id.name matches :s45p0) or (asset_id.name matches :s46p0) or (asset_id.name matches :s47p0) or (asset_id.name matches :s48p0) or (asset_id.name matches :s49p0) or (asset_id.name matches :s50p0) or (asset_id.name matches :s51p0) or (asset_id.name matches :s52p0) or (asset_id.name matches :s53p0) or (asset_id.name matches :s54p0) or (asset_id.name matches :s55p0) or (asset_id.name matches :s56p0) or (asset_id.name matches :s57p0) or (asset_id.name matches :s58p0) or (asset_id.name matches :s59p0) or (asset_id.name matches :s60p0) or (asset_id.name matches :s61p0) or (asset_id.name matches :s62p0) or (asset_id.name matches :s63p0) or (asset_id.name matches :s64p0) or (asset_id.name matches :s65p0) or (asset_id.name matches :s66p0) or (asset_id.name matches :s67p0) or (asset_id.name matches :s68p0) or (asset_id.name matches :s69p0) or (asset_id.name matches :s70p0) or (asset_id.name matches :s71p0) or (asset_id.name matches :s72p0) or (asset_id.name matches :s73p0) or (asset_id.name matches :s74p0) or (asset_id.name matches :s75p0) or (asset_id.name matches :s76p0) or (asset_id.name matches :s77p0))) limit 1 timeout 58435 return</command> id=227581 state=RUNNABLE at com.orientechnologies.common.concur.lock.ODistributedCounter.increment(ODistributedCounter.java:46) at com.orientechnologies.common.concur.lock.OReadersWriterSpinLock.acquireReadLock(OReadersWriterSpinLock.java:78) at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.release(O2QCache.java:446) at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.releasePage(ODurableComponent.java:166) at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.access$3300(OSBTree.java:69) at com.orientechnologies.orient.core.index.sbtree.local.OSBTree$OSBTreeCursorForward.next(OSBTree.java:2044) at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine$OSBTreeIndexCursor.nextEntry(OSBTreeIndexEngine.java:263) at com.orientechnologies.orient.core.index.OIndexAbstractCursor.next(OIndexAbstractCursor.java:97) at com.orientechnologies.orient.core.index.OIndexAbstractCursor.next(OIndexAbstractCursor.java:34) at com.orientechnologies.orient.core.index.OIndexChangesWrapper.next(OIndexChangesWrapper.java:154) at com.orientechnologies.orient.core.index.OIndexChangesWrapper.next(OIndexChangesWrapper.java:18) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.serialIterator(OCommandExecutorSQLSelect.java:1637) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:1585) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchValuesFromIndexCursor(OCommandExecutorSQLSelect.java:2466) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2280) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:1017) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:203) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:527) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:509) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:485) at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:70) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3400) at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:3318) at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:69) at org.sonatype.nexus.repository.storage.BrowseNodeEntityAdapter.getByPath(BrowseNodeEntityAdapter.java:414) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.lambda$10(BrowseNodeStoreImpl.java:249) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$Lambda$355/697800031.apply(Unknown Source) at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$1(OrientOperations.java:56) at org.sonatype.nexus.orient.transaction.OrientOperations$$Lambda$134/946596238.call(Unknown Source) at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56) at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:200) at org.sonatype.nexus.transaction.Operations.call(Operations.java:146) at org.sonatype.nexus.orient.transaction.OrientOperations.call(OrientOperations.java:56) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.getByPath(BrowseNodeStoreImpl.java:248) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.lambda$6(BrowseNodeStoreImpl.java:204) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$Lambda$351/29735908.apply(Unknown Source) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1351) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl.getByPath(BrowseNodeStoreImpl.java:209) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$ab06f26f.CGLIB$getByPath$6(<generated>) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$ab06f26f$$FastClassByGuice$$d55799fd.invoke(<generated>) at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272) at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.repository.storage.BrowseNodeStoreImpl$$EnhancerByGuice$$ab06f26f.getByPath(<generated>) at org.sonatype.nexus.repository.storage.BrowseNodeStore$getByPath.call(Unknown Source) at org.sonatype.nexus.coreui.BrowseComponent.read(BrowseComponent.groovy:82) at org.sonatype.nexus.coreui.BrowseComponent$$EnhancerByGuice$$4d527c25.CGLIB$read$13(<generated>) at org.sonatype.nexus.coreui.BrowseComponent$$EnhancerByGuice$$4d527c25$$FastClassByGuice$$85c1779c.invoke(<generated>)
- mentioned in
-
Page Loading...