Details
Major Description
On migration from Nexus 2 to Nexus 3 instance, a customer was experiencing "low heap memory" ERROR log messages from OrientDB queries for one particular repository, despite Nexus 3 having the recommended memory settings. The messages suggest some database queries could not be executed and therefore migration was in an unknown state. Lack of trust in the the migration due to these messages is preventing this customer from migrating to Nexus 3 from Nexus 2. It is unclear what part of the migration actually failed.
In some cases low heap memory may eventually result in java.lang.OutOfMemoryError errors in the nexus log.
A heap dump was captured from Nexus 3 at the time of the ERRORS as well as logs and support zips.
Example nexus.log low heap memory ERROR
2018-10-02 19:51:08,781+0200 ERROR [event-9-thread-2670] admin com.google.common.eventbus.EventBus.nexus.async - Could not dispatch event EntityBatchEvent{events=[AssetUpdatedEvent{metadata=AttachedEntityMetadata{schema=asset, document=#62:63009{bucket:#42:5,format:maven2,last_updated:Tue Oct 02 17:41:09 CEST 2018,attributes:[5],component:#55:9514,name:com/example/tme/cpfs/cpfs-web/6.50/cpfs-web-6.50.war,size:51180644,content_type:application/java-archive,created_by:system,created_by_ip:system,blob_ref:default@1378CB2D-9B16B9BA-91CED78F-C2A150D7-42909953:3713a6b2-b81d-4208-8a39-c816a8cc09e1,last_downloaded:Tue Oct 02 17:41:09 CEST 2018,blob_created:Tue Oct 02 17:41:09 CEST 2018,blob_updated:Tue Oct 02 17:41:09 CEST 2018} v2}, remoteNodeId=null}]} to subscriber org.sonatype.nexus.repository.search.IndexRequestProcessor@411710d2 method [public void org.sonatype.nexus.repository.search.IndexRequestProcessor.on(org.sonatype.nexus.common.entity.EntityBatchEvent)]
com.orientechnologies.orient.core.exception.OCommandExecutionException: Cannot execute query "SELECT FROM asset WHERE bucket = #42:5 AND component = #55:9514": low heap memory
DB name="component"
at com.orientechnologies.orient.core.sql.OSoftQueryResultList.throwCanExecuteException(OSoftQueryResultList.java:688)
at com.orientechnologies.orient.core.sql.OSoftQueryResultList.checkQueue(OSoftQueryResultList.java:78)
at com.orientechnologies.orient.core.sql.OSoftQueryResultList.size(OSoftQueryResultList.java:88)
at java.util.Collections$SynchronizedCollection.size(Collections.java:2015)
at com.orientechnologies.orient.core.sql.query.OBasicResultSet.size(OBasicResultSet.java:66)
at com.orientechnologies.orient.core.sql.query.OBasicResultSet$1.hasNext(OBasicResultSet.java:86)
at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
at com.google.common.collect.Iterators$5.computeNext(Iterators.java:637)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
at org.sonatype.nexus.repository.search.DefaultComponentMetadataProducer.getMetadata(DefaultComponentMetadataProducer.java:86)
at org.sonatype.nexus.repository.search.SearchFacetImpl.json(SearchFacetImpl.java:202)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
at org.sonatype.nexus.repository.search.SearchServiceImpl.lambda$0(SearchServiceImpl.java:316)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.sonatype.nexus.repository.search.SearchServiceImpl.bulkPut(SearchServiceImpl.java:313)
at org.sonatype.nexus.repository.search.SearchFacetImpl.bulkPut(SearchFacetImpl.java:133)
at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:270)
at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
at org.sonatype.nexus.repository.search.IndexRequest.bulkApply(IndexRequest.java:92)
at org.sonatype.nexus.repository.search.IndexRequestProcessor.lambda$0(IndexRequestProcessor.java:126)
at java.util.Optional.ifPresent(Optional.java:159)
at org.sonatype.nexus.repository.search.IndexRequestProcessor.doUpdateSearchIndex(IndexRequestProcessor.java:122)
at org.sonatype.nexus.repository.search.IndexRequestProcessor.maybeUpdateSearchIndex(IndexRequestProcessor.java:102)
at java.util.HashMap.forEach(HashMap.java:1288)
at org.sonatype.nexus.repository.search.IndexBatchRequest.apply(IndexBatchRequest.java:93)
at org.sonatype.nexus.repository.search.IndexRequestProcessor.process(IndexRequestProcessor.java:90)
at org.sonatype.nexus.repository.search.IndexRequestProcessor.on(IndexRequestProcessor.java:86)
at sun.reflect.GeneratedMethodAccessor529.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
at org.sonatype.nexus.internal.event.AffinityBarrier.lambda$1(AffinityBarrier.java:91)
at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40)
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
