Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-14174

Component Database got corrupted after OOM exception.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 3.0.0, 3.4.0, 3.6.0
    • Fix Version/s: None
    • Component/s: Database
    • Labels:
    • Story Points:
      5
    • Sprint:
      Core Sprint 103, Core Sprint 104

      Description

      After an OutOfMemoryError, the component database was corrupted. A "repair database --fix-links" and "rebuild index *" was required to fix the database.

      2017-08-25 13:49:31,045+0800 ERROR [Thread-14954] xxxxxx com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=component}} Exception `675151B2` in storage `component`
      java.lang.OutOfMemoryError: GC overhead limit exceeded
      	at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.doLoad(O2QCache.java:328)
      	at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.load(O2QCache.java:291)
      	at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.loadPage(ODurableComponent.java:148)
      	at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.loadPage(ODurableComponent.java:142)
      	at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.access$2900(OSBTree.java:75)
      	at com.orientechnologies.orient.core.index.sbtree.local.OSBTree$OSBTreeCursorForward.next(OSBTree.java:2081)
      	at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine$OSBTreeIndexCursor.nextEntry(OSBTreeIndexEngine.java:271)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract$IndexValuesIterator.hasNext(OCommandExecutorSQLResultsetAbstract.java:114)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.serialIterator(OCommandExecutorSQLSelect.java:1617)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:1566)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:525)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:488)
      	at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:74)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3200)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:3138)
      	at com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:78)
      	at com.orientechnologies.orient.core.sql.query.OSQLAsynchQuery.run(OSQLAsynchQuery.java:74)
      	at com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:33)
      	at com.orientechnologies.orient.core.sql.query.OSQLNonBlockingQuery$1.run(OSQLNonBlockingQuery.java:282)
      	at java.lang.Thread.run(Thread.java:748)
      
      2017-08-25 15:21:09,799+0800 ERROR [qtp107108037-228] *UNKNOWN com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=component}} Exception `2522253C` in storage `component`
      com.orientechnologies.orient.core.exception.OStorageException: Error during transaction commit
      	DB name="component"
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.makeRollback(OAbstractPaginatedStorage.java:2678)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1699)
      	at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:541)
      	at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:99)
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2891)
      	at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2860)
      	at org.sonatype.nexus.repository.storage.StorageTxImpl.commit(StorageTxImpl.java:173)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53)
      	at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
      	at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191)
      	at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
      	at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:66)
      	at com.sun.proxy.$Proxy206.commit(Unknown Source)
      	at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:67)
      	at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
      	at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77)
      	at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55)
      	at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl$$EnhancerByGuice$$4592afb1.doPut(<generated>)
      	at org.sonatype.nexus.repository.maven.internal.MavenFacetImpl.put(MavenFacetImpl.java:201)
      	at org.sonatype.nexus.repository.maven.internal.proxy.MavenProxyFacet.store(MavenProxyFacet.java:88)
      ...
      Caused by: java.lang.NullPointerException: null
      	at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.findBucket(OSBTree.java:1765)
      	at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.get(OSBTree.java:200)
      	at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.get(OSBTreeIndexEngine.java:128)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doGetIndexValue(OAbstractPaginatedStorage.java:2115)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.getIndexValue(OAbstractPaginatedStorage.java:2090)
      	at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:157)
      	at com.orientechnologies.orient.core.index.OIndexMultiValues.put(OIndexMultiValues.java:44)
      	at com.orientechnologies.orient.core.index.OIndexAbstract.putInSnapshot(OIndexAbstract.java:956)
      	at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:790)
      	at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:762)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitIndexes(OAbstractPaginatedStorage.java:1730)
      	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1688)
      	... 187 common frames omitted
      

      There have been many timeout errors throughout the day:

      2017-08-25 13:47:53,376+0800 ERROR [qtp1295782900-18724] xxxxxx org.sonatype.nexus.extdirect.internal.ExtDirectServlet - Failed to invoke action method: coreui_Component.readAssets, java-method: org.sonatype.nexus.coreui.ComponentComponent.readAssets
      java.lang.IllegalStateException: Timed out reading query result from queue 1b622e46 after 60 seconds
      	at org.sonatype.nexus.repository.storage.OrientAsyncHelper$QueueConsumingIterable.hasNext(OrientAsyncHelper.java:201)
      

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            sohri Sam Ohri
            Last Updated By:
            Peter Lynch
            Votes:
            5 Vote for this issue
            Watchers:
            13 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title