Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-18982

Repository reads can fail in HA cluster when backup task is running

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Parked
    • Affects Version/s: 3.13.0
    • Fix Version/s: None
    • Component/s: Backup, HA
    • Notability:
      n/a

      Description

      Read requests (HEAD, GET) to a hosted repository can fail in an HA cluster when a backup task is running.

      This occurs because a database write transaction can be initiated during retrieval:

      https://github.com/sonatype/nexus-internal/blob/release-3.13.0-01/plugins/nexus-repository-maven/src/main/java/org/sonatype/nexus/repository/maven/internal/MavenFacetImpl.java#L180

      When the last downloaded timestamp needs to be updated:

      https://github.com/sonatype/nexus-internal/blob/release-3.13.0-01/components/nexus-repository/src/main/java/org/sonatype/nexus/repository/storage/Asset.java#L229

      I suspect this could also fail in a single node (non-HA) instance, but I haven't tested it. 

      This problem was observed in 3.13.0.

       

      2019-01-25 05:23:49,854+0000 WARN [qtp891410167-269630] c0002762.prod.cloud.fedex.com *UNKNOWN org.sonatype.nexus.repository.view.handlers.ExceptionHandler - Nexus Repository Manager is in read-only mode: HEAD /com/foo/bar/fubar-1.0.0.zip: com.orientechnologies.orient.server.distributed.OWriteOperationNotPermittedException: Cannot execute write operation (Transaction Commit) on node '2B5F2EAA-26C150A1-1C3ED86B-F5AAFAA8-CF52EEBE' because is non a master
      2019-01-25 05:23:49,905+0000 ERROR [qtp891410167-269129] c0002762.prod.cloud.fedex.com *UNKNOWN com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPooled - Error on transaction commit `4E73F3DB`
      com.orientechnologies.orient.server.distributed.OWriteOperationNotPermittedException: Cannot execute write operation (Transaction Commit) on node '2B5F2EAA-26C150A1-1C3ED86B-F5AAFAA8-CF52EEBE' because is non a master
      at com.orientechnologies.orient.server.distributed.impl.ODistributedStorage.checkNodeIsMaster(ODistributedStorage.java:1948)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedStorage.commit(ODistributedStorage.java:1399)
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:541)
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:99)
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2908)
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2870)
      at org.sonatype.nexus.repository.storage.StorageTxImpl.commit(StorageTxImpl.java:181)
      at sun.reflect.GeneratedMethodAccessor148.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53)
      at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
      at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191)
      at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
      at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:66)
      at com.sun.proxy.$Proxy224.commit(Unknown Source)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              rseddon Rich Seddon
              Last Updated By:
              Rich Seddon Rich Seddon
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title