Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-26504

Newly created compact blob store task has "multinode=true"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.29.2
    • Fix Version/s: 3.30.0
    • Component/s: Scheduled Tasks
    • Labels:
    • Notability:
      2

      Description

      1. Start up Nexus Repo in HA-C mode with three nodes
      2. Create an "admin - compact blob store" task
      3. Run the task via scheduled start

      Observe that it runs on all three nodes. This causes concurrency issues, such as the one below.

      Expected: This task is not safe to run across nodes, it should have "multinode=false" in its configuration. We'll need an upgrade step to fix existing tasks.

       

      2021-01-25 12:30:00,074-0600 INFO [quartz-9-thread-2] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'Storage facet cleanup' [repository.storage-facet-cleanup] state change RUNNING -> WAITING (OK)
      2021-01-25 12:30:00,136-0600 INFO [quartz-9-thread-3] *SYSTEM org.sonatype.nexus.blobstore.compact.internal.CompactBlobStoreTask - Task log: /Users/rseddon/temp/foo/sonatype-work/nexus3/log/tasks/blobstore.compact-20210125123000132.log
      2021-01-25 12:30:00,139-0600 INFO [quartz-9-thread-3] *SYSTEM org.sonatype.nexus.blobstore.file.FileBlobStore - Begin deleted blobs processing
      2021-01-25 12:30:00,144-0600 INFO [quartz-9-thread-3] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'aaa' [blobstore.compact] state change RUNNING -> WAITING (OK)
      2021-01-25 12:30:02,514-0600 WARN [quartz-9-thread-3] *SYSTEM org.sonatype.nexus.transaction.RetryController - Exceeded retry limit: 8/8 (com.orientechnologies.orient.core.exception.OConcurrentModificationException: Cannot UPDATE the record #158:1 because the version is not the latest. Probably you are updating an old record or it has been modified by another user (db=v3 your=v2)
      DB name="config")
      2021-01-25 12:30:02,522-0600 WARN [quartz-9-thread-3] *SYSTEM org.sonatype.nexus.quartz.internal.orient.JobStoreImpl - Execution failed
      com.orientechnologies.orient.core.exception.OConcurrentModificationException: Cannot UPDATE the record #158:1 because the version is not the latest. Probably you are updating an old record or it has been modified by another user (db=v3 your=v2)
      DB name="config"
      at com.orientechnologies.orient.core.conflict.OVersionRecordConflictStrategy.checkVersions(OVersionRecordConflictStrategy.java:56)
      at com.orientechnologies.orient.core.conflict.OVersionRecordConflictStrategy.onUpdate(OVersionRecordConflictStrategy.java:43)
      at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkAndIncrementVersion(OAbstractPaginatedStorage.java:4545)
      at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateRecord(OAbstractPaginatedStorage.java:4022)
      at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitEntry(OAbstractPaginatedStorage.java:4607)
      at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1857)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedStorage.commit(ODistributedStorage.java:1384)
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:541)
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:99)
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2908)
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2870)
      at com.orientechnologies.orient.server.distributed.impl.task.OTxTask.execute(OTxTask.java:132)
      at com.orientechnologies.orient.server.distributed.impl.OSynchronizedTaskWrapper.execute(OSynchronizedTaskWrapper.java:70)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedAbstractPlugin$1.call(ODistributedAbstractPlugin.java:609)
      at com.orientechnologies.orient.core.db.OScenarioThreadLocal.executeAsDistributed(OScenarioThreadLocal.java:70)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedAbstractPlugin.executeOnLocalNode(ODistributedAbstractPlugin.java:605)
      at com.sonatype.nexus.hazelcast.internal.orient.SharedHazelcastPlugin.lambda$1(SharedHazelcastPlugin.java:101)
      at org.sonatype.nexus.orient.entity.EntityHook.asRemote(EntityHook.java:89)
      at com.sonatype.nexus.hazelcast.internal.orient.SharedHazelcastPlugin.executeOnLocalNode(SharedHazelcastPlugin.java:101)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedWorker.onMessage(ODistributedWorker.java:359)
      at com.orientechnologies.orient.server.distributed.impl.ODistributedWorker.run(ODistributedWorker.java:127)
      2021-01-25 12:30:02,536-0600 INFO [periodic-8-thread-1] *SYSTEM org.sonatype.nexus.rapture.internal.HealthCheckCacheManager - Health check status changed from true to false for Transactions
      2021-01-25 12:30:05,566-0600 INFO [Timer-2] *SYSTEM com.orientechnologies.orient.server.distributed.impl.OConflictResolverDatabaseRepairer - [964A20B0-D9952B83-818125DF-A2B1E24C-964BBC0D]->[[964A20B0-D9952B83-818125DF-A2B1E24C-964BBC0D, AC7C9998-5D48E1C7-E8E6F4EA-BFAB9F9E-42CEC05F, C77A3050-2FF62D03-D7CD5A94-0D24A900-5A19D31D]] Auto repairing completed. Sent 2 fix messages for 2 records (reqId=1.149)

       

       

        Attachments

          Activity

            People

            Assignee:
            mpiggott Matthew Piggott
            Reporter:
            rseddon Rich Seddon
            Last Updated By:
            Olu Shiyanbade Olu Shiyanbade
            Team:
            NXRM - Groot
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title