Details
-
Type:
Bug
-
Status: Open
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 3.22.1
-
Fix Version/s: None
-
Component/s: HA, Repository
-
Labels:None
-
Notability:3
Description
There seems to be a race condition around repository deletion in an HA cluster. The symptoms are that the repository deletion succeeds on 2 nodes, but on the third node.
On the third node, a message comes in reassigning the master for the component database:
[2] 2020-06-21 04:00:40,529-0400 INFO [OrientDB DistributedWorker node=123D284A-D7B19FDB-91B6965B-1D35B162-33A361AB db=config id=0] *SYSTEM com.sonatype.nexus.hazelcast.internal.orient.SharedHazelcastPlugin - [123D284A-D7B19FDB-91B6965B-1D35B162-33A361AB] Reassigning ownership of clusters for database component...
And while this was being processed the repository delete failed because the database was temporarily in read only mode while processing the above:
[2] 2020-06-21 04:00:40,532-0400 ERROR [OrientDB DistributedWorker node=123D284A-D7B19FDB-91B6965B-1D35B162-33A361AB db=config id=0] *SYSTEM org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl - Failed to replicate: OrientConfigurationDeletedEvent{metadata=AttachedEntityMetadata{schema=repository, document=#54:9
Unknown macro: {repository_name}v1}, remoteNodeId=3D3249DB-D19BDFD3-78C7F2E5-FF5A1B03-46431717}
[2] org.sonatype.nexus.common.app.NotWritableException: Unable to delete repository when database is frozen.. Database not writable on node: 123D284A-D7B19FDB-91B6965B-1D35B162-33A361AB
[2] at org.sonatype.nexus.orient.internal.status.OrientStatusHealthCheckStore.checkWritable(OrientStatusHealthCheckStore.java:98)
[2] at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
[2] at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
[2] at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
[2] at org.sonatype.nexus.orient.internal.freeze.OrientFreezeService.checkWritable(OrientFreezeService.java:117)
[2] at org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl.delete(RepositoryManagerImpl.java:401)
[2] at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
[2] at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)