-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.6.2
-
Fix Version/s: 3.9.0
-
Component/s: Repository Health Check, Upgrade
-
Labels:
com.sonatype.nexus.plugins.healthcheck.service.impl.ConfigDatabaseUpgrade_1_4 is prone to failure.
An upgrade step should be:
- self-contained - it should use the low-level DB API to make changes, it should definitely not rely on any entity adapters (as they won't have been registered yet in the lifecycle)
- repeatable - applying the same upgrade step multiple times should not produce an error (it should check what it needs to do)
With ConfigDatabaseUpgrade_1_4 it's calling ConfigPropertyEntityAdapter.register(tx) which is bad, because not only does that apply the delta schema change that ConfigDatabaseUpgrade_1_4 originally intended to make - it also tries to apply the entire current schema which may include future changes post config-1.4.
Note that https://issues.sonatype.org/browse/NEXUS-13442 is the backlog ticket to make overall schema registration more robust.
Example failure 1
2018-01-30 01:51:54,987-0500 ERROR [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} Exception `32C8F4D8` in storage `config`
com.orientechnologies.orient.core.exception.OStorageException: File 'healthcheckconfig_property_name_idx.sbt' already exists.
DB name="config"
at com.orientechnologies.orient.core.storage.cache.local.OWOWCache.createFile(OWOWCache.java:1244)
at com.orientechnologies.orient.core.storage.cache.local.OWOWCache.addFile(OWOWCache.java:533)
at com.orientechnologies.orient.core.storage.cache.local.twoq.O2QCache.addFile(O2QCache.java:186)
at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperation.commitChanges(OAtomicOperation.java:415)
at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.endAtomicOperation(OAtomicOperationsManager.java:468)
at com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.endAtomicOperation(OAtomicOperationsManager.java:412)
at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.endAtomicOperation(ODurableComponent.java:116)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.create(OSBTree.java:152)
at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.create(OSBTreeIndexEngine.java:82)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.addIndexEngine(OAbstractPaginatedStorage.java:1992)
at com.orientechnologies.orient.core.index.OIndexAbstract.create(OIndexAbstract.java:243)
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:129)
at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:40)
at com.orientechnologies.orient.core.index.OIndexManagerShared.createIndex(OIndexManagerShared.java:169)
at com.orientechnologies.orient.core.index.OIndexManagerProxy.createIndex(OIndexManagerProxy.java:87)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1897)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1867)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1857)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.createIndex(OClassImpl.java:1853)
at com.sonatype.nexus.plugins.healthcheck.service.impl.ConfigPropertyEntityAdapter.defineType(ConfigPropertyEntityAdapter.java:69)
at org.sonatype.nexus.orient.entity.EntityAdapter.defineType(EntityAdapter.java:179)
at org.sonatype.nexus.orient.entity.EntityAdapter.register(EntityAdapter.java:152)
at org.sonatype.nexus.orient.entity.EntityAdapter.register(EntityAdapter.java:175)
at com.sonatype.nexus.plugins.healthcheck.service.impl.ConfigDatabaseUpgrade_1_4.apply(ConfigDatabaseUpgrade_1_4.java:73)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.lambda$3(UpgradeServiceImpl.java:195)
at java.util.ArrayList.forEach(Unknown Source)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.doUpgrade(UpgradeServiceImpl.java:137)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.doStart(UpgradeServiceImpl.java:93)
at org.sonatype.nexus.common.stateguard.StateGuardLifecycleSupport.start(StateGuardLifecycleSupport.java:67)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$9c5f1aa8.CGLIB$start$2(<generated>)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$9c5f1aa8$$FastClassByGuice$$9e083ad6.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76)
at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191)
at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55)
at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$9c5f1aa8.start(<generated>)
at org.sonatype.nexus.extender.NexusLifecycleManager.startComponent(NexusLifecycleManager.java:153)
at org.sonatype.nexus.extender.NexusLifecycleManager.to(NexusLifecycleManager.java:93)
at org.sonatype.nexus.extender.NexusContextListener.frameworkEvent(NexusContextListener.java:188)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1429)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Unknown Source)
Example failure 2, HA specific
2018-02-05 10:15:28,697-0600 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl - Upgrade config from 1.2 to 1.3 2018-02-05 10:15:28,727-0600 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl - Upgrade config from 1.3 to 1.4 2018-02-05 10:15:32,098-0600 WARN [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl - Problem upgrading config from 1.3 to 1.4 com.orientechnologies.orient.server.distributed.ODistributedException: Error on creating cluster 'healthcheckconfig' on distributed nodes: local and remote ids assigned are different at com.orientechnologies.orient.server.distributed.impl.ODistributedStorage.addCluster(ODistributedStorage.java:1615) at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.addCluster(ODatabaseDocumentTx.java:1383) at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClusters(OSchemaShared.java:1236) at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.doCreateClass(OSchemaShared.java:992) at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:379) at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:364) at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.createClass(OSchemaShared.java:249) at com.orientechnologies.orient.core.metadata.schema.OSchemaProxy.createClass(OSchemaProxy.java:67) at org.sonatype.nexus.orient.entity.EntityAdapter.register(EntityAdapter.java:152) at org.sonatype.nexus.orient.entity.EntityAdapter.register(EntityAdapter.java:176) at com.sonatype.nexus.plugins.healthcheck.service.impl.ConfigDatabaseUpgrade_1_4.apply(ConfigDatabaseUpgrade_1_4.java:80) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.lambda$3(UpgradeServiceImpl.java:195) at java.util.ArrayList.forEach(ArrayList.java:1249) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.doUpgrade(UpgradeServiceImpl.java:137) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl.doStart(UpgradeServiceImpl.java:93) at org.sonatype.nexus.common.stateguard.StateGuardLifecycleSupport.start(StateGuardLifecycleSupport.java:67) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$dbe05174.CGLIB$start$4(<generated>) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$dbe05174$$FastClassByGuice$$2ae549d6.invoke(<generated>) at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191) at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77) at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55) at org.sonatype.nexus.upgrade.internal.UpgradeServiceImpl$$EnhancerByGuice$$dbe05174.start(<generated>) at org.sonatype.nexus.extender.NexusLifecycleManager.startComponent(NexusLifecycleManager.java:155) at org.sonatype.nexus.extender.NexusLifecycleManager.to(NexusLifecycleManager.java:95) at org.sonatype.nexus.extender.NexusContextListener.frameworkEvent(NexusContextListener.java:189) at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1429) at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308) at java.lang.Thread.run(Thread.java:748)