Details
-
Bug
-
Resolution: Fixed
-
Major
-
3.39.0, 3.38.1
-
2
-
NXRM Immortals Sprint 39
-
3
-
support-escalated
-
non-concept
Description
Nexus upgrade can fail with NullPointerException as following:
2022-05-30 14:06:55,948+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1 - Found reconciliation logs for 1 blob stores to migrate 2022-05-30 14:06:55,960+1000 WARN [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Problem upgrading OrientBlobReconciliationLogMigrator from 1.0 to 1.1 java.lang.NullPointerException: null at org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1.lambda$7(OrientBlobReconciliationLogMigrator_1_1.java:142) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
Steps to Reproduce:
Step-1). Start Nexus 3.32.1 (or try using some other version of nexus)
DEMO_DIR="/Cases/Upgrade_Issue"
${DEMO_DIR}/nexus-3.32.1-01/bin/nexus start
tail -f ${DEMO_DIR}/sonatype-work/nexus3/log/*
Step-2). Creating raw-hosted repo and upload some assets to raw-hosted
curl -v -X 'POST' \ 'http://localhost:8081/service/rest/v1/repositories/raw/hosted' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "name": "raw-hosted", "online": true, "storage": { "blobStoreName": "default", "strictContentTypeValidation": true, "writePolicy": "allow_once" }, "cleanup": { "policyNames": [ ] }, "component": { "proprietaryComponents": true }, "raw": { "contentDisposition": "ATTACHMENT" } }' curl -u "admin:admin123" -T <(echo 'hello-1') "http://localhost:8081/repository/raw-hosted/test/hello-1.txt" curl -u "admin:admin123" -T <(echo 'hello-2') "http://localhost:8081/repository/raw-hosted/test/hello-2.txt" curl -u "admin:admin123" -T <(echo 'hello-3') "http://localhost:8081/repository/raw-hosted/test/hello-3.txt" curl -u "admin:admin123" -T <(echo 'hello-4') "http://localhost:8081/repository/raw-hosted/test/hello-4.txt" curl -u "admin:admin123" -T <(echo 'hello-5') "http://localhost:8081/repository/raw-hosted/test/hello-5.txt"
Step-3). Verify if the directories are created inside
ls -lart ${DEMO_DIR}/sonatype-work/nexus3/log/blobstore/default
Step-4). Create couple of blobstores so that we can promote "default"
curl -v -X 'POST' \ 'http://localhost:8081/service/rest/v1/blobstores/file' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "softQuota": { "type": "spaceRemainingQuota", "limit": "104857600" }, "path": "test-blobstore-A", "name": "test-blobstore-A" }' curl -v -X 'POST' \ 'http://localhost:8081/service/rest/v1/blobstores/file' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "softQuota": { "type": "spaceRemainingQuota", "limit": "104857600" }, "path": "test-blobstore-B", "name": "test-blobstore-B" }' curl -v -X 'POST' \ 'http://localhost:8081/service/rest/v1/blobstores/file' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "softQuota": { "type": "spaceRemainingQuota", "limit": "104857600" }, "path": "test-blobstore-C", "name": "test-blobstore-C" }'
Step-5). Promoting "default" blobstore to group blobstore and add members to it.
curl -v -X 'POST' \ 'http://localhost:8081/service/rest/v1/blobstores/group/promote/default' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '' curl -v -X 'PUT' \ 'http://localhost:8081/service/rest/v1/blobstores/group/default' \ -u admin:admin123 \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "softQuota": { "type": "spaceRemainingQuota", "limit": "104857600" }, "members": [ "test-blobstore-A", "test-blobstore-B", "default-promoted" ], "fillPolicy": "writeToFirst" }'
Step-6). Perform Upgrade to nexus nexus-3.39.0-01
${DEMO_DIR}/nexus-3.32.1-01/bin/nexus stop cd "${DEMO_DIR}" curl -LO https://download.sonatype.com/nexus/3/nexus-3.39.0-01-unix.tar.gz mv ${DEMO_DIR}/sonatype-work ${DEMO_DIR}/sonatype-work_ORIGINAL tar -xzvf nexus-3.39.0-01-unix.tar.gz rm -rf ${DEMO_DIR}/sonatype-work mv ${DEMO_DIR}/sonatype-work_ORIGINAL ${DEMO_DIR}/sonatype-work ${DEMO_DIR}/nexus-3.39.0-01/bin/nexus start tail -f ${DEMO_DIR}/sonatype-work/nexus3/log/*
Step-7). Notice the error in nexus.log
2022-05-30 14:06:55,275+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Start UPGRADE 2022-05-30 14:06:55,419+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - - - - - - - - - - - - - - - - - - - - - - - - - - Begin upgrade - - - - - - - - - - - - - - - - - - - - - - - - - 2022-05-30 14:06:55,419+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Checkpoint component 2022-05-30 14:06:55,642+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Checkpoint config 2022-05-30 14:06:55,776+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Checkpoint security 2022-05-30 14:06:55,863+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - - - - - - - - - - - - - - - - - - - - - - - - - - Apply upgrade - - - - - - - - - - - - - - - - - - - - - - - - - 2022-05-30 14:06:55,864+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Upgrade docker from 1.0 to 1.1 2022-05-30 14:06:55,875+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Upgrade component from 1.15 to 1.16 2022-05-30 14:06:55,902+1000 INFO [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.index.OIndexRebuildOutputListener - $ANSI{green {db=component}} - Rebuilding index component.asset_bucket_rid_idx (estimated 5 items)... 2022-05-30 14:06:55,907+1000 INFO [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.index.OIndexRebuildOutputListener - $ANSI{green {db=component}} --> OK, indexed 5 items in 6 ms 2022-05-30 14:06:55,947+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Upgrade OrientBlobReconciliationLogMigrator from 1.0 to 1.1 2022-05-30 14:06:55,948+1000 INFO [FelixStartLevel] *SYSTEM org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1 - Found reconciliation logs for 1 blob stores to migrate 2022-05-30 14:06:55,960+1000 WARN [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - Problem upgrading OrientBlobReconciliationLogMigrator from 1.0 to 1.1 java.lang.NullPointerException: null at org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1.lambda$7(OrientBlobReconciliationLogMigrator_1_1.java:142) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812) at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464) at org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1.getBlobStorePath(OrientBlobReconciliationLogMigrator_1_1.java:143) at org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1.copyFiles(OrientBlobReconciliationLogMigrator_1_1.java:114) at java.util.Optional.ifPresent(Optional.java:159) at org.sonatype.nexus.repository.upgrade.OrientBlobReconciliationLogMigrator_1_1.apply(OrientBlobReconciliationLogMigrator_1_1.java:96) at org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl.lambda$3(UpgradeServiceImpl.java:190) at java.util.ArrayList.forEach(ArrayList.java:1259) at org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl.doUpgrade(UpgradeServiceImpl.java:131) at org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl.doStart(UpgradeServiceImpl.java:96) at org.sonatype.nexus.common.stateguard.StateGuardLifecycleSupport.start(StateGuardLifecycleSupport.java:69) at org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl$$EnhancerByGuice$$304863509.GUICE$TRAMPOLINE(<generated>) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:74) at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:193) at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:57) at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:75) at com.google.inject.internal.InterceptorStackCallback.invoke(InterceptorStackCallback.java:55) at org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl$$EnhancerByGuice$$304863509.start(<generated>) at org.sonatype.nexus.extender.NexusLifecycleManager.startComponent(NexusLifecycleManager.java:199) at org.sonatype.nexus.extender.NexusLifecycleManager.to(NexusLifecycleManager.java:111) at org.sonatype.nexus.extender.NexusContextListener.moveToPhase(NexusContextListener.java:319) at org.sonatype.nexus.extender.NexusContextListener.frameworkEvent(NexusContextListener.java:216) at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1597) at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308) at java.lang.Thread.run(Thread.java:748) 2022-05-30 14:06:55,962+1000 WARN [FelixStartLevel] *SYSTEM org.sonatype.nexus.upgrade.internal.orient.UpgradeServiceImpl - - - - - - - - - - - - - - - - - - - - - - - - - - Rollback upgrade - - - - - - - - - - - - - - - - - - - - - - - - -
Workaround:
1. Move the dir "sonatype-work/nexus3/log/blobstore/default" to the original blobstore name (eg default-original).
# mv ${DEMO_DIR}/sonatype-work/nexus3/log/blobstore/default ${DEMO_DIR}/sonatype-work/nexus3/log/blobstore/default-original
2. Perform the upgrade again (start nexus 3.39.1)