Details
-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
None
-
3.19.0
-
Windows 10, MacOSX
-
5
-
NXRM MadMax Sprint 21
-
3
-
3
Description
I noticed that if I had a large layer deleted from my database to simulate a condition where I would need to run the "Repair - Reconcile component database from blob store" task, that it would error with an out of heap error (below) despite me having the recommended heap for my machine/hardware.
2019-08-01 18:30:43,816-0400 INFO [qtp341555250-206] admin org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'dockermac' [blobstore.rebuildComponentDB] state change WAITING -> RUNNING 2019-08-01 18:30:43,840-0400 INFO [quartz-3-thread-3] *SYSTEM org.sonatype.nexus.blobstore.restore.RestoreMetadataTask - Task log: /Users/jtom/Documents/Work/nexus/NX3/nexus-internal/target/sonatype-work/nexus3/log/tasks/blobstore.rebuildComponentDB-20190801183043832.log 2019-08-01 18:30:48,304-0400 INFO [quartz-3-thread-3] *SYSTEM com.sonatype.nexus.blobstore.restore.internal.DockerRestoreBlobStrategy - Restoring a v2 generated asset/component: v2/-/blobs/sha256:7fa8ea8ddbeedf89f7e47e9f8f0d7fdc7e238d1faf3d2b33b785a801bbcd5f23 2019-08-01 18:31:03,433-0400 ERROR [quartz-3-thread-3] *SYSTEM org.quartz.core.JobRunShell - Job nexus.70f68763-e7bc-4315-9313-cdb2502d9da7 threw an unhandled Exception: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3664) at java.lang.String.<init>(String.java:207) at java.lang.StringBuilder.toString(StringBuilder.java:407) at com.google.common.io.CharStreams.toString(CharStreams.java:164) at com.google.common.io.CharStreams$toString.call(Unknown Source) at org.sonatype.nexus.repository.docker.internal.V2ManifestUtilImpl$_manifestDigest_closure1.doCall(V2ManifestUtilImpl.groovy:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034) at groovy.lang.Closure.call(Closure.java:418) at groovy.lang.Closure.call(Closure.java:434) at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1630) at org.codehaus.groovy.runtime.NioGroovyMethods.withCloseable(NioGroovyMethods.java:1759) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54) at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:56) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) at org.sonatype.nexus.repository.docker.internal.V2ManifestUtilImpl.manifestDigest(V2ManifestUtilImpl.groovy:154) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.getV2Digest(DockerRestoreFacetImpl.java:281) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.getDigest(DockerRestoreFacetImpl.java:271) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.doSaveAsset(DockerRestoreFacetImpl.java:235) 2019-08-01 18:31:03,435-0400 ERROR [quartz-3-thread-3] *SYSTEM org.quartz.core.ErrorLogger - Job (nexus.70f68763-e7bc-4315-9313-cdb2502d9da7 threw an exception. org.quartz.SchedulerException: Job threw an unhandled exception. at org.quartz.core.JobRunShell.run(JobRunShell.java:213) at org.sonatype.nexus.quartz.internal.QuartzThreadPool.lambda$0(QuartzThreadPool.java:143) at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40) at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Arrays.java:3664) at java.lang.String.<init>(String.java:207) at java.lang.StringBuilder.toString(StringBuilder.java:407) at com.google.common.io.CharStreams.toString(CharStreams.java:164) at com.google.common.io.CharStreams$toString.call(Unknown Source) at org.sonatype.nexus.repository.docker.internal.V2ManifestUtilImpl$_manifestDigest_closure1.doCall(V2ManifestUtilImpl.groovy:155) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034) at groovy.lang.Closure.call(Closure.java:418) at groovy.lang.Closure.call(Closure.java:434) at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1630) at org.codehaus.groovy.runtime.NioGroovyMethods.withCloseable(NioGroovyMethods.java:1759) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.metaclass.ReflectionMetaMethod.invoke(ReflectionMetaMethod.java:54) at org.codehaus.groovy.runtime.metaclass.NewInstanceMetaMethod.invoke(NewInstanceMetaMethod.java:56) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) at org.sonatype.nexus.repository.docker.internal.V2ManifestUtilImpl.manifestDigest(V2ManifestUtilImpl.groovy:154) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.getV2Digest(DockerRestoreFacetImpl.java:281) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.getDigest(DockerRestoreFacetImpl.java:271) at org.sonatype.nexus.repository.docker.internal.DockerRestoreFacetImpl.doSaveAsset(DockerRestoreFacetImpl.java:235) 2019-08-01 18:31:03,439-0400 INFO [quartz-3-thread-3] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'dockermac' [blobstore.rebuildComponentDB] state change RUNNING -> WAITING (FAILED)
The test image I used was couchbase which currently has a ~400MB layer. I originally noticed testing a 1.6GB foreign layer but it is reproducable without that.
This may be an improvement (optimization) but I'm starting with bug.