Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-18816

Slow delete performance when using REST API

    XMLWordPrintable

    Details

    • Story Points:
      3

      Description

      Deletions of yum components are taking 60 seconds and sometimes failing with 500.

      We can see database query reaching 60 second timeout.

      2019-01-10 04:51:29,684+0000 WARN  [qtp1990821672-1160244]  xxxxx org.sonatype.nexus.siesta.internal.UnexpectedExceptionMapper - (ID f3abb01b-7dda-4c74-8f03-4eeb03d78911) Unexpected exception: java.lang.IllegalStateException: Timed out reading query result from queue 11e742fa after 60 seconds

      java.lang.IllegalStateException: Timed out reading query result from queue 11e742fa after 60 seconds

       at org.sonatype.nexus.repository.storage.OrientAsyncHelper$QueueConsumingIterable.hasNext(OrientAsyncHelper.java:201)

       at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1811)

       at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)

       at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)

       at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)

       at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)

       at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)

       at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

       at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464)

       at org.sonatype.nexus.repository.browse.internal.BrowseServiceImpl.getById(BrowseServiceImpl.java:309)

       at org.sonatype.nexus.repository.browse.internal.BrowseServiceImpl.getComponentById(BrowseServiceImpl.java:282)

       at org.sonatype.nexus.repository.rest.internal.resources.ComponentsResource.getComponent(ComponentsResource.java:210)

       at org.sonatype.nexus.repository.rest.internal.resources.ComponentsResource.deleteComponent(ComponentsResource.java:228)

       

      We can also see the the following thread is taking time:

      event-6-thread-2281 - priority:5 - threadId:0x00007fe298055800 - nativeId:0x7e23 - nativeId (decimal):32291 - state:RUNNABLE
      stackTrace:
      java.lang.Thread.State: RUNNABLE
      at java.util.zip.Inflater.inflateBytes(Native Method)
      at java.util.zip.Inflater.inflate(Inflater.java:259)

      • locked <0x0000000707ae4788> (a java.util.zip.ZStreamRef)
        at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.read(GzipCompressorInputStream.java:311)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
      • locked <0x0000000707aec6a8> (a java.io.BufferedInputStream)
        at com.google.common.io.CountingInputStream.read(CountingInputStream.java:63)
        at org.sonatype.nexus.common.hash.MultiHashingInputStream.read(MultiHashingInputStream.java:66)
        at java.io.FilterInputStream.read(FilterInputStream.java:107)
        at com.google.common.io.ByteStreams.copy(ByteStreams.java:109)
        at org.sonatype.nexus.repository.yum.internal.utils.YumMetadataUtils.readCompressedMetadata(YumMetadataUtils.java:79)
        at org.sonatype.nexus.repository.yum.internal.utils.YumMetadataUtils.readCompressedMetadata(YumMetadataUtils.java:59)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.readMetadataAndAppendToRepomd(CreateRepoServiceImpl.java:371)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.writeRepomd(CreateRepoServiceImpl.java:355)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.convertDirectoriesToMetadata(CreateRepoServiceImpl.java:184)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.buildMetadata(CreateRepoServiceImpl.java:150)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.buildMetadata(CreateRepoServiceImpl.java:134)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl.buildMetadataUsingCaching(CreateRepoServiceImpl.java:120)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl$$EnhancerByGuice$$7fce96dd.CGLIB$buildMetadataUsingCaching$0(<generated>)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl$$EnhancerByGuice$$7fce96dd$$FastClassByGuice$$6d83d52f.invoke(<generated>)
        at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
        at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76)
        at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
        at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
        at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:77)
        at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:55)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoServiceImpl$$EnhancerByGuice$$7fce96dd.buildMetadataUsingCaching(<generated>)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl.buildMetadata(CreateRepoFacetImpl.java:190)
        at org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl.on(CreateRepoFacetImpl.java:175)
        at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
        at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:144)
      • locked <0x00000006cec7b2a8> (a com.google.common.eventbus.Subscriber$SynchronizedSubscriber)
        at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
        at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40)
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        Locked ownable synchronizers:
        <0x00000006ec0c4ee8> (a java.util.concurrent.ThreadPoolExecutor$Worker)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bradbeck Brad Beck
              Reporter:
              msurani Mahendra Surani
              Last Updated By:
              Peter Lynch
              Team:
              NXRM - Morpheus
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title