Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-17896

concurrent requests for large npm metadata can lead to OutOfMemory during serialization

    XMLWordPrintable

    Details

    • Story Points:
      13

      Description

      1. Install nexus 3.13.0
      2. Create npm-proxy for https://registry.npmjs.org
      3. Create npm-group group repo containing npm-proxy as member.
      4. Make concurrent requests to GET http://localhost:8081/repository/npm-group/npm ( example of 20 concurrent requests was tested )
      At the time of this report, the anticipated response size was 8MB of JSON:

      127.0.0.1 - - [04/Sep/2018:13:17:45 -0300] "GET /repository/npm-group/npm HTTP/1.1" 200 - 8104610 53829 "Apache-HttpClient/4.5.3 (Java/1.8.0_152)"

      5. Within a short time frame, Nexus will start throwing OutOfMemory errors - either:
      OutOfMemoryError: Java heap space
      OutOfmemoryError: GC overhead limit exceeded

      The stack trace in the application log may take the following forms:

      2018-09-04 13:17:13,505-0300 ERROR [qtp1980459634-66] *SYSTEM org.sonatype.nexus.internal.web.ErrorPageServlet - Unexpected exception
      java.lang.OutOfMemoryError: Java heap space
      	at java.util.Arrays.copyOf(Arrays.java:3236)
      	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
      	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
      	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
      	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
      	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
      	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
      	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._flushBuffer(WriterBasedJsonGenerator.java:1946)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._writeString(WriterBasedJsonGenerator.java:973)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator.writeString(WriterBasedJsonGenerator.java:377)
      	at com.fasterxml.jackson.databind.ser.std.StringSerializer.serialize(StringSerializer.java:41)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
      	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
      	at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3893)
      	at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3189)
      	at org.sonatype.nexus.repository.npm.internal.NpmJsonUtils.serialize(NpmJsonUtils.java:97)
      	at org.sonatype.nexus.repository.npm.internal.NpmJsonUtils.bytes(NpmJsonUtils.java:112)
      	at org.sonatype.nexus.repository.npm.internal.NpmFacetUtils.toContent(NpmFacetUtils.java:154)
      	at org.sonatype.nexus.repository.npm.internal.NpmProxyFacetImpl.getPackageRoot(NpmProxyFacetImpl.java:222)
      
      
      2018-09-04 13:17:38,581-0300 ERROR [qtp1980459634-61] *SYSTEM org.sonatype.nexus.internal.web.ErrorPageServlet - Unexpected exception
      java.lang.OutOfMemoryError: Java heap space
      	at java.util.Arrays.copyOf(Arrays.java:3236)
      	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
      	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
      	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
      	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
      	at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
      	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
      	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._flushBuffer(WriterBasedJsonGenerator.java:1946)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator._writeFieldName(WriterBasedJsonGenerator.java:161)
      	at com.fasterxml.jackson.core.json.WriterBasedJsonGenerator.writeFieldName(WriterBasedJsonGenerator.java:139)
      	at com.fasterxml.jackson.databind.ser.std.StdKeySerializers$StringKeySerializer.serialize(StdKeySerializers.java:263)
      	at com.fasterxml.jackson.databind.ser.std.StdKeySerializers$Dynamic.serialize(StdKeySerializers.java:225)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:707)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
      	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:718)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:639)
      	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
      	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
      	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
      	at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3893)
      	at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3189)
      

      Other symptoms may include gradually slowing requests to the same package metadata.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                moncef Moncef Ben-Soula
                Reporter:
                plynch Peter Lynch
                Last Updated By:
                Joe Tom
                Team:
                NXRM - Cypher
              • Votes:
                3 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Date of First Response: