Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-9705

expire cache task on rubygems repositories may lead to /api/v1/dependecies api performance degradation

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.12.0
    • Fix Version/s: 2.12.1
    • Component/s: RubyGems
    • Labels:
    • Story Points:
      1
    • Sprint:
      Sprint 61 - Chinook

      Description

      Problem

      Expiring Cache on a rubygems proxy repository can cause Nexus to make a single HEAD and GET request for every gem and dependency gem referenced by a single request listing gems to /api/v1/dependencies?gems=<list-of-gems>.

      Before the Expire cache task is run, Nexus will not do this. Instead it passes the request to the remote as received.

      For example:

      • request /api/v1/dependencies?gems=foo,bar
      • Nexus passes this single request as is, to any remote repository it is proxying
      • Expire Cache on the Nexus repository
      • sending the same request to Nexus causes Nexus to send both a HEAD and a GET as follows:
        HEAD /api/v1/dependencies?gems=foo
        GET /api/v1/dependencies?gems=foo
        HEAD /api/v1/dependencies?gems=bar
        GET /api/v1/dependencies?gems=bar

      For long lists of gems, the many additional outbound requests can lead to a significant drop in response time as compared to when the single request is passed through to the remote 'as is'.

      I've attached a jmeter test one can use the reproduce the issue. The attachment also contains a nexus log, conf directory, and csv format of performance data before and after expiring cache.

      The key log lines in the attached nexus.log are as follows:

      START
      2016-01-20 16:52:30,566-0400 DEBUG [qtp699358581-561]  org.eclipse.jetty.server.Server - REQUEST /nexus/content/groups/rubygems.org-group/api/v1/dependencies on AsyncHttpConnection@4b6fa6fb,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=-5,l=34,c=0},r=1
      
      STOP
      2016-01-20 16:56:17,246-0400 DEBUG [qtp699358581-556]  org.eclipse.jetty.server.Server - RESPONSE /nexus/content/groups/rubygems.org-group/api/v1/dependencies  200 handled=true
      
      EXPIRE CACHE TASK RUN
      
      TEST RUN 2 START
      2016-01-20 16:59:38,230-0400 DEBUG [qtp699358581-561]  org.eclipse.jetty.server.Server - REQUEST /nexus/content/groups/rubygems.org-group/api/v1/dependencies on AsyncHttpConnection@25b9627a,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=-5,l=34,c=0},r=1
      
      TEST RUN 2 STOP
      2016-01-20 17:06:11,864-0400 DEBUG [qtp699358581-265]  org.eclipse.jetty.server.Server - RESPONSE /nexus/content/groups/rubygems.org-group/api/v1/dependencies  200 handled=true
      
      

      As you can see, it takes 3m47s for the first run, the second run takes 6m33 seconds for the same set of 20 requests repeated 10 times.

        Attachments

          Activity

            People

            Assignee:
            plynch Peter Lynch
            Reporter:
            plynch Peter Lynch
            Last Updated By:
            Peter Lynch
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title