Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-29547

already cached npm package tgz files are not served when request includes Accept header for abbreviated npm metadata and metadata cannot be cached

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.14.21
    • Fix Version/s: 2.15.0
    • Component/s: NPM
    • Labels:
    • Story Points:
      5
    • Sprint:
      NXRM MadMax Sprint 21, NXRM MadMax Sprint 22
    • Notability:
      3

      Description

      Problem

      Some older npm clients ( specifically "classic" yarn ) incorrectly per npm spec send the following header when making NPM TGZ file requests:

      Accept: application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*
      

      The header value is only relevant for npm package metadata requests, NOT tgz requests.

      Some known clients sending this header value on tgz requests are:

      • yarn/1.22.10 npm/? node/v14.13.0 darwin x64
      • yarn/1.22.10 npm/? node/v16.2.0 darwin x64
      • yarn/1.22.4 npm/? node/v12.14.1 linux x64
      • yarn/1.3.2 npm/? node/v10.2.0 darwin x64
      • yarn/1.7.0 npm/? node/v12.13.0 linux x64

      Repository 2 anticipates that tgz requests may arrive before the associated package metadata is cached ( implemented in NEXUS-8570 ). This was implemented a in PR 1278 / merged here

      • only checks remote metadata if package can't be found in cached metadata
      • handles new package situation
      • avoids unnecessary metadata fetch if package is in the cache
      • might still have unnecessary check if package is not in remote metadata
      • but again expiry check should help in that regard

      But, NEXUS-12821 is implemented to now store and serve two versions of a package metadata - the full metadata and abbreviated metadata.

      This creates the following scenario when the Accept header value indicates abbreviated metadata for tgz request:

      • npm tgz IS CACHED ALREADY in a proxy repo
      • proxy remote is not reachable or "Allow proxy" repo setting is set to False or already cached package metadata is expired ( item max age or cache invalidated )
      • when tgz request arrives for already cached tgz file for that package
      • then repo 2 checks for the abbreviated metadata contains that version
      • then since abbreviated metadata cannot be cached ( remote offline), repo 2 returns 404 not found for tgz request, failing builds

      Expected

      If the Accept header for a tgz request comes in with value that indicates abbreviated metadata, and package metadata needs to be checked before replying, don't return a 404 not found if the already cached full package metadata says the requested tgz version exists and the tgz is already cached locally.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dsawa Dawid Sawa
              Reporter:
              plynch Peter Lynch
              Last Updated By:
              Rich Seddon Rich Seddon
              Team:
              NXRM - Mad Max
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title