Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-26611

conda proxy repositories do not support inbound conditional HTTP GET HEAD

    XMLWordPrintable

    Details

    • Notability:
      2

      Description

      Conda proxy repositories do not support conditional GET or HEAD INBOUND HTTP requests. The lack of this support means performance is poor because for inbound requests, NXRM will always respond with 200 and content even if that content is not newer than the conditional HTTP headers.

      Reproduce

      1. Create a Conda Proxy repository named 'conda-proxy' to the official conda repository
      2. Create a Conda proxy repository named 'conda-proxy-proxy' with a remote url to your NXRM conda proxy repository created in step 1
      3. Request a package from conda-proxy-proxy repository. ie.

      curl -H 'If-Modified-Since: Thu, 03 Dec 2020 22:39:31 GMT' -v http://localhost:8081/repository/conda-proxy-proxy/main/linux-64/mkl-service-2.3.0-py39he8ac12f_0.tar.bz2 -o /dev/null
      

      4. NXRM makes an outbound request to conda-proxy and caches the package in into conda-proxy-proxy
      5. Now Invalidate cache in conda-proxy-proxy repo.
      6. Make another request for the same package. NXRM sends a conditional GET request to conda-proxy repo and that NXRM repo returns 200 response and the entire content instead of a 304 indicating that content has not changed.

      Note that the official conda repository properly supports conditional headers:

      This request:

      curl -H 'If-Modified-Since: Thu, 03 Dec 2020 22:39:31 GMT' -v https://repo.anaconda.com/pkgs/main/linux-64/mkl-service-2.3.0-py39he8ac12f_0.tar.bz2 -o /dev/null
      

      will return a 304 response as expected.

      Even for metadata this is a problem - note here actual requests that conda can send and NXRM incorrectly responds with 200.

      192.168.2.87 - - [05/Feb/2021:17:48:04 -0400] "GET /repository/conda-proxy-proxy/main/noarch/repodata.json HTTP/1.1" 200 - 3165291 46 "conda/4.6.11 requests/2.21.0 CPython/3.7.3 Linux/4.19.121-linuxkit debian/9 glibc/2.24" {Accept=*/*, Accept-Encoding=gzip, deflate, compress, identity, Connection=keep-alive, Content-Type=application/json, Host=192.168.2.87:8081, If-Modified-Since=Thu, 04 Feb 2021 21:06:13 GMT, If-None-Match=W/"3c212f885c018b9946a1c8725bb08b2c", User-Agent=conda/4.6.11 requests/2.21.0 CPython/3.7.3 Linux/4.19.121-linuxkit debian/9 glibc/2.24} [qtp554029079-73]
      192.168.2.87 - - [05/Feb/2021:17:48:05 -0400] "GET /repository/conda-proxy-proxy/main/linux-64/repodata.json HTTP/1.1" 200 - 25011375 370 "conda/4.6.11 requests/2.21.0 CPython/3.7.3 Linux/4.19.121-linuxkit debian/9 glibc/2.24" {Accept=*/*, Accept-Encoding=gzip, deflate, compress, identity, Connection=keep-alive, Content-Type=application/json, Host=192.168.2.87:8081, If-Modified-Since=Thu, 04 Feb 2021 22:55:34 GMT, If-None-Match=W/"80244e09dc776fd3607ac0d87d07e1f7-3", User-Agent=conda/4.6.11 requests/2.21.0 CPython/3.7.3 Linux/4.19.121-linuxkit debian/9 glibc/2.24} [qtp554029079-298]
      

      Expected

      Inbound requests containing valid If-Modified-Since or If-None-Match headers should be respected according to the content requested and the official HTTP specs for Conda format repositories.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dbradicich Damian Bradicich
              Reporter:
              plynch Peter Lynch
              Last Updated By:
              Damian Bradicich Damian Bradicich
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title