Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-34950

PyPI package versions published using twine before upgrading to 3.41.0 or later are missing from /simple index preventing discovery by clients

Details

    • 3
    • NXRM Immortals Sprint 42, NXRM Immortals Sprint 43, NXRM Immortals Sprint 44, NXRM Immortals Sprint 45, NXRM Optimus Sprint 47
    • 2
    • non-concept
    • 2

    Description

      ISSUE

      When a user uploads a PyPi asset via twine to a PyPi (hosted) repository using H2 or Postgres on a Nexus Repo version lower than 3.41.0, the pypi asset was generated inside repo with md5 checksum, not sha256.

      The problem is that when a user upgrade to the version 3.41.0 or 3.41.1 the Index via /simple is not being generated correctly, because that index uses the PyPi links generated by NXRM which in turn require the sha256 checksum (by default now as of 3.41.0 per fix version of NEXUS-24127) of the PyPi assets to create the PyPi links. So the result is all versions that were published without sha256 checksums do not appear in the /simple index page.

      The bug requires:

      • pypi packages uploaded via twine into Nexus Repo 3.40.1 or earlier
      • Nexus repo upgrade to 3.41.0 or 3.41.1
      • Nexus Repo must be using H2 or PostgreSQL database

      The issue does not affect:

      • Nexus Repo using OrientDB
      • new uploads for pypi package versions into 3.41.0 or greater

      NOTE: This issue do not happen if the database in use if OrientDB or if the user upload PyPi assets via twine from version 3.41.0 or higher

      REPRODUCE STEPS:

      1. Install NXRM 3.38.0 with H2 (or PostgreSQL) and with pypi-hosted
      2. With twine, upload some unique package "66483project" which does not exist in pypi-proxy (DO NOT USE nexus's upload page. Also, the reason I couldn't repro initially was I was uploading "twine" and "simpleproject" which both are existed in pypi-proxy)
      3. Access http://localhost:8081/repository/pypi-hosted/simple/66483project/, and confirm it shows the version you published
      4. Upgrade to 3.41.0
      5. From the Browse page for pypi-hosted, delete /simple/66483project if exists.
      6. Access http://localhost:8081/repository/pypi-hosted/simple/66483project/ again, and you don't see the version you published.

      Expected

      1. pypi packages and their versions that were published into Nexus Repo 3.40.1 and earlier , without sha256 hashes, still need to present in the /simple index, along with the packages published after upgrading to later product versions.
      2. Per the release note, Nexus users would expect all existing assets would have sha256 attributes.

        As recommended in PEP 503, hosted PyPI repositories now provide SHA256 hashes instead of MD5 in the /simple web interface's href attributes that link to package files being served.

      This may require a task to automatically run on upgrade to fix this for impacted customers.

      Attachments

        Issue Links

          Activity

            People

              mkearns Michael Kearns
              plynch Peter Lynch
              Ken Williams Ken Williams
              NXRM - Optimus
              Michael Kearns Michael Kearns
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                tigCommentSecurity.panel-title