Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-35259

PyPI INDEX type asset is always created by accessing it via group repo

    Details

    • Story Points:
      2
    • Sprint:
      NXRM Immortals Sprint 43, NXRM Immortals Sprint 44
    • Notability:
      3
    • InvestmentLayer:
      support-escalated
    • Aha Concept:
      non-concept

      Description

      ISSUE:

      Accessing any PyPI INDEX type asset via PyPI group repository always generate a new blob.
      This means any single pip command used by a user may generate potentially unnecessary blob files.

      REPRODUCE STEPS:

      1. Install NXRM 3.41.1 with H2 or PostgreSQL
      2. Create pypi-proxy (https://pypi.org), pypi-hosted, and pypi-group repositories, with default settings
      3. Accessing some index page via pypi-group, for example:
        curl -I "http://localhost:8081/repository/pypi-group/simple/datadog-api-client/"
      4. Check how many properties files
        find ./sonatype-work/nexus3/blobs/default/content/vol-* -name '*.properties' | wc -l
      5. Repeat the curl and find commands

      EXPECTED BEHAVIOUR:

      The pypi-group repo should serve the cached asset.

      ACTUAL BEHAVIOUR:

      Nexus always create a new set of .properties and .bytes files.
      With above example, it keeps adding a new .properties file for "@BlobStore.blob-name=/datadog-api-client" (not "/simple/datadog-api-client/").

      Also, the nexus.log always shows below "CacheInfo missing for ..."

      2022-09-15 18:20:08,182+1000 WARN  [qtp1051197544-106] *UNKNOWN com.sonatype.nexus.repository.pypi.datastore.internal.PyPiGroupFacet - CacheInfo missing for Content{payload=BlobPayload{blob=org.sonatype.nexus.blobstore.file.FileBlobStore$FileBlob@20d7c0a8, contentType='text/html'}, attributes='{etag=32ca90876f390475aa3d86bfe6704c9722ffe0fe, org.sonatype.nexus.repository.content.Asset=AssetData{assetId=7, path='/datadog-api-client', kind='INDEX', componentId=null, component=null, assetBlobId=11, assetBlob=AssetBlobData{assetBlobId=11, blobRef=default@09d5198d-68276d76-c5933475-c5a47000-2f524593:f3df8d4d-6839-4e24-ba2d-227ee763343c, blobSize=9263, contentType='text/html', checksums={sha1=32ca90876f390475aa3d86bfe6704c9722ffe0fe, sha256=b07d041721030cf278ad1a959d8cef23b928c53bf9f7c3712da5ec1bb8b4a6cf, md5=a8160020285bf0cea45acc6eb033a77d}, blobCreated=2022-09-15T08:16:49.804Z, createdBy='anonymous', createdByIp='127.0.0.1'}, lastDownloaded=2022-09-15T18:10:36.128+10:00} AbstractRepositoryContent{repositoryId=3, attributes=NestedAttributesMap{parent=null, key='attributes', backing=null}, created=2022-09-15T18:10:36.126+10:00, lastUpdated=2022-09-15T18:16:49.825+10:00}, last_modified=2022-09-15T18:16:49.804+10:00}'}, assuming stale content.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              mpiggott Matthew Piggott
              Reporter:
              hosako Hajime Osako
              Last Updated By:
              Matthew Piggott Matthew Piggott
              Team:
              NXRM - Optimus
              Owner:
              Matthew Piggott Matthew Piggott
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title