Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-35259

PyPI INDEX type asset is always created by accessing it via group repo

    XMLWordPrintable

Details

    • 2
    • NXRM Immortals Sprint 43, NXRM Immortals Sprint 44
    • 3
    • non-concept
    • 2

    Description

      ISSUE:

      Accessing any PyPI INDEX type asset via PyPI group repository always generate a new blob.
      This means any single pip command used by a user may generate potentially unnecessary blob files.

      REPRODUCE STEPS:

      1. Install NXRM 3.41.1 with H2 or PostgreSQL
      2. Create pypi-proxy (https://pypi.org), pypi-hosted, and pypi-group repositories, with default settings
      3. Accessing some index page via pypi-group, for example: 
        curl -I "http://localhost:8081/repository/pypi-group/simple/datadog-api-client/"
      4. Check how many properties files 
        find ./sonatype-work/nexus3/blobs/default/content/vol-* -name '*.properties' | wc -l
      5. Repeat the curl and find commands

      EXPECTED BEHAVIOUR:

      The pypi-group repo should serve the cached asset.

      ACTUAL BEHAVIOUR:

      Nexus always create a new set of .properties and .bytes files.
      With above example, it keeps adding a new .properties file for "@BlobStore.blob-name=/datadog-api-client" (not "/simple/datadog-api-client/").

      Also, the nexus.log always shows below "CacheInfo missing for ..."

      2022-09-15 18:20:08,182+1000 WARN  [qtp1051197544-106] *UNKNOWN com.sonatype.nexus.repository.pypi.datastore.internal.PyPiGroupFacet - CacheInfo missing for Content{payload=BlobPayload{blob=org.sonatype.nexus.blobstore.file.FileBlobStore$FileBlob@20d7c0a8, contentType='text/html'}, attributes='{etag=32ca90876f390475aa3d86bfe6704c9722ffe0fe, org.sonatype.nexus.repository.content.Asset=AssetData{assetId=7, path='/datadog-api-client', kind='INDEX', componentId=null, component=null, assetBlobId=11, assetBlob=AssetBlobData{assetBlobId=11, blobRef=default@09d5198d-68276d76-c5933475-c5a47000-2f524593:f3df8d4d-6839-4e24-ba2d-227ee763343c, blobSize=9263, contentType='text/html', checksums={sha1=32ca90876f390475aa3d86bfe6704c9722ffe0fe, sha256=b07d041721030cf278ad1a959d8cef23b928c53bf9f7c3712da5ec1bb8b4a6cf, md5=a8160020285bf0cea45acc6eb033a77d}, blobCreated=2022-09-15T08:16:49.804Z, createdBy='anonymous', createdByIp='127.0.0.1'}, lastDownloaded=2022-09-15T18:10:36.128+10:00} AbstractRepositoryContent{repositoryId=3, attributes=NestedAttributesMap{parent=null, key='attributes', backing=null}, created=2022-09-15T18:10:36.126+10:00, lastUpdated=2022-09-15T18:16:49.825+10:00}, last_modified=2022-09-15T18:16:49.804+10:00}'}, assuming stale content.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. NEXUS-35530 Maven REPOSITORY_METADATA type asset is always created by accessing it via group repo

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              mpiggott Matthew Piggott
              hosako Hajime Osako
              Michael Oliverio Michael Oliverio
              NXRM - Optimus
              Matthew Piggott Matthew Piggott
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                tigCommentSecurity.panel-title