Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28666

Incorrect deployment handling for RPMs with same file name but with different headers.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: New
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.32.0
    • Fix Version/s: None
    • Component/s: Staging, Yum
    • Labels:
    • Notability:
      3

      Description

      Moving yum assets from one yum hosted to another repo may cause Browse Node Corruption.

      Suppose we have two RPM files with same filename as "helloworld-10-1000.el8.noarch.rpm" but with different rpm header/metadata. For example One of the RPM has release set to 1000 and other 2000

      Example:

      # rpm -qi helloworld
      
      Name : helloworld
      Version : 10
      Release : 1000.el8 ----> another RPM with same filename has '2000.el8'
      Architecture: noarch
      Install Date: Tue Aug 10 10:48:14 2021
      Group : Utilities
      Size : 22
      License : GPL
      Signature : (none)
      Source RPM : helloworld-10-1000.el8.src.rpm
      Build Date : Tue Aug 10 10:45:38 2021
      Build Host : fe4b7ef1c427
      Relocations : (not relocatable)
      URL : http://helloworld.example.com
      Summary : A test script
      Description :
      A test script inside a simple RPM package
      

      So in above scenario when moving asset from one yum repo to another can cause issues like user will not be able to download the desired asset as the filename is same for both the assets and it will cause browser node issue.

      Steps to reproduce:

      1). Create two Yum Hosted repositories as following and the deployment policy is set to 'Disable redeploy' (allow_once) for both of them.

      yum-releases
      yum-releases-prd

      2). Create a Tag

      TAG="test-tag"
      NEXUS_URL="http://localhost:8081"
      MOVE_FROM_REPO="yum-releases"
      MOVE_TO_REPO="yum-releases-prd"
      RPM_NAME="helloworld"
      
      cat > /tmp/tags.json << EOF
      {
       "name": "${TAG}",
       "attributes": {
       "someKey": "someValue"
       }
      }
      EOF
      
      curl -v -u admin:admin123 -X POST -d@/tmp/tags.json -H 'Content-Type: application/json' "${NEXUS_URL}/service/rest/v1/tags"
      

       
      3). Upload the 'RPM1/helloworld-10-1000.el8.noarch.rpm' to "yum-releases" repo and the n associate it with the above created Tag:

      curl -v -u admin:admin123 --upload-file ./RPM1/helloworld-10-1000.el8.noarch.rpm "${NEXUS_URL}/repository/${MOVE_FROM_REPO}/com/example/2021.2.100.005/helloworld-10-1000.el8.noarch.rpm"
      
      curl -v -u admin:admin123 -X POST  "${NEXUS_URL}/service/rest/v1/tags/associate/${TAG}?repository=${MOVE_FROM_REPO}&name=${RPM_NAME}"
      

      4). Get the SHA256 of the deployed asset which is deployed to "yum-releases"

      RPM_NAME_1_SHA256=$(curl -s -u 'admin:admin123' -X GET "${NEXUS_URL}/service/rest/v1/search/assets?q=name=${RPM_NAME}" | jq -r '.items[] | select(.repository == "yum-releases") | .checksum | .sha256')
      
      echo ${RPM_NAME_1_SHA256}
      

      5). Using Staging API move the above mentioned asset from "yum-releases" to "yum-releases-prd"

      curl -v -u admin:admin123 -v -X POST "${NEXUS_URL}/service/rest/v1/staging/move/${MOVE_TO_REPO}?repository=${MOVE_FROM_REPO}&tag=${TAG}&sha256=${RPM_NAME_1_SHA256}"
      
      {
        "status" : 200,
        "message" : "Move Successful",
        "data" : {
          "destination" : "yum-releases-prd",
          "components moved" : [ {
            "name" : "helloworld",
            "version" : "10-1000.el8"
          } ]
        }
      }
      

      6). Uploading another rpm to "yum-releases" (same rpm filename, different "2000.el8" value in metadata). Here we can find that RPM inside the attachment with "RPM2/helloworld-10-1000.el8.noarch.rpm". After uploading it to "yum-releases" associate it with tag.

      curl -v -u admin:admin123 --upload-file ./RPM2/helloworld-10-1000.el8.noarch.rpm "${NEXUS_URL}/repository/${MOVE_FROM_REPO}/com/example/2021.2.100.005/helloworld-10-1000.el8.noarch.rpm"
      
      curl -v -u admin:admin123 -X POST  "${NEXUS_URL}/service/rest/v1/tags/associate/${TAG}?repository=${MOVE_FROM_REPO}&name=${RPM_NAME}"
      

      7). Get the SHA256 of this newly deployed asset which is deployed to "yum-releases"

      RPM_NAME_2_SHA256=$(curl -s -u 'admin:admin123' -X GET "${NEXUS_URL}/service/rest/v1/search/assets?q=name=${RPM_NAME}" | jq -r '.items[] | select(.repository == "yum-releases") | .checksum | .sha256')
      
      echo ${RPM_NAME_2_SHA256}
      

      8). Promote same rpm using "/staging/move" API from "yum-releases" to "yum-releases-prd" using sha256 (RPM_NAME_2_SHA256) which we got in the above command.

      curl -v -u admin:admin123 -v -X POST "${NEXUS_URL}/service/rest/v1/staging/move/${MOVE_TO_REPO}?repository=${MOVE_FROM_REPO}&tag=${TAG}&sha256=${RPM_NAME_2_SHA256}"
      
      {
        "status" : 200,
        "message" : "Move Successful",
        "data" : {
          "destination" : "yum-releases-prd",
          "components moved" : [ {
            "name" : "helloworld",
            "version" : "10-2000.el8"
          } ]
        }
      }
      

      9). Notice that even if the above API execited successfully, it produced Browse Node Colision and hence we immediately see the following WARNING in the nexus.log.

      2021-08-12 11:49:08,269+1000 INFO  [event-13-thread-32]  admin org.sonatype.nexus.repository.yum.orient.internal.createrepo.OrientCreateRepoFacetImpl - Scheduling rebuild of yum metadata to start in 60 seconds
      2021-08-12 11:49:08,287+1000 INFO  [event-13-thread-36]  admin org.sonatype.nexus.repository.yum.orient.internal.createrepo.OrientCreateRepoFacetImpl - Scheduling rebuild of yum metadata to start in 60 seconds
      2021-08-12 11:49:11,578+1000 WARN  [event-13-thread-32]  admin org.sonatype.nexus.transaction.RetryController - Exceeded retry limit: 8/8 (org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeCollisionException: Node already has an asset
      	DB name="component")
      2021-08-12 11:49:11,584+1000 WARN  [event-13-thread-32]  admin org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager - Problem generating browse nodes for Asset{metadata=AttachedEntityMetadata{schema=asset, document=#44:3{bucket:#33:2,format:yum,last_updated:Thu Aug 12 11:49:08 AEST 2021,attributes:[4],component:#40:0,name:com/example/2021.2.100.005/helloworld-10-1000.el8.noarch.rpm,size:6716,content_type:application/x-rpm,created_by:admin,created_by_ip:127.0.0.1,blob_ref:default@CF74E985-1F1DC122-4817686F-588F9917-838A7B8A:3044a683-0097-491e-8941-c00e922ba77c,last_downloaded:null,blob_created:Thu Aug 12 11:47:04 AEST 2021,blob_updated:Thu Aug 12 11:47:04 AEST 2021} v1}, name=com/example/2021.2.100.005/helloworld-10-1000.el8.noarch.rpm}
      org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeCollisionException: Node already has an asset
      	DB name="component"
      	at org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeEntityAdapter.createAssetNode(BrowseNodeEntityAdapter.java:306)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeStore.lambda$1(OrientBrowseNodeStore.java:168)
      	at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$2(OrientOperations.java:62)
      	at org.sonatype.nexus.transaction.OperationPoint.lambda$0(OperationPoint.java:53)
      	at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64)
      	at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:57)
      	at org.sonatype.nexus.transaction.Operations.proceedWithTransaction(Operations.java:232)
      	at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:223)
      	at org.sonatype.nexus.transaction.Operations.run(Operations.java:175)
      	at org.sonatype.nexus.orient.transaction.OrientOperations.run(OrientOperations.java:62)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeStore.createAssetNode(OrientBrowseNodeStore.java:168)
      	at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
      	at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
      	at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:54)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager.createBrowseNodes(OrientBrowseNodeManager.java:137)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager.createFromAsset(OrientBrowseNodeManager.java:80)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeEventHandler.lambda$0(OrientBrowseNodeEventHandler.java:56)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeEventHandler.handle(OrientBrowseNodeEventHandler.java:86)
      	at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeEventHandler.on(OrientBrowseNodeEventHandler.java:56)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      

      Actual Behaviour:

      1. Above is the actual behaviour where we see that Nexus responded with 200 but later it caused "Problem generating browse nodes for Asset"
      2. User can not download the asset deployed as the URL is same for the RPM.

      Expected Behaviour:

      1. Either Browse Node Corruption should not have happened.

      2. The API call returning a success code (200) however it resulted in browse node collision, so nexus should have some mechanism to instruct the user or to avoid this scenario which can lead to browse node collision.

      3. After step8 users might believe that the staging move is successfully completed but now there is no way (curl / yum client / ui) to download the second asset which is moved to the "yum-releases-prd" repo.

      So even if the second asset is present in the database and can be searched using the asset search API it can not be downloaded with utilities like yum/curl/wget/nexus ui ..etc

      4. As the yum RPM file name is same so nexus should have stopped users request of moving it to the other repo with some error message.

        Attachments

        1. RPM1.zip
          2 kB
        2. RPM2.zip
          2 kB

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            jsensharma Jay Kumar SenSharma
            Last Updated By:
            Michael Oliverio Michael Oliverio
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Date of First Response:

                tigCommentSecurity.panel-title