Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-31201

Repair - Reconcile component database from blob store task run on H2/PostgreSQL results in creating duplicate blobs in blobstore

Details

    • 5
    • NXRM Sentinels Sprint 28, NXRM Sentinels Sprint 29
    • 2
    • non-concept
    • 2

    Description

      Running the "Repair - Reconcile component database from blob store" task on newDB results in "duplicate" blobs being generated.

      To reproduce:

      1.  On a fresh Nexus 3.37.3 instance running orientDB and default settings, upload a component to a repo.

      2. Once uploaded, stop the instance and via the database delete the assets and component e.g.

      java -jar nexus-3.37.3-02/lib/support/nexus-orient-console.jar
      connect plocal:sonatype-work/nexus3/db/component admin admin
      delete from asset
      delete from component
      exit
      

      This creates a scenario for reconcile to run.

      3. Migrate the instance to Postgres.

      4. Make a note of the default blob store size:

      du -sh sonatype-work/nexus3/blobs/default/content

      5. Configure and run the reconcile task ("restore blob metadata" option selected only).

      Actual:

      The data deleted from step 2 above should be restored, however check the size of the blob store again and observe it has approximately doubled in size. Nexus proceeds to generate new blob Ids and files (both bytes and properties) for the restored assets where the new blob property has a @BlobStore.blob-name property with a leading slash, as well as a new uploader (system) and a new creation date:

      Here is an existing blob:

      cat sonatype-work/nexus3/blobs/default/content/vol-02/chap-45/c0e0d7aa-f721-43c5-b9db-9e9b52df2560.properties
      #2022-02-10 17:57:45,953+0000
      #Thu Feb 10 17:57:45 GMT 2022
      @BlobStore.created-by=admin
      size=64
      @Bucket.repo-name=maven-releases
      creationTime=1644515865953
      @BlobStore.created-by-ip=127.0.0.1
      @BlobStore.content-type=text/plain
      @BlobStore.blob-name=com/foo/bar/1.0.1/bar-1.0.1.pom.sha256
      sha1=de689b13c9d43b6286b32930307312fab3c8119b

      Here is one generated by the reconcile task with a leading slash, new uploader and new creation date:

      cat sonatype-work/nexus3/blobs/default/content/vol-25/chap-14/bacd7af1-c347-4597-b842-a3557fa6318d.properties
      #2022-02-10 18:19:33,263+0000
      #Thu Feb 10 18:19:33 GMT 2022
      @BlobStore.created-by=system
      size=64
      @Bucket.repo-name=maven-releases
      creationTime=1644517173263
      @BlobStore.created-by-ip=system
      @BlobStore.content-type=text/plain
      @BlobStore.blob-name=/com/foo/bar/1.0.1/bar-1.0.1.pom.sha256
      sha1=de689b13c9d43b6286b32930307312fab3c8119b

      Expected:

      The reconcile task should not create such duplicates. Instead as exactly stated in NEXUS-28414:

      • Original blob is re-attached to the Asset rather than creating new blobs.
      • Original dates are preserved

      And in addition to this the original uploader and uploader IP are also preserved.

       

      Attachments

        Issue Links

          Activity

            People

              mpiggott Matthew Piggott
              hardeepn Hardeep Nagra
              Michael Oliverio Michael Oliverio
              NXRM - Sentinels
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                tigCommentSecurity.panel-title