Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-18672

hard delete of s3 storage blobs is disjointed from Compact Blob Store scheduled task

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.14.0
    • Fix Version/s: None
    • Component/s: Blobstore, Documentation
    • Labels:
    • Notability:
      4

      Description

      As of 3.13.0, Nexus Repository Manager formally includes S3 blobstore official support.

      Prior to this, only File based blob stores were supported.

      All deletions inside NXRM are soft-deletes from the blob store - meaning sets of assets are 'marked' for deletion, but not actually physically deleted.

      For File based blobstores, a Compact Blob Store task is required to "empty trash" and hard delete blobs - after hard deletes, it is anticipated that the blob is not recoverable from storage.

      The S3 Storage blob stores have no implementation of BlobStoreSupport.doCompact(), and therefore they are not affected by the Compact Blob Store task. You can still select S3 blobstores in the Compact blob store task config.

      The s3 Blobstore Configuration inside Nexus does have a "Expiration Days" setting in days.
      When an s3 blobstore is created in Nexus, it adds an S3 lifecycle configuration to expire soft-deleted blobs.

      When an asset is soft-deleted by Nexus, the asset in S3 is tagged "deleted". This is how the S3 lifecycle knows how to expire blobs and make them permanently removed from S3 storage.

      Problem

      Admin user expectation was that the Compact Blob Store task is solely responsible for cleaning up soft-deleted blobs. There is no public documentation ( save OSS code and the one UI field text ) that explains how S3 blobs are hard deleted or the Compact blobstore task is not responsible for this in the case of S3.

      Now there are at least two possible ways to hard delete blobs.

      Ideas

      • Is there a way we can consolidate hard-deletion operations to be more intuitive?
      • Can we expand and improve the documentation about this?
      • Can we prevent any blobstore implementation not implementing doCompact() from being selectable by the Compact Blob Store task?
      • Can we elaborate on how backups of S3 buckets may be affected by Nexus 3 standard database restore procedures?

      Acceptance

      • Restrict the compact task to file blob stores
      • Compact Blob Store -> potentially rename to 'Compact File Blob Store'
      • Clear up the documentation
      • Make sure that any┬ácompact tasks pointing to S3 blob stores doesn't gum up the UI or blow anything up

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            plynch Peter Lynch
            Last Updated By:
            Joe Tom
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Date of First Response:

                tigCommentSecurity.panel-title