Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-26446

Import external files try importing already imported assets

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Not A Bug
    • Affects Version/s: 3.29.0
    • Fix Version/s: None
    • Component/s: import-export
    • Labels:
    • Notability:
      3

      Description

      SYMPTOM:

      Using the "Repository - Import external files" from NXRM2 directory to NXRM3 could generate "org.sonatype.nexus.repository.IllegalOperationException: Repository does not allow updating assets: xxxx" errors against already imported assets by just re-running the same task.

      REPRODUCE STEPS:

      NOTE: To simplify the steps, using NXRM3 only.

      1. Create "maven-hosted" repo on NXRM3 version which supports import/export tasks.
      2. Upload an asset, for example junit 4.21.
      3. Export the "maven-hosted" repo into "/var/tmp/exported" directory with the export assets task.
      4. Create a new hosted repo "maven-imported-hosted" with "Disable redeploy" policy.
      5. Import from "/var/tmp/exported" directory into "maven-imported-hosted" repo with the import external files task.
        Confirm junit 4.21 is imported successfully.
      6. Add com.sonatype.nexus.exportimport.internal = DEBUG logging, then re-run the same import task.
        Confirm the task log says "Skipping /var/tmp/exported/junit/junit/4.21/junit-4.21.jar as it has already been imported." and no ERROR.
      7. Touch some files. For example: find /var/tmp/exported -type f -exec touch {} +
      8. Re-run the import task again.

      EXPECTED BEHAVIOUR:

      Similar to when sys admins use the rsync command, relying on the file modification time tends to cause sync issue due to file system differences or unexpected touch, so (ideally) Nexus should check some hash of each asset.

      ACTUAL BEHAVIOUR:

      Currently RepositoryImportService.java gets the timestamp from getLastUpdatedTimestamp(file), and if lastTimestamp == timestamp, then it considers already in the delta file.
      Because I touched all files under /var/tmp/exported, Nexus re-tried to import all assets, which not only caused unnecessary ERROR below, but main problem is the task would take longer if the source directory were large.

      2021-01-19 07:24:31,041+0000 ERROR [quartz-9-thread-5]  *SYSTEM com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportService - Import of file /var/tmp/share/sonatype/export-tmp/junit/junit/4.21/junit-4.21.jar into repository maven-imported-hosted failed
      org.sonatype.nexus.repository.IllegalOperationException: Repository does not allow updating assets: maven-imported-hosted
              at org.sonatype.nexus.repository.storage.StorageTxImpl.isDuplicateBlob(StorageTxImpl.java:852)
              at org.sonatype.nexus.repository.storage.StorageTxImpl.attachBlob(StorageTxImpl.java:769)
              at sun.reflect.GeneratedMethodAccessor431.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      ...
      

      Also, this means Nexus administrators need to be careful when they copy the source directory, so that modified time is preserved.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            hosako Hajime Osako
            Last Updated By:
            Peter Lynch Peter Lynch
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title