Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28872

IllegalOperationException npm tgz cannot be updated via NpmContentFacetImpl.saveTarball import task of npm content

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: New
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.34.0
    • Fix Version/s: None
    • Component/s: import-export, NPM
    • Labels:
    • Notability:
      4

      Description

      During import task of 2.14.13 source repository to 3.34.0 using Postgres, the following exceptions were noticed:

      2021-09-07 14:16:16,803+0000 ERROR [quartz-11-thread-18]  *SYSTEM com.sonatype.nexus.exportimport.datastore.internal.importtask.RepositoryImportServiceImpl - Import of file /data/sonatype-work/nexus/storage/npmjs-internal/@exampleorg/examplepackage/examplepackage/-/examplepackage-1.0.115.tgz into repository npmjs-internal failed
      org.sonatype.nexus.repository.IllegalOperationException: npmjs-internal/@exampleorg/examplepackage/-/examplepackage-1.0.115.tgz cannot be updated
      	at org.sonatype.nexus.repository.content.facet.ContentFacetSupport.throwNotAllowed(ContentFacetSupport.java:306)
      	at org.sonatype.nexus.repository.content.facet.ContentFacetSupport.checkAttachAllowed(ContentFacetSupport.java:295)
      	at org.sonatype.nexus.repository.content.fluent.internal.FluentAssetBuilderImpl.save(FluentAssetBuilderImpl.java:135)
      	at com.sonatype.nexus.repository.content.npm.internal.NpmContentFacetImpl.saveTarball(NpmContentFacetImpl.java:177)
      	at com.sonatype.nexus.repository.content.npm.internal.NpmHostedFacetImpl.putPackageBlob(NpmHostedFacetImpl.java:290)
      	at com.sonatype.nexus.repository.content.npm.NpmUploadHandlerImpl.doPut(NpmUploadHandlerImpl.java:158)
      	at com.sonatype.nexus.repository.content.npm.NpmUploadHandlerImpl.handle(NpmUploadHandlerImpl.java:119)
      	at org.sonatype.nexus.repository.upload.internal.UploadManagerImpl.handle(UploadManagerImpl.java:158)
      	at com.sonatype.nexus.exportimport.datastore.internal.importtask.RepositoryImportServiceImpl.lambda$0(RepositoryImportServiceImpl.java:164)
      	at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource$RepositoryImportFileVisitor.visitFile(RepositoryImportSource.java:83)
      	at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource$RepositoryImportFileVisitor.visitFile(RepositoryImportSource.java:1)
      	at java.nio.file.Files.walkFileTree(Files.java:2670)
      	at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource.walk(RepositoryImportSource.java:45)
      	at com.sonatype.nexus.exportimport.datastore.internal.importtask.RepositoryImportServiceImpl.walkImportDirectory(RepositoryImportServiceImpl.java:139)
      	at com.sonatype.nexus.exportimport.datastore.internal.importtask.RepositoryImportServiceImpl.doImport(RepositoryImportServiceImpl.java:122)
      	at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportTask.execute(RepositoryImportTask.java:64)
      	at org.sonatype.nexus.repository.RepositoryTaskSupport.execute(RepositoryTaskSupport.java:79)
      	at org.sonatype.nexus.scheduling.TaskSupport.call(TaskSupport.java:100)
      	at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.doExecute(QuartzTaskJob.java:143)
      	at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.execute(QuartzTaskJob.java:106)
      	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
      	at org.sonatype.nexus.quartz.internal.QuartzThreadPool.lambda$0(QuartzThreadPool.java:145)
      	at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40)
      	at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
      	at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

      Diagnosis

      NXRM 3 import task will attempt to import any tgz file it finds, at any path under the source directory, as long as it contains a valid looking package.json file.

      This presents some situations to be aware of.

      When importing from a NXRM 2 repository storage location, it is possible due to some historical bugs in NXRM 2 that tgz files could be present in path names that do not match the package.json metadata within. If two or more tgz exist with different names but same package metadata, then which one will get imported first is somewhat indeterminate. Theoretically NXRM 3 could end up importing and then serving a different tgz file than what NXRM 2 did. First import wins.
      If deployment policy on target repo is allow redeploy, this can create a "last import wins" scenario.

      NXRM 2 will not generate npm package metadata for npm packages which are not at the expected path locations. So in theory no npm clients would have ever consumed the tgz packages at incorrect locations from NXRM 2.

      Expected

      Import task should detect it is importing from a NXRM 2 source repository and handle this case specially. For any tgz file at a path name which does not match its self-contained metadata and where that package should be accessible in NXRM 2, do not import that package. Log a very clear message explaining why the import task did not perform the import - because NXRM 3 detected it was importing from NXRM 2 repo and path was wrong.

      For scenarios where packages were not being imported from a repo 2 source, then attempt to import any tgz at any path within the source dir and use the package metadata to learn what it is ( current behaviour).

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            plynch Peter Lynch
            Last Updated By:
            Peter Lynch Peter Lynch
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:

                tigCommentSecurity.panel-title