Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 3.29.2
-
Fix Version/s: 3.30.0
-
Component/s: import-export, Repository
-
Labels:
-
Epic Link:
-
Notability:n/a
Description
Start with a maven format source directory containing artifacts what when the sha512 sum is computed, results in a sha512sum that starts with the text "caff". When the import task tries to generate the sha512 sum, the sum fails content validation and thus never gets created.
2021-01-09 08:17:30,470-0400 WARN [quartz-9-thread-2] *SYSTEM org.sonatype.nexus.repository.storage.StorageTxImpl - An exception occurred determining the content type of asset pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar.sha512 in repository central-hosted 2021-01-09 08:17:30,472-0400 ERROR [quartz-9-thread-2] *SYSTEM com.sonatype.nexus.exportimport.orient.internal.importtask.OrientRepositoryImportService - Import of file /import-data/central-releases/pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar into repository central-hosted failed org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar.sha512 at org.sonatype.nexus.repository.mime.DefaultContentValidator.determineContentType(DefaultContentValidator.java:95) at org.sonatype.nexus.repository.maven.internal.MavenContentValidator.determineContentType(MavenContentValidator.java:85) at org.sonatype.nexus.repository.storage.StorageTxImpl.determineContentType(StorageTxImpl.java:1019) at org.sonatype.nexus.repository.storage.StorageTxImpl.buildStorageHeaders(StorageTxImpl.java:760) at org.sonatype.nexus.repository.storage.StorageTxImpl.createBlob(StorageTxImpl.java:719) at sun.reflect.GeneratedMethodAccessor271.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53) at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:69) at com.sun.proxy.$Proxy237.createBlob(Unknown Source) at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.doPut(MavenFacetImpl.java:342) at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:49) at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.put(MavenFacetImpl.java:306) at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetUtils.addHashes(MavenFacetUtils.java:189) at org.sonatype.nexus.repository.maven.internal.orient.MavenUploadHandler.putChecksumFiles(MavenUploadHandler.java:130) at org.sonatype.nexus.repository.maven.internal.orient.MavenUploadHandler.doPut(MavenUploadHandler.java:112) at org.sonatype.nexus.repository.maven.MavenUploadHandlerSupport.handle(MavenUploadHandlerSupport.java:190) at org.sonatype.nexus.repository.upload.internal.UploadManagerImpl.handle(UploadManagerImpl.java:136) at com.sonatype.nexus.exportimport.orient.internal.importtask.OrientRepositoryImportService.lambda$1(OrientRepositoryImportService.java:166) at org.sonatype.nexus.transaction.OperationPoint.lambda$0(OperationPoint.java:53) at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64) at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:57) at org.sonatype.nexus.transaction.Operations.proceedWithTransaction(Operations.java:232) at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:223) at org.sonatype.nexus.transaction.Operations.run(Operations.java:175) at com.sonatype.nexus.exportimport.orient.internal.importtask.OrientRepositoryImportService.lambda$0(OrientRepositoryImportService.java:162) at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource$RepositoryImportFileVisitor.visitFile(RepositoryImportSource.java:83) at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource$RepositoryImportFileVisitor.visitFile(RepositoryImportSource.java:1) at java.nio.file.Files.walkFileTree(Files.java:2670) at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportSource.walk(RepositoryImportSource.java:45) at com.sonatype.nexus.exportimport.orient.internal.importtask.OrientRepositoryImportService.walkImportDirectory(OrientRepositoryImportService.java:140) at com.sonatype.nexus.exportimport.orient.internal.importtask.OrientRepositoryImportService.doImport(OrientRepositoryImportService.java:125) at com.sonatype.nexus.exportimport.internal.importtask.RepositoryImportTask.execute(RepositoryImportTask.java:64) at org.sonatype.nexus.repository.RepositoryTaskSupport.execute(RepositoryTaskSupport.java:79) at org.sonatype.nexus.scheduling.TaskSupport.call(TaskSupport.java:100) at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.doExecute(QuartzTaskJob.java:143) at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.execute(QuartzTaskJob.java:106) at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
sha512sum pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar caff198f65d21d25d582f467d1ac78dc72903a7aace9a5380503839032cf663dd5871221f6488123b5fa2a4b07b728ba1827657d052889d65fa874e93fc368cf ../pmd-test/pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar
A sample partial 56GB import from Central repository found several artifacts which trigger this problem:
> grep 'Detected content type' tasks/repository.import-20210108175634047.log | grep -E "256|512" org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: org/mortbay/jetty/jetty-util/6.1.15.rc2/jetty-util-6.1.15.rc2.jar.sha512 org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: org/codehaus/fabric3/fabric3-management-api/0.6/fabric3-management-api-0.6.jar.sha256 org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: org/apache/apache-jar-resource-bundle/1.4/apache-jar-resource-bundle-1.4-sources.jar.asc.sha256 org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: woodstox/wstx-asl/1.8.2/wstx-asl-1.8.2.pom.sha256 org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: pmd/pmd-jdk14/4.2/pmd-jdk14-4.2.jar.sha512 org.sonatype.nexus.repository.InvalidContentException: Detected content type [audio/x-caf], but expected [text/plain]: turbine/turbine/2.2b1/turbine-2.2b1.jar.sha512
This appears to be a variation of issues previously seen where checksums are being processed by content validation algorithms. Now that sha512 and sha256 sums are being generated and supported by NXRM, the requirement to not process these sum files by content validation does not appear to be implemented.
Expected
As in NEXUS-20340, avoid complex mime-type content validation for Maven hashes, including sha256 and sha512. Instead use a much simpler algorithm for proxying content validation. For import where NXRM generates these checksums, use no validation since it is implied the hash will be valid. Make sure all uses of content validation are fixed, including proxy repo processing and hosted repo import.
