Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-26419

proxied GA level maven-metadata.xml validation is all or nothing and instead should use a transparent conversion

Details

    • 5
    • NXRM MadMax Sprint 58
    • n/a
    • 2

    Description

      NXRM 2 ( and others like bintray and Artifactory ) repository managers can have a Maven 2 repository serving GA level metadata ( example org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml ) that contains a
      root level <version> element. Example:

      <?xml version="1.0" encoding="UTF-8"?>
      <metadata modelVersion="1.1.0">
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>versions-maven-plugin</artifactId>
        <version>1.0-alpha-1-SNAPSHOT</version>
        <versioning>
          <latest>2.8.1</latest>
          <release>2.8.1</release>
          <versions>
            <version>1.0-alpha-1-SNAPSHOT</version>
            <version>1.0-alpha-1</version>
            <version>1.0-alpha-2-SNAPSHOT</version>
            <version>1.0-alpha-2</version>
            <version>1.0-alpha-3-SNAPSHOT</version>
            <version>1.0-alpha-3</version>
            <version>1.0-beta-1-SNAPSHOT</version>
            <version>1.0-SNAPSHOT</version>
            <version>1.0</version>
            <version>1.1-SNAPSHOT</version>
            <version>1.1</version>
            <version>1.2-SNAPSHOT</version>
            <version>1.2</version>
            <version>1.3-SNAPSHOT</version>
            <version>1.3</version>
            <version>1.3.1</version>
            <version>2.0</version>
            <version>2.1</version>
            <version>2.1-with-increment-atlassian-1</version>
            <version>2.2</version>
            <version>2.3</version>
            <version>2.4</version>
            <version>2.5</version>
            <version>2.6</version>
            <version>2.7</version>
            <version>2.8.1</version>
          </versions>
          <lastUpdated>20201103144205</lastUpdated>
        </versioning>
      </metadata>
      

      Real world examples:
      https://repository.sonatype.org/content/repositories/sonatype-grid/org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      https://packages.atlassian.com/mvn/maven-external/org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      https://plugins.gradle.org/m2/org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      https://repository.cloudera.com/artifactory/cloudera-repos/com/fasterxml/jackson/core/jackson-databind/maven-metadata.xml

      Problem

      The main problem here is that <version>1.0-alpha-1-SNAPSHOT</version> at the root of the metadata is not valid for GA level metadata and NXRM 3 will deny serving the metadata.

      See https://maven.apache.org/ref/3.6.3/maven-repository-metadata/repository-metadata.html which states the <version> element is only valid
      for "The version that this directory represents, if any. It is used for artifact snapshots only." This means it should only be present at GA-baseSnapshotVersion levels like org/codehaus/mojo/versions-maven-plugin/1.0-alpha-1-SNAPSHOT/maven-metadata.xml .

      A proxy repository in NXRM 3 with any Version Policy (Release, Snapshot, Mixed) will interpret the sample maven-metadata.xml
      as not valid and return 404 to any request for that file. The nexus.log contains three WARN messages per proxy repository failure:

      The code that introduced the validation was based on the assumptions stated in this comment for issue NEXUS-16853 where enhanced maven-metadata.xml validation was introduced.

      WARN messages for Version policy release
      2021-01-16 16:01:25,411-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.maven.internal.validation.MavenMetadataContentValidator - maven-metadata.xml path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml does not match the expected path org/codehaus/mojo/versions-maven-plugin/1.0-alpha-1-SNAPSHOT/maven-metadata.xml
      2021-01-16 16:01:25,412-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl - org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      2021-01-16 16:01:25,416-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.view.handlers.ExceptionHandler - Invalid content: GET /org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml: org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      
      WARN messages for Version policy Snapshot
      2021-01-16 16:01:25,516-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.maven.internal.validation.MavenMetadataContentValidator - maven-metadata.xml path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml does not match the expected path org/codehaus/mojo/versions-maven-plugin/1.0-alpha-1-SNAPSHOT/maven-metadata.xml
      2021-01-16 16:01:25,517-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl - org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      2021-01-16 16:01:25,523-0400 WARN  [qtp784617076-405] admin org.sonatype.nexus.repository.view.handlers.ExceptionHandler - Invalid content: GET /org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml: org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      
      WARN messages for Version policy Mixed - a concept of NXRM 3 ONLY
      2021-01-16 17:08:57,713-0400 WARN  [qtp1083918476-261] admin org.sonatype.nexus.repository.maven.internal.validation.MavenMetadataContentValidator - maven-metadata.xml path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml does not match the expected path org/codehaus/mojo/versions-maven-plugin/1.0-alpha-1-SNAPSHOT/maven-metadata.xml
      2021-01-16 17:08:57,713-0400 WARN  [qtp1083918476-261] admin org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl - org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      2021-01-16 17:08:57,719-0400 WARN  [qtp1083918476-261] admin org.sonatype.nexus.repository.view.handlers.ExceptionHandler - Invalid content: GET /org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml: org.sonatype.nexus.repository.InvalidContentException: Invalid maven-metadata.xml GAV org.codehaus.mojo, versions-maven-plugin, 1.0-alpha-1-SNAPSHOT does not match request path org/codehaus/mojo/versions-maven-plugin/maven-metadata.xml
      

      NXRM 3 takes an all or nothing approach to proxying this metadata. By default ( since 3.16.0 ) it will consider it not valid, and return 404.
      IMPACT: repo 3 now prevents proxying any of the valid versions that may be listed in that metadata for the given GA and also help storing not valid metadata in NXRM 3. This means previous working builds can fail.

      A global property can be set affecting all Maven 2 format repos in NXRM 3 that disables the metadata validation completely, thus allowing the metadata
      to be accepted and returned to requestors.

      Workaround

      Although not ideal because it is a global setting affecting every Maven repository in the entire NXRM instance AND setting this property to false will disable other metadata validation not related to this specific issue,
      adding this property to data-dir/etc/nexus.properties will allow the metadata to pass validation and be served from NXRM 3.

       
      nexus.maven.metadata.validation.enabled=false
      

      This property was introduced in NXRM 3.16.0 as part of NEXUS-16853.

      Other workarounds can be complex, like to not proxy the remote at all. This has its own problems.

      Expected

      The all or nothing approach can be improved by a middle ground which adheres to storing only valid maven-metadata, but also the
      repo configuration dictated by an NXRM administrator.

      When a remote returns a GA level maven-metadata.xml with a 200 response

      • AND file passes content validation (mime) and validation exclusive of this issue
      • AND if the remote indicates it has newer maven-metadata.xml file than what is presently cached locally
      • AND contains a root level <version> element, any value OR any <versioning><version> values which do not match the proxy repo version policy

      Then

      • do not fail content validation
      • do not return 404

      Instead:

      • throw out any root level <version> element, as that is only applicable to GAV SNAPSHOT versioned maven-metadata
      • if the proxy repository Version policy is Release, then
        • throw out all <version> elements that do not contain release style Maven versions
        • merge the remote metadata with any already cached metadata, excluding snapshot versions and root version element in the local cached metadata as well
        • return 200 response and the rebuilt maven-metadata.xml
      • if the proxy repository Version policy is Snapshot, then
        • throw out all <version> elements that do not contain Snapshot style Maven versions
        • merge the remote metadata with any already cached metadata, excluding release versions in the local cached metadata as well
        • return 200 response and the rebuilt maven-metadata.xml
      • if the proxy repository Version policy is Mixed, then
        • accept <versioning><version> elements containing either release style or snapshot style versions
        • merge the remote metadata with any already cached metadata
        • return 200 response and the rebuilt maven-metadata.xml
      • if the inbound request was to a group repository with proxy members
        • the resulting merged metadata returned from the group should be the merged results of each member
        • since group repos don't contain a version policy, a version policy of Mixed is implied for Maven groups

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              plynch Peter Lynch
              Grace Lee Grace Lee
              NXRM - Mad Max
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                tigCommentSecurity.panel-title