Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-27530

proxying nested repodata/repomd.xml from remote YUM repositories causes automatic download attempt of primary.xml.gz at wrong remote path

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: New
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.30.1
    • Fix Version/s: None
    • Component/s: Yum
    • Labels:
    • Notability:
      n/a

      Description

      Create a yum proxy named yummirror-test with remote URL of http://mirror.centos.org/centos/ as our documentation states is supported:
      https://help.sonatype.com/repomanager3/formats/yum-repositories#YumRepositories-ProxyingYumRepositories

      Make a request for nested metadata at

      curl -v http://localhost:8081/repository/yummirror-test/7/updates/x86_64/repodata/repomd.xml

      You get:

      curl -v http://localhost:8081/repository/yummirror-test/7/updates/x86_64/repodata/repomd.xml
      *   Trying ::1:8081...
      * connect to ::1 port 8081 failed: Connection refused
      *   Trying 127.0.0.1:8081...
      * Connected to localhost (127.0.0.1) port 8081 (#0)
      > GET /repository/yummirror-test/7/updates/x86_64/repodata/repomd.xml HTTP/1.1
      > Host: localhost:8081
      > User-Agent: curl/7.75.0
      > Accept: */*
      > 
      * Mark bundle as not supporting multiuse
      < HTTP/1.1 200 OK
      < Date: Wed, 05 May 2021 19:39:04 GMT
      < Server: Nexus/3.30.0-01 (PRO)
      < X-Content-Type-Options: nosniff
      < Content-Security-Policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
      < X-XSS-Protection: 1; mode=block
      < Last-Modified: Fri, 30 Apr 2021 14:51:44 GMT
      < ETag: "bbf-5c131c01f6dfa"
      < Content-Type: application/xml
      < Content-Length: 1538
      < 
      <?xml version="1.0" encoding="UTF-8" standalone="no"?><repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
       <revision>1619794281</revision>
      <data type="filelists">
        <checksum type="sha256">0d5839251d55d8dd5a39d7717c91b823bf592dc21bdf3100dea1179c22956204</checksum>
        <open-checksum type="sha256">35e446dfc9be1739a55e0f35e6ef7601d56813e4f7a093b83ed46c763821ff29</open-checksum>
        <location href="repodata/0d5839251d55d8dd5a39d7717c91b823bf592dc21bdf3100dea1179c22956204-filelists.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>5222801</size>
        <open-size>92288992</open-size>
      </data>
      <data type="primary">
        <checksum type="sha256">7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee</checksum>
        <open-checksum type="sha256">5a5d983cddf97976d7d89194e6ef30c5037f3759545f42da844f39ef9d25941e</open-checksum>
        <location href="repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>4716536</size>
        <open-size>37200529</open-size>
      </data>
      
      
      <data type="other">
        <checksum type="sha256">9048baae4bf0315e02f8435c088b0a07ea0719635f1e1ab2b907045fc1864047</checksum>
        <open-checksum type="sha256">235091361481edbf5ec3eef6bd914e3cd9478dad85c02318672c689dc6419c29</open-checksum>
        <location href="repodata/9048baae4bf0315e02f8435c088b0a07ea0719635f1e1ab2b907045fc1864047-other.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>416278</size>
        <open-size>6322250</open-size>
      </data>
      </repomd>
      

      NXRM will get( in this case, subject to change as the remote example updates)

      <?xml version="1.0" encoding="UTF-8"?>
      <repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
       <revision>1619794281</revision>
      <data type="filelists">
        <checksum type="sha256">0d5839251d55d8dd5a39d7717c91b823bf592dc21bdf3100dea1179c22956204</checksum>
        <open-checksum type="sha256">35e446dfc9be1739a55e0f35e6ef7601d56813e4f7a093b83ed46c763821ff29</open-checksum>
        <location href="repodata/0d5839251d55d8dd5a39d7717c91b823bf592dc21bdf3100dea1179c22956204-filelists.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>5222801</size>
        <open-size>92288992</open-size>
      </data>
      <data type="primary">
        <checksum type="sha256">7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee</checksum>
        <open-checksum type="sha256">5a5d983cddf97976d7d89194e6ef30c5037f3759545f42da844f39ef9d25941e</open-checksum>
        <location href="repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>4716536</size>
        <open-size>37200529</open-size>
      </data>
      <data type="primary_db">
        <checksum type="sha256">565ca1d48cd57e7523cc4382a66bd991924df5865a2bb55e66d521661bb92809</checksum>
        <open-checksum type="sha256">caae2a710a8e5a57b7c4cdccc4fd2b63816eced36d7186642ca2005131d2df04</open-checksum>
        <location href="repodata/565ca1d48cd57e7523cc4382a66bd991924df5865a2bb55e66d521661bb92809-primary.sqlite.bz2"/>
        <timestamp>1619794304</timestamp>
        <database_version>10</database_version>
        <size>8337925</size>
        <open-size>44091392</open-size>
      </data>
      <data type="other_db">
        <checksum type="sha256">3cf8c38c7ac38fc7370a4f8e3d06a9e29dbae1ae32d929753d967bbaadef21df</checksum>
        <open-checksum type="sha256">227346975626e66c2ca688a21c5ca1c6398cd037115995eab2e0471f8bd8d896</open-checksum>
        <location href="repodata/3cf8c38c7ac38fc7370a4f8e3d06a9e29dbae1ae32d929753d967bbaadef21df-other.sqlite.bz2"/>
        <timestamp>1619794291</timestamp>
        <database_version>10</database_version>
        <size>621197</size>
        <open-size>6547456</open-size>
      </data>
      <data type="other">
        <checksum type="sha256">9048baae4bf0315e02f8435c088b0a07ea0719635f1e1ab2b907045fc1864047</checksum>
        <open-checksum type="sha256">235091361481edbf5ec3eef6bd914e3cd9478dad85c02318672c689dc6419c29</open-checksum>
        <location href="repodata/9048baae4bf0315e02f8435c088b0a07ea0719635f1e1ab2b907045fc1864047-other.xml.gz"/>
        <timestamp>1619794290</timestamp>
        <size>416278</size>
        <open-size>6322250</open-size>
      </data>
      <data type="filelists_db">
        <checksum type="sha256">4ac0a308d1f54585d79ab63f5a39db8f5f7005b8e0dcfbd9626a37b0588e632c</checksum>
        <open-checksum type="sha256">6a1a8fbcc1f184510ea7d73b7248b130eb7a3ff6f65ab633146bc99330adffb1</open-checksum>
        <location href="repodata/4ac0a308d1f54585d79ab63f5a39db8f5f7005b8e0dcfbd9626a37b0588e632c-filelists.sqlite.bz2"/>
        <timestamp>1619794298</timestamp>
        <database_version>10</database_version>
        <size>4901187</size>
        <open-size>42722304</open-size>
      </data>
      </repomd>
      
      

      Problem

      The problem is that NXRM makes outbound requests for metadata related files automatically at the wrong paths, different than a yum client would. NXRM makes these requests:

      2021-05-05 16:39:04,493-0300 DEBUG [qtp653732622-256] *UNKNOWN org.sonatype.nexus.httpclient.outbound - http://mirror.centos.org/centos/7/updates/x86_64/repodata/repomd.xml > GET /centos/7/updates/x86_64/repodata/repomd.xml HTTP/1.1
      2021-05-05 16:39:05,495-0300 DEBUG [qtp653732622-256] *UNKNOWN org.sonatype.nexus.httpclient.outbound - http://mirror.centos.org/centos/7/updates/x86_64/repodata/repomd.xml < HTTP/1.1 200 OK @ 1.002 s
      2021-05-05 16:39:05,495-0300 INFO  [qtp653732622-256] *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for yummirror-test changed from READY to AVAILABLE - reason n/a for n/a
      2021-05-05 16:39:05,544-0300 DEBUG [qtp653732622-256] *UNKNOWN org.sonatype.nexus.httpclient.outbound - http://mirror.centos.org/centos/repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz > GET /centos/repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz HTTP/1.1
      2021-05-05 16:39:05,609-0300 DEBUG [qtp653732622-256] *UNKNOWN org.sonatype.nexus.httpclient.outbound - http://mirror.centos.org/centos/repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz < HTTP/1.1 404 Not Found @ 65.30 ms
      2021-05-05 16:39:05,611-0300 WARN  [qtp653732622-256] *UNKNOWN org.sonatype.nexus.repository.yum.orient.internal.proxy.OrientYumProxyFacetImpl - Failed to fetch metadata file for path repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz and repository yummirror-test. Received status code 404
      2021-05-05 16:39:05,612-0300 WARN  [qtp653732622-256] *UNKNOWN org.sonatype.nexus.repository.yum.orient.internal.metadata.OrientRepomdChecksumAndSizeUpdater - Unable to update checksum and size for primary file because primary could not be retrieved: repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz
      

      Expected

      If NXRM is going to automatically download metadata files referenced inside of repomd.xml files, then it should due this with the same logic that a yum client would. In other words, NXRM should be downloading this file :

      http://mirror.centos.org/centos/7/updates/x86_64/repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz

      NOT THIS FILE:

      http://mirror.centos.org/centos/repodata/7bc33051039e6250e0949ff5a1f034555dbd9e87412e83bc972fd1402b76b4ee-primary.xml.gz

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            plynch Peter Lynch
            Last Updated By:
            Rich Seddon Rich Seddon
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:

                tigCommentSecurity.panel-title