Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-24416

make yum repodata metadata for a single level available as soon as possible during metadata rebuild of multiple levels


    • Type: Improvement
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.24.0
    • Fix Version/s: 3.26.0
    • Component/s: Yum
    • Notability:


      When rpms are uploaded to a yum hosted repository, a metadata rebuild task is scheduled to run in the future ( default 60 seconds in the future, or from the end of last rebuild).

      When the metadata rebuild runs, it will attempt to process as many repodata "levels" as it detects as needing a rebuild.

      When a repository contains a lot of levels to rebuild, the total task completion time can take a noticeable time to complete. ie. 20+seconds for example.

      The processing order of the task at a high level currently is:
      1. Detect repodata levels needing rebuild ( short amount of time )
      2. Iterate over and rebuild metadata for all levels to temporary files ( majority of task time )
      3. After all levels are processed, write out all new metadata for all levels ( short amount of time )


      After the task is started, a client may download a repodata/repomd.xml file for one level. This file references other files at that level.

      After the task finishes, the same client may request the repodata filelists.xml.gz file which will now have been replaced by step 3 in the current design. This fails builds.


      Design the task in such a way to write out the metadata per level as soon as it is known. At a high level this looks like this:

      1. Detect repodata levels needing rebuild ( short amount of time )
      2. Iterate over levels needing rebuild - for each level: ( long relative total time )

      1. rebuild metadata for level to temporary files ( short amount of time )
      2. write out new metadata for level immediately ( very short time )
      3. continue to next level

      This design will help minimize the chances of build failure when a build is happening while the task is running.




            sonofreichuk Sergii Onofreichuk
            plynch Peter Lynch
            Last Updated By:
            Peter Lynch Peter Lynch
            NXRM - Trinity
            1 Vote for this issue
            10 Start watching this issue


              Date of First Response: