Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-6801

prevent createrepo/mergerepo from creating sqlite rpm metadata

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Release Note:
      Yes
    • Sprint:
      Sprint 27, Sprint 28

      Description

      According to all accounts I could find, the sqllite databases created using the --database ( defaults to true since 2010, createrepo-0.9.9) option to createrepo/mergerepo is an *client optimization only*.

      Where access to Nexus is highly concurrent, and all yum clients seem to support the basic mode without sqlite files on the server just fine, we should therefore stop asking createrepo.mergerepo to create the sqllite databases.

      This in theory should improve metadata creation performance(TBD), increase locking throughput and avoid the possibility of having xml based metadata out of sync from sqllite data.

      Since 2010, createrepo and mergerepo have added the --no-database option to prevent creation of the sqlite metadata. Before 0.9.9 version, no sqlite data was being created unless specifically asked.

      These commits changed the default createrepo/mergerepo behavior from not creating sqlite databases to creating them.

      http://createrepo.baseurl.org/gitweb?p=createrepo.git;a=commit;h=1891306b9f4dc98567bcd9507708d8e089e2c69c

      http://createrepo.baseurl.org/gitweb?p=createrepo.git;a=commit;h=bc3df9bece50098d9dc1c82f0addac36cc1db67a

      Further, YUM documentation ( and code if you look hard enough) clarifies that if sqlite files are not found, YUM falls back to getting the XML files instead.

      http://yum.baseurl.org/gitweb?p=yum.git;a=blob;f=docs/yum.conf.5;h=97acbe43eb63660a86beb4c1ea5b322bdf1303ad;hb=HEAD#l676


      Our code uses --database option, thereby always creating sqlite databases on the server.

      GenerateMetadataTask --database option

      MergeMetadataTask -d option

      In summary:

      • our commands should remove -d, --database and add --no-database
      • we should clearly specify we require createrepo 0.9.9 or greater to be used OR provide some override to not pass the --no-database option to createrepo/mergerepo in case the local version does not recognize this option
      • if implemented, YUM clients can call yum clean dbcache locally to clean up any previously "downloaded from Nexus" sqlite databases and from then on generate their own sqlite files from the xml data only
      • YUM clients could configure 'mddownloadpolicy' in yum.conf to be xml, as a very slight optimization when known to be talking to Nexus, but not required
      • on upgrade after this change, delete any sqlite files in Nexus repos to avoid clients from downloading them in the future and getting even more confused, or something more slightly confusing, always responding 404 even if they exist

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              rseddon Rich Seddon
              Reporter:
              plynch Peter Lynch
              Last Updated By:
              Joe Tom Joe Tom
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title