Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-16545

NXRM 3 Yum Metadata Generation takes forever

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Release Note:
      Yes

      Description

      We recently migrated completely from nexus2 to nexus3, now that it has full yum support (hosted). Recently, we noticed that when we were doing our rpm builds/uploads, that when we would deploy new instances, we were not seeing the expected artifacts on the machines. I dug into this a bit and observed the following:

      [root@pkgsvcs-cdb3ca log]# grep 'event-10-thread-499' nexus.log
2018-03-13 00:11:53,744+0000 INFO [event-10-thread-499] jenkins org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl - Rebuilding yum metadata for repository snapshot-rpms
2018-03-13 00:11:53,744+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: OPEN -> ACTIVE
2018-03-13 00:11:53,744+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: ACTIVE
2018-03-13 00:11:53,745+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Counting assets with query: select count(*) from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=RPM}
2018-03-13 00:26:49,108+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Finding assets with query: select from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=COMPS}
2018-03-13 00:26:49,149+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Finding assets with query: select from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=REPODATA}
2018-03-13 00:26:49,161+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/b5b377a566e923835e71bde8e8c2a3c9dc39279bb4d49f2d1b5bbec48350a760-filelists.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,161+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/b5b377a566e923835e71bde8e8c2a3c9dc39279bb4d49f2d1b5bbec48350a760-filelists.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,161+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,161+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/b5b377a566e923835e71bde8e8c2a3c9dc39279bb4d49f2d1b5bbec48350a760-filelists.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 00:26:49,165+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/f6a3101a65ee327d76cfcd4a34a6343e7d9c7ff144a089f27b5b89b1b8dee5b5-primary.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,165+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/f6a3101a65ee327d76cfcd4a34a6343e7d9c7ff144a089f27b5b89b1b8dee5b5-primary.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,165+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,165+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/f6a3101a65ee327d76cfcd4a34a6343e7d9c7ff144a089f27b5b89b1b8dee5b5-primary.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 00:26:49,179+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/002e077e897e65b99b47e6705e1144b0b8e086dc4034308343b3b3fbc7f9eabd-other.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,179+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/002e077e897e65b99b47e6705e1144b0b8e086dc4034308343b3b3fbc7f9eabd-other.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,179+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 00:26:49,179+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/002e077e897e65b99b47e6705e1144b0b8e086dc4034308343b3b3fbc7f9eabd-other.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 00:26:49,196+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/repomd.xml as [application/xml, application/x-xml, text/xml]
2018-03-13 00:26:49,196+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/repomd.xml as [application/xml, text/xml, application/x-xml]
2018-03-13 00:26:49,196+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/xml, application/x-xml, text/xml]
2018-03-13 00:26:49,196+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/repomd.xml declared as application/xml, determined as application/xml
2018-03-13 00:26:49,198+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: ACTIVE -> OPEN
2018-03-13 00:26:49,236+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: OPEN
2018-03-13 00:26:49,236+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: OPEN -> CLOSED
2018-03-13 00:26:49,236+0000 DEBUG [event-10-thread-499] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: CLOSED

      ==========

      [root@pkgsvcs-cdb3ca log]# grep 'event-10-thread-533' nexus.log
2018-03-13 01:21:44,924+0000 INFO [event-10-thread-533] jenkins org.sonatype.nexus.repository.yum.internal.createrepo.CreateRepoFacetImpl - Rebuilding yum metadata for repository snapshot-rpms
2018-03-13 01:21:44,924+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: OPEN -> ACTIVE
2018-03-13 01:21:44,924+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: ACTIVE
2018-03-13 01:21:44,925+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Counting assets with query: select count(*) from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=RPM}
2018-03-13 01:36:44,021+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Finding assets with query: select from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=COMPS}
2018-03-13 01:36:44,046+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.AssetEntityAdapter - Finding assets with query: select from asset where (attributes.yum.asset_kind = :p0) and (bucket=[#11|https://support.sonatype.com/hc/requests/11]:56), parameters: \{p0=REPODATA}
2018-03-13 01:36:44,051+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/e31c8020129321350fd322fb4c3d6ad0f6beca0bc540fb98921421483028170f-filelists.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,051+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/e31c8020129321350fd322fb4c3d6ad0f6beca0bc540fb98921421483028170f-filelists.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,051+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,051+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/e31c8020129321350fd322fb4c3d6ad0f6beca0bc540fb98921421483028170f-filelists.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 01:36:44,060+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/158a6f7ebfee803454689bca7aa05786ab92ef9843e5dae79e4d4e8d2b60c99e-primary.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,060+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/158a6f7ebfee803454689bca7aa05786ab92ef9843e5dae79e4d4e8d2b60c99e-primary.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,060+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,060+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/158a6f7ebfee803454689bca7aa05786ab92ef9843e5dae79e4d4e8d2b60c99e-primary.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 01:36:44,065+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/3de6dcd91fa36eed84cae64422afb99996ecda0f6c97dcd0c612d97d26f15b3e-other.xml.gz as [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,065+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/3de6dcd91fa36eed84cae64422afb99996ecda0f6c97dcd0c612d97d26f15b3e-other.xml.gz as [application/x-gzip, application/gzip, application/x-tgz, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,065+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/gzip, application/gzip-compressed, application/gzipped, application/x-gunzip, application/x-gzip, application/x-gzip-compressed, gzip/document]
2018-03-13 01:36:44,065+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/3de6dcd91fa36eed84cae64422afb99996ecda0f6c97dcd0c612d97d26f15b3e-other.xml.gz declared as application/x-gzip, determined as application/x-gzip
2018-03-13 01:36:44,069+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support detects repodata/repomd.xml as [application/xml, application/x-xml, text/xml]
2018-03-13 01:36:44,069+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Mime support assumes repodata/repomd.xml as [application/xml, text/xml, application/x-xml]
2018-03-13 01:36:44,069+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - content/name types intersection [application/xml, application/x-xml, text/xml]
2018-03-13 01:36:44,069+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.DefaultContentValidator - Content repodata/repomd.xml declared as application/xml, determined as application/xml
2018-03-13 01:36:44,074+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: ACTIVE -> OPEN
2018-03-13 01:36:44,100+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: OPEN
2018-03-13 01:36:44,100+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioning: OPEN -> CLOSED
2018-03-13 01:36:44,100+0000 DEBUG [event-10-thread-533] jenkins org.sonatype.nexus.repository.storage.StorageTxImpl - Transitioned: CLOSED

      =========
      In both of these cases, regenerating the metadata is taking 15 minutes.  That is a major detriment and will cause inefficiencies for developers, slowing down a company's ability to handle CI/CD

        Attachments

          Issue Links

          relates

          Bug - A problem which impairs or prevents the functions of the product. NEXUS-19702 Regression: Yum metadata leaks between trees in a hosted yum repository.

          • Major - Major loss of function.
          • Closed
          testing discovered

          Bug - A problem which impairs or prevents the functions of the product. NEXUS-17263 Yum Metadata Rebuild task says it's done and waiting but it's still working

          • Medium - Medium
          • Open

          Bug - A problem which impairs or prevents the functions of the product. NEXUS-17285 Yum component properties update does not reflect in UI summary pane

          • Medium - Medium
          • Closed
          mentioned in

          Page Loading...

            Activity

              People

              Assignee:
              jstephens Joseph Stephens
              Reporter:
              mmercer-apixio Marc Mercer
              Last Updated By:
              Peter Lynch Peter Lynch
              Team:
              Nexus - Formats
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title