Dev - Nexus
  1. Dev - Nexus
  2. NEXUS-4640

Very slow performance of snapshot remover

    Details

    • Global Rank:
      18113
    • Story Points:
      15

      Description

      I've been getting quite a few reports of snapshot removal tasks taking a very long time to run (over 24 hours in some cases). On RSO the weekly snapshot removal job takes around 4 hours.

      This is far too slow. We need to do some profiling, figure out why this takes so long, and see if we can implement some improvements.

      1. Launcher-snapshots-2011-12-07-14-48.snapshot
        3.99 MB
        Peter Lynch

        Issue Links

          Activity

          Hide
          Peter Lynch added a comment -

          After Tamas's alternate implementation, we have a different result that is much improved using latest master branch.

          jvm 1    | 2011-12-07 14:39:04 INFO  [pool-1-thread-1] - org.sonatype.nexus.feeds.DefaultFeedRecorder - Removing snapshots from repository repo1 : Process started on Wed Dec 07 14:39:04 NST 2011, not yet finished.
          jvm 1    | 2011-12-07 14:39:04 INFO  [pool-1-thread-1] - org.sonatype.nexus.maven.tasks.DefaultSnapshotRemover - Removing old SNAPSHOT deployments from repo1 repository.
          jvm 1    | 2011-12-07 14:41:22 INFO  [pool-1-thread-1] - org.sonatype.nexus.proxy.maven.maven2.M2Repository - Clearing NFC cache in repository ID='repo1' from path='/'
          jvm 1    | 2011-12-07 14:48:41 INFO  [pool-1-thread-1] - org.sonatype.nexus.feeds.DefaultFeedRecorder - Removing snapshots from repository repo1 : Process started on Wed Dec 07 14:39:04 NST 2011, finished successfully on Wed Dec 07 14:48:41 NST 2011
          
          Show
          Peter Lynch added a comment - After Tamas's alternate implementation, we have a different result that is much improved using latest master branch. jvm 1 | 2011-12-07 14:39:04 INFO [pool-1-thread-1] - org.sonatype.nexus.feeds.DefaultFeedRecorder - Removing snapshots from repository repo1 : Process started on Wed Dec 07 14:39:04 NST 2011, not yet finished. jvm 1 | 2011-12-07 14:39:04 INFO [pool-1-thread-1] - org.sonatype.nexus.maven.tasks.DefaultSnapshotRemover - Removing old SNAPSHOT deployments from repo1 repository. jvm 1 | 2011-12-07 14:41:22 INFO [pool-1-thread-1] - org.sonatype.nexus.proxy.maven.maven2.M2Repository - Clearing NFC cache in repository ID='repo1' from path='/' jvm 1 | 2011-12-07 14:48:41 INFO [pool-1-thread-1] - org.sonatype.nexus.feeds.DefaultFeedRecorder - Removing snapshots from repository repo1 : Process started on Wed Dec 07 14:39:04 NST 2011, finished successfully on Wed Dec 07 14:48:41 NST 2011
          Hide
          Tamás Cservenák added a comment -

          Second pull request, wrt "anti-optimization" Peter found out:
          https://github.com/sonatype/nexus/pull/131

          Show
          Tamás Cservenák added a comment - Second pull request, wrt "anti-optimization" Peter found out: https://github.com/sonatype/nexus/pull/131
          Hide
          Peter Lynch added a comment -

          Adding performance snapshot of 1.10.0 nexus-oss-webapp after Tamas's Parent-o-matic implementation was added

          Show
          Peter Lynch added a comment - Adding performance snapshot of 1.10.0 nexus-oss-webapp after Tamas's Parent-o-matic implementation was added
          Hide
          Peter Lynch added a comment -

          Notable differences between 1.9.2.3 and latest master branch:

          • latest master now generates GA level metadata files, 1.9.2.3 did not
          • latest master now stores under storage/<repoid>/.nexus proxy attributes and these get generated at end of snapshot removal, 1.9.2.3 generates these as well, but they get stored outside storage in nexus/proxy/attributes folder

          The above is intended behavior.

          Show
          Peter Lynch added a comment - Notable differences between 1.9.2.3 and latest master branch: latest master now generates GA level metadata files, 1.9.2.3 did not latest master now stores under storage/<repoid>/.nexus proxy attributes and these get generated at end of snapshot removal, 1.9.2.3 generates these as well, but they get stored outside storage in nexus/proxy/attributes folder The above is intended behavior.
          Hide
          Peter Lynch added a comment -

          prformance improvement and no noticeable regressions

          Show
          Peter Lynch added a comment - prformance improvement and no noticeable regressions

            People

            • Assignee:
              Peter Lynch
              Reporter:
              Rich Seddon
              Last Updated By:
              Rich Seddon
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Date of First Response: