Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28898

contention rebuilding Maven hosted archetype-catalog.xml during repository 2 to 3 migration

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Sprint:
      NXRM Neo Sprint 17, NXRM Neo Sprint 16
    • Notability:
      2

      Description

      During a repository 2.14.13 migration to repository 3.34.0 Postgres instance, the migration processing of NXRM asset records from repo 2 maven hosted repositories into the postgres database became very slow, making the migration process essentially ineffective.

      Upon analysis of the NXRM 3 threaddumps, 37 of the 48 total change-processing-threads had identical stacks, across 5 thread dumps spread across 15 seconds, all indicating rebuilding of maven archetype-catalog.xml file, and reading data from a database query related to that processing.

      The key paths suggesting rebuild are

      at org.sonatype.nexus.content.maven.internal.recipe.MavenArchetypeCatalogFacetImpl.rebuildArchetypeCatalog(MavenArchetypeCatalogFacetImpl.java:113) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenArchetypeCatalogFacetImpl.on(MavenArchetypeCatalogFacetImpl.java:97)
      
      change-processing-thread stack showing slow rebuild of archetype-catalog.xml
      at java.net.SocketInputStream.socketRead0(Native Method) 
      at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
      at java.net.SocketInputStream.read(SocketInputStream.java:171) 
      at java.net.SocketInputStream.read(SocketInputStream.java:141) 
      at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161) 
      at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:128) 
      at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:113) 
      at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73) 
      at org.postgresql.core.PGStream.receiveChar(PGStream.java:443) 
      at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2057) 
      at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:323) 
      - locked [0x00000005d7e33e90] (a org.postgresql.core.v3.QueryExecutorImpl) 
      at org.postgresql.jdbc.PgConnection.executeTransactionCommand(PgConnection.java:849) 
      at org.postgresql.jdbc.PgConnection.commit(PgConnection.java:871) 
      at org.postgresql.jdbc.PgConnection.setAutoCommit(PgConnection.java:813) 
      at com.zaxxer.hikari.pool.ProxyConnection.setAutoCommit(ProxyConnection.java:414) 
      at com.zaxxer.hikari.pool.HikariProxyConnection.setAutoCommit(HikariProxyConnection.java) 
      at org.apache.ibatis.transaction.jdbc.JdbcTransaction.resetAutoCommit(JdbcTransaction.java:125) 
      at org.apache.ibatis.transaction.jdbc.JdbcTransaction.close(JdbcTransaction.java:89) 
      at org.apache.ibatis.executor.BaseExecutor.close(BaseExecutor.java:90) 
      at org.apache.ibatis.executor.CachingExecutor.close(CachingExecutor.java:64) 
      at org.sonatype.nexus.datastore.mybatis.EntityExecutor.close(EntityExecutor.java:186) 
      at org.apache.ibatis.session.defaults.DefaultSqlSession.close(DefaultSqlSession.java:260) 
      at org.sonatype.nexus.datastore.mybatis.MyBatisDataSession.close(MyBatisDataSession.java:103) 
      at org.sonatype.nexus.transaction.UnitOfWork.doCloseSession(UnitOfWork.java:258) 
      at org.sonatype.nexus.transaction.UnitOfWork.close(UnitOfWork.java:192) 
      at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:56) 
      at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:75) 
      at com.google.inject.internal.InterceptorStackCallback.invoke(InterceptorStackCallback.java:55) 
      at org.sonatype.nexus.repository.content.store.AssetStore$$EnhancerByGuice$$713105658.transactionalSave( ) 
      at org.sonatype.nexus.repository.content.store.ContentStoreSupport.save(ContentStoreSupport.java:119) 
      at org.sonatype.nexus.repository.content.fluent.internal.FluentAssetBuilderImpl.save(FluentAssetBuilderImpl.java:138) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenContentFacetImpl.saveAsset(MavenContentFacetImpl.java:355) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenContentFacetImpl.save(MavenContentFacetImpl.java:278) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenContentFacetImpl.put(MavenContentFacetImpl.java:229) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenArchetypeCatalogFacetImpl.rebuildArchetypeCatalog(MavenArchetypeCatalogFacetImpl.java:113) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenArchetypeCatalogFacetImpl.on(MavenArchetypeCatalogFacetImpl.java:97) 
      at sun.reflect.GeneratedMethodAccessor1307.invoke(Unknown Source) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
      at java.lang.reflect.Method.invoke(Method.java:498) 
      at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87) 
      at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72) 
      at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) 
      at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67) 
      at com.google.common.eventbus.Dispatcher$ImmediateDispatcher.dispatch(Dispatcher.java:186) 
      at com.google.common.eventbus.EventBus.post(EventBus.java:212) 
      at org.sonatype.nexus.internal.event.EventManagerImpl.post(EventManagerImpl.java:127) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenContentFacetImpl.publishEvents(MavenContentFacetImpl.java:327) 
      at org.sonatype.nexus.content.maven.internal.recipe.MavenContentFacetImpl.maybeUpdateComponentAttributes(MavenContentFacetImpl.java:320) 
      at com.sonatype.nexus.migration.repository.migrators.datastore.Maven2RepositoryMigratorSupport.hardLinkContent(Maven2RepositoryMigratorSupport.java:85) 
      at com.sonatype.nexus.migration.repository.migrators.RepositoryMigratorSupport.ingestContent(RepositoryMigratorSupport.java:174) 
      at com.sonatype.nexus.migration.repository.migrators.RepositoryMigratorSupport.createOrUpdate(RepositoryMigratorSupport.java:153) 
      at com.sonatype.nexus.migration.repository.migrators.RepositoryMigratorSupport.processChange(RepositoryMigratorSupport.java:131) 
      at com.sonatype.nexus.migration.repository.RepositoryMigrator$processChange$3.call(Unknown Source) 
      at com.sonatype.nexus.migration.repository.ProcessChangesStep$_submit_closure2.doCall(ProcessChangesStep.groovy:340) 
      at com.sonatype.nexus.migration.repository.ProcessChangesStep$_submit_closure2.doCall(ProcessChangesStep.groovy) 
      at sun.reflect.GeneratedMethodAccessor660.invoke(Unknown Source) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
      at java.lang.reflect.Method.invoke(Method.java:498) 
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98) 
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) 
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:264) 
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034) 
      at groovy.lang.Closure.call(Closure.java:420) 
      at groovy.lang.Closure.call(Closure.java:414) 
      at groovy.lang.Closure.run(Closure.java:501) 
      at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40) 
      at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) 
      at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) 
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
      at java.lang.Thread.run(Thread.java:748)
      

      Expected

      While having an up to date archetype-catalog.xml would be desired after migration is finished, the value of constantly updating it during migration and slowing the overall migration process, compared with of value it brings ( only relevant to using the maven-archetype-plugin to generate templated maven projects when actually using NXRM 3) does not align.

      Remove rebuilding the maven-archetype-catalog.xml as a bottleneck to migration from repository 2 to repository 3.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dbradicich Damian Bradicich
              Reporter:
              plynch Peter Lynch
              Last Updated By:
              Vijay Swaminathan Vijay Swaminathan
              Team:
              NXRM - Neo
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title