Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-13959

PyPI hosted repository throws ORecordDuplicatedException when uploading multiple packages with the same name/version

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.2
    • Fix Version/s: None
    • Component/s: PyPI
    • Labels:
      None

      Description

      In some circumstances PyPI hosted repos will raise a 500 error when attempting to upload a package where an existing package with the following conditions exists:

      • Same name
      • Same version
      • Package name contains upper case letters and/or underscores

      This is happening for me when I'm trying to upload two artifacts for the same version of a package to support Python 2/Pyhton 2, so the file names are something like Flask_ShellPlus-0.0.3-cp27-none-any.whl and Flask_ShellPlus-0.0.3-cp36-none-any.whl

      Stack trace below, but here's my guess as to the problem:  I notice that the component names in a PyPI repo are stored in some normalized format (lower cased and some punctuation stripped/mangled).  The exception in the log is a duplicate record exception on component bucket/group/name/version.  I'd guess that maybe when the code looks for an existing component to put the uploaded file in, it uses the unmangled name, but then creates the component using the mangled name.

       

      Stack trace:

       

      {{com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #10:25110: found duplicated key 'OCompositeKey{keys=[#9:15, null, sqlalchemy, 1.1.12]}' in index 'component_bucket_group_name_version_idx' previously assigned to the record #10:24932
      at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:70) [na:na]
      at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:34) [na:na]
      at com.orientechnologies.orient.core.index.OIndexAbstract.putInSnapshot(OIndexAbstract.java:965) [na:na]
      at com.orientechnologies.orient.core.index.OIndexAbstract.applyIndexTxEntry(OIndexAbstract.java:1151) [na:na]
      at com.orientechnologies.orient.core.index.OIndexAbstract.addTxOperation(OIndexAbstract.java:762) [na:na]
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:101) [na:na]
      at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1211) [na:na]
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:606) [na:na]
      at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:156) [na:na]
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2689) [na:na]
      at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2658) [na:na]
      at org.sonatype.nexus.repository.storage.StorageTxImpl.commit(StorageTxImpl.java:170) [na:na]
      at sun.reflect.GeneratedMethodAccessor1819.invoke(Unknown Source) [na:na]
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [na:1.8.0_91]
      at java.lang.reflect.Method.invoke(Method.java:498) [na:1.8.0_91]
      at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53) [na:na]
      at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39) [na:na]
      at org.sonatype.nexus.common.stateguard.StateGuard$TransitionImpl.run(StateGuard.java:191) [na:na]
      at org.sonatype.nexus.common.stateguard.TransitionsInterceptor.invoke(TransitionsInterceptor.java:56) [na:na]
      at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:66) [na:na]
      at com.sun.proxy.$Proxy160.commit(Unknown Source) [na:na]
      at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:65) [na:na]
      at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:53) [na:na]
      at org.sonatype.nexus.repository.pypi.internal.HostedHandlers.lambda$2(HostedHandlers.java:101) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:45) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:41) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.view.Context$proceed$0.call(Unknown Source) [na:na]
      at org.sonatype.nexus.repository.pypi.internal.PyPiRecipeSupport$_closure1.doCall(PyPiRecipeSupport.groovy:90) [na:na]
      at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source) [na:na]
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [na:1.8.0_91]
      at java.lang.reflect.Method.invoke(Method.java:498) [na:1.8.0_91]
      at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) [na:na]
      at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) [na:na]
      at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) [na:na]
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019) [na:na]
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1084) [na:na]
      at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019) [na:na]
      at groovy.lang.Closure.call(Closure.java:426) [na:na]
      at org.codehaus.groovy.runtime.ConvertedClosure.invokeCustom(ConvertedClosure.java:53) [na:na]
      at org.codehaus.groovy.runtime.ConversionHandler.invoke(ConversionHandler.java:105) [na:na]
      at com.sun.proxy.$Proxy123.handle(Unknown Source) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46) [na:na]
      at org.sonatype.nexus.repository.view.Context.proceed(Context.java:79) [na:na]
      at org.sonatype.nexus.repository.view.Context.start(Context.java:102) [na:na]
      at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:58) [na:na]
      at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43) [na:na]
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:198) [na:na]
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:160) [na:na]
      at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:117) [na:na]
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [javax.servlet-api:3.1.0]
      at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287) [com.google.inject:4.0.0]
      at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277) [com.google.inject:4.0.0]
      at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182) [com.google.inject:4.0.0]
      at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71) [com.google.inject:4.0.0]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) [com.google.inject:4.0.0]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112) [org.apache.shiro.web:1.2.4]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112) [org.apache.shiro.web:1.2.4]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) [org.apache.shiro.web:1.2.4]
      at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85) [org.sonatype.nexus.security:3.0.2.02]
      at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) [org.apache.shiro.web:1.2.4]
      at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) [org.apache.shiro.core:1.2.4]
      at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) [org.apache.shiro.core:1.2.4]
      at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) [org.apache.shiro.core:1.2.4]
      at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) [org.apache.shiro.web:1.2.4]
      at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101) [org.sonatype.nexus.security:3.0.2.02]
      at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [org.apache.shiro.web:1.2.4]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97) [com.codahale.metrics.servlet:3.0.2]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:63) [org.sonatype.nexus.base:3.0.2.02]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:97) [org.sonatype.nexus.base:3.0.2.02]
      at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82) [com.google.inject:4.0.0]
      at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104) [com.google.inject:4.0.0]
      at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133) [com.google.inject:4.0.0]
      at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130) [com.google.inject:4.0.0]
      at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203) [com.google.inject:4.0.0]
      at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130) [com.google.inject:4.0.0]
      at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73) [org.sonatype.nexus.bootstrap:3.0.2.02]
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668) [org.eclipse.jetty.servlet:9.3.7.v20160115]
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [org.eclipse.jetty.servlet:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) [org.eclipse.jetty.security:9.3.7.v20160115]
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [org.eclipse.jetty.servlet:9.3.7.v20160115]
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) [org.eclipse.jetty.server:9.3.7.v20160115]
      at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175) [com.codahale.metrics.jetty9:3.0.2]
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.Server.handle(Server.java:517) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) [org.eclipse.jetty.server:9.3.7.v20160115]
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [org.eclipse.jetty.io:9.3.7.v20160115]
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [org.eclipse.jetty.io:9.3.7.v20160115]
      at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) [org.eclipse.jetty.io:9.3.7.v20160115]
      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) [org.eclipse.jetty.util:9.3.7.v20160115]
      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) [org.eclipse.jetty.util:9.3.7.v20160115]
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) [org.eclipse.jetty.util:9.3.7.v20160115]
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) [org.eclipse.jetty.util:9.3.7.v20160115]
      at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]}}{{}}

       

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            staticsnow@gmail.com Mike Poindexter
            Last Updated By:
            Peter Lynch Peter Lynch
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Date of First Response:

                tigCommentSecurity.panel-title