Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-17466

S3 Blob Store really slow to upload with docker

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Docker, S3
    • Labels:
      None

      Description

      I setup the s3 blob store tonight and assigned it to a docker repo which hosts our base containers. In our normal CI job that will build all the base containers we where observing that it could take as much as 45 minutes to upload a container.

      The server (even the network in and out) was largely idle during everything, almost no CPU, memory or anything else.

      I tried to manually upload some large files to the S3 bucket we are using from the server and uploaded a 10GB file in a few seconds.

      Digging around in some of the stacktrace output I noticed most of the HTTP threads where blocked and waiting on a lock to free up to do the upload. From what I could tell the whole thing is single threaded for uploads.

      Is this true? is there anything I can do to increase concurrency or performance?

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              mattc Matthew Christiansen
              Last Updated By:
              Peter Lynch Peter Lynch
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Date of First Response:

                  tigCommentSecurity.panel-title