I setup the s3 blob store tonight and assigned it to a docker repo which hosts our base containers. In our normal CI job that will build all the base containers we where observing that it could take as much as 45 minutes to upload a container.
The server (even the network in and out) was largely idle during everything, almost no CPU, memory or anything else.
I tried to manually upload some large files to the S3 bucket we are using from the server and uploaded a 10GB file in a few seconds.
Digging around in some of the stacktrace output I noticed most of the HTTP threads where blocked and waiting on a lock to free up to do the upload. From what I could tell the whole thing is single threaded for uploads.
Is this true? is there anything I can do to increase concurrency or performance?