The AWS SDK for S3 defaults to a max of 50 connections. If more than 50 connections are leased in the pool it will lead to exceptions. One observed case of this leads to a permanent state where the connections are stuck. We should expose options to increase the connection pool limit, change failure behavior, retry policies and/or timeout configuration.
- Advanced client configuration options are exposed and used
Other notes: Each S3 blob store creates its own client with corresponding connection pool. httpcomponents documentation (admittedly version 3 and not 4) specifies to try and share the HttpClient in a single application. http://hc.apache.org/httpclient-3.x/performance.html