Webhook destination site not responding can use up the connection pool

If Webhook destination remote site fails to respond the this can lead to outbound connection pool being fully used up, build up of threads and bring Nexus to a halt.

2018-12-12 14:47:59,558+0000 ERROR [event-10-thread-97] admin org.sonatype.nexus.internal.webhooks.WebhookServiceImpl - Failed to send webhook request: WebhookRequest{id='d4e68cb6-2ecb-45ef-9e55-0e6226ba6b11', webhook=rm:repository:asset, payload=org.sonatype.nexus.repository.webhooks.RepositoryAssetWebhook$RepositoryAssetWebhookPayload@54a81a40, url=https://webhook.site/c3ed696b-9544-40f2-82d3-7eb956f03404}

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool

 at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:313)

 at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:279)

 at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:191)

 at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)

 at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)

 at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)

 at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)

 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)

 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

 at org.apache.http.client.HttpClient$execute$0.call(Unknown Source)

With proxy repositories, we have auto block feature to prevent this. There is no autoblock for webhooks.

Reproduced this by increasing the http timeout to 500 sec and the throttle charles proxy when running builds deploying to a repository with webhook updates.