[NEXUS-32449] blobstore outbound http connection or file handle leak while rebuilding Helm index - Sonatype JIRA

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: 3.38.0, 3.38.1
Fix Version/s: 3.42.0
Component/s: helm, S3
Labels:
- performance
- supportant

Notability:
3

Description

A customer had an S3 blobstore configured for their helm repositories:

Only requests related to helm repositories were causing failures, other repo formats with their own blobstore were not experiencing problems.

There is a leak of either HTTP connections for S3 / Azure blobstores, or file handles for File based blobstores.

Example for S3 blobstore:

2022-05-25 10:41:47,881+0000 ERROR [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Error while rebuilding Helm index for repository helm-public
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool

Full messages:

2022-05-25 10:40:04,747+0000 INFO  [event-27-thread-12208]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:05,747+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:06,330+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:07,248+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:08,248+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:08,796+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:09,630+0000 INFO  [event-27-thread-12208]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:10,630+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:11,267+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:12,065+0000 INFO  [event-27-thread-12209]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:13,065+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:13,698+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:14,220+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:15,220+0000 INFO  [event-27-thread-12208]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:15,833+0000 INFO  [event-27-thread-12208]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:16,321+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:17,321+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:17,849+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:18,401+0000 INFO  [event-27-thread-12209]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:19,402+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:20,004+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:20,449+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:21,450+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:22,025+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:23,980+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:24,980+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:40:25,600+0000 INFO  [event-27-thread-12205]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Finished rebuilding Helm index for repository helm-public
2022-05-25 10:40:25,900+0000 INFO  [event-27-thread-12207]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Scheduling rebuild of Helm metadata to start in 1 seconds
2022-05-25 10:40:26,900+0000 INFO  [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Rebuilding Helm index for repository helm-public
2022-05-25 10:41:07,921+0000 ERROR [periodic-4-thread-1]  *SYSTEM org.sonatype.nexus.blobstore.s3.internal.S3BlobStoreMetricsStore - Cannot write blob store metrics
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:394)
	at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5950)
	at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1812)
	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1772)
	at com.amazonaws.services.s3.EncryptingAmazonS3Client.putObject(EncryptingAmazonS3Client.java:157)
	at com.amazonaws.services.s3.EncryptingAmazonS3Client.putObject(EncryptingAmazonS3Client.java:150)
	at org.sonatype.nexus.blobstore.s3.internal.S3PropertiesFile.write(S3PropertiesFile.java:84)
	at org.sonatype.nexus.blobstore.s3.internal.S3PropertiesFile.write(S3PropertiesFile.java:1)
	at org.sonatype.nexus.blobstore.CloudBlobPropertiesSupport.store(CloudBlobPropertiesSupport.java:55)
	at org.sonatype.nexus.blobstore.BlobStoreMetricsStoreSupport.flushProperties(BlobStoreMetricsStoreSupport.java:173)
	at org.sonatype.nexus.blobstore.BlobStoreMetricsStoreSupport.lambda$0(BlobStoreMetricsStoreSupport.java:134)
	at org.sonatype.nexus.scheduling.internal.PeriodicJobServiceImpl.lambda$1(PeriodicJobServiceImpl.java:90)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:282)
	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
	at com.amazonaws.http.conn.$Proxy343.get(Unknown Source)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	... 28 common frames omitted
2022-05-25 10:41:47,881+0000 ERROR [event-27-thread-12206]  svc.rit-nexus-all com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl - Error while rebuilding Helm index for repository helm-public
com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1490)
	at com.amazonaws.services.s3.EncryptingAmazonS3Client.getObject(EncryptingAmazonS3Client.java:164)
	at com.amazonaws.services.s3.EncryptingAmazonS3Client.getObject(EncryptingAmazonS3Client.java:171)
	at org.sonatype.nexus.blobstore.s3.internal.S3BlobStore$S3Blob.doGetInputStream(S3BlobStore.java:692)
	at org.sonatype.nexus.blobstore.BlobSupport.getInputStream(BlobSupport.java:87)
	at org.sonatype.nexus.repository.mime.DefaultContentValidator.determineContentType(DefaultContentValidator.java:71)
	at org.sonatype.nexus.repository.content.facet.AssetBlobValidators.lambda$0(AssetBlobValidators.java:61)
	at org.sonatype.nexus.repository.content.facet.ContentFacetSupport.determineContentType(ContentFacetSupport.java:326)
	at org.sonatype.nexus.repository.content.facet.ContentFacetSupport.checkContentType(ContentFacetSupport.java:311)
	at org.sonatype.nexus.repository.content.fluent.internal.FluentAssetBuilderImpl.makePermanent(FluentAssetBuilderImpl.java:211)
	at org.sonatype.nexus.repository.content.fluent.internal.FluentAssetBuilderImpl.lambda$0(FluentAssetBuilderImpl.java:106)
	at org.sonatype.nexus.repository.content.fluent.internal.FluentAssetBuilderImpl.save(FluentAssetBuilderImpl.java:136)
	at com.sonatype.repository.helm.datastore.internal.recipe.HelmContentFacetImpl.putIndex(HelmContentFacetImpl.java:104)
	at com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl.createIndexYaml(CreateIndexFacetImpl.java:180)
	at com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl.updateIndexYaml(CreateIndexFacetImpl.java:173)
	at com.sonatype.repository.helm.datastore.internal.createindex.CreateIndexFacetImpl.on(CreateIndexFacetImpl.java:158)
	at sun.reflect.GeneratedMethodAccessor637.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
	at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:144)
	at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
	at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40)
	at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
	at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: com.amazonaws.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
		at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
		at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
		at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
		at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
		at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1338)
		at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1312)
		at com.amazonaws.services.s3.AmazonS3Client.doesObjectExist(AmazonS3Client.java:1393)
		at org.sonatype.nexus.blobstore.s3.internal.S3PropertiesFile.exists(S3PropertiesFile.java:94)
		at org.sonatype.nexus.blobstore.s3.internal.S3BlobAttributes.load(S3BlobAttributes.java:44)
		at org.sonatype.nexus.blobstore.s3.internal.S3BlobStore.exists(S3BlobStore.java:827)
		at com.palominolabs.metrics.guice.TimedInterceptor.invoke(TimedInterceptor.java:26)
		at org.sonatype.nexus.blobstore.group.BlobStoreGroup.lambda$10(BlobStoreGroup.java:268)
		at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
		at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
		at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
		at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
		at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
		at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
		at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
		at org.sonatype.nexus.blobstore.group.BlobStoreGroup.deleteHard(BlobStoreGroup.java:269)
		at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
		at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
		at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:54)
		at org.sonatype.nexus.blobstore.api.BlobStore.deleteIfTemp(BlobStore.java:342)
		at org.sonatype.nexus.repository.view.payloads.TempBlob.close(TempBlob.java:99)
		at com.sonatype.repository.helm.datastore.internal.recipe.HelmContentFacetImpl.putIndex(HelmContentFacetImpl.java:108)
		... 15 common frames omitted
	Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
		at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)
		at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:282)
		at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.lang.reflect.Method.invoke(Method.java:498)
		at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
		at com.amazonaws.http.conn.$Proxy343.get(Unknown Source)
		at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
		at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
		at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
		at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330)
		at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
		... 46 common frames omitted
Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:282)
	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
	at com.amazonaws.http.conn.$Proxy343.get(Unknown Source)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1330)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	... 37 common frames omitted

Expected

Do not leak S3 blobstore connections or file handles for Helm Repositories.

Workaround

To clear the leak, a restart of Nexus Repository is required.

If the leak surfaces after a long period of time, then more frequent restarts might help avoid the issue, along with increasing the S3 default connection pool size.

Until the problem is fixed, then not using S3 blobstores for Helm format repositories might be preferred.

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Peter Lynch

Last Updated By:: Eugene Bulatnikov

Team:: NXRM - Sentinels

Votes:: 3 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 05/25/22 06:51 PM

Updated:: 08/09/22 01:44 PM

Resolved:: 08/09/22 01:44 PM

Date of First Response:: 02/Jun/22 12:27 AM