Details
-
Type:
Bug
-
Status: Waiting for Review
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 3.37.0
-
Fix Version/s: None
-
Component/s: IQ Integration, NPM
-
Labels:
-
Story Points:5
-
Sprint:NXRM MadMax Sprint 31, NXRM MadMax Sprint 32, NXRM MadMax Sprint 39
-
Notability:2
Description
SYMPTOM:
A npm proxy repository, which remote URL is another Nexus npm proxy repo, does not get the latest metadata if the remote Nexus uses "Remove Quarantined Versions" feature.
REPRODUCE STEPS:
- Create "npm-test-proxy" with https://registry.npmjs.org/ and enabling "Remove Quarantined Versions"
- Create "npm-proxy-proxy" which remote URL uses above 'npm-test-proxy', and for testing purpose, type "0" for Maximum metadata age.
- Populate some data:
_BASE_URL1="http://localhost:8081/repository/npm-test-proxy" _BASE_URL2="http://localhost:8081/repository/npm-proxy-proxy" curl -I ${_BASE_URL1%/}/acorn/-/acorn-3.3.0.tgz curl -I ${_BASE_URL1%/}/acorn/-/acorn-8.7.0.tgz curl -I ${_BASE_URL2%/}/acorn/-/acorn-3.3.0.tgz curl -I ${_BASE_URL2%/}/acorn/-/acorn-8.7.0.tgz
- Download current metadata
$ curl -D- -s -o acorn_p1_1.json ${_BASE_URL1%/}/acorn HTTP/1.1 200 OK date: Wed, 02 Mar 2022 00:50:30 GMT server: Nexus/3.37.0-01 (PRO) x-content-type-options: nosniff content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation x-xss-protection: 1; mode=block last-modified: Mon, 27 Dec 2021 10:33:31 GMT <<< etag: W/"0c60fe55ed4ab74a25931ce134755a2c" <<< content-type: application/json transfer-encoding: chunked
$ curl -D- -s -o acorn_p2_1.json ${_BASE_URL2%/}/acorn HTTP/1.1 200 OK date: Wed, 02 Mar 2022 00:56:23 GMT server: Nexus/3.37.0-01 (PRO) x-content-type-options: nosniff content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation x-xss-protection: 1; mode=block last-modified: Mon, 27 Dec 2021 10:33:31 GMT <<< etag: W/"0c60fe55ed4ab74a25931ce134755a2c" <<< content-type: application/json transfer-encoding: chunked
Also, check the other attributes from the Browse page, such as Blob created, Blob updated, (two) last_modified (http://localhost:8081/#browse/browse:npm-test-proxy:acorn)
- Enable org.apache.http (or org.apache.http.headers) DEBUG logging.
- Enable IQ Audit and Quarantine capability on "npm-test-proxy"
- Re-download the npm metadata from "npm-test-proxy"
$ curl -D- -s -o acorn_p1_2.json ${_BASE_URL1%/}/acorn HTTP/1.1 200 OK date: Wed, 02 Mar 2022 01:31:44 GMT server: Nexus/3.37.0-01 (PRO) x-content-type-options: nosniff content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation x-xss-protection: 1; mode=block last-modified: Mon, 27 Dec 2021 10:33:31 GMT <<< etag: W/"0c60fe55ed4ab74a25931ce134755a2c" <<< content-type: application/json transfer-encoding: chunked
No change in the last-modified and etag but the file size is different:
$ ls -l acorn_p1_*.json -rwxrwxrwx 1 hosako staff 352579 2 Mar 10:50 acorn_p1_1.json -rwxrwxrwx 1 hosako staff 258483 2 Mar 11:31 acorn_p1_2.json
- Re-download the metadata from "npm-proxy-proxy"
$ curl -D- -s -o acorn_p2_2.json ${_BASE_URL2%/}/acorn HTTP/1.1 200 OK date: Wed, 02 Mar 2022 01:33:22 GMT server: Nexus/3.37.0-01 (PRO) x-content-type-options: nosniff content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation x-xss-protection: 1; mode=block last-modified: Mon, 27 Dec 2021 10:33:31 GMT <<< etag: W/"0c60fe55ed4ab74a25931ce134755a2c" <<< content-type: application/json transfer-encoding: chunked
and downloaded exact same json file...
$ ls -l acorn_p2_*.json -rwxrwxrwx 1 hosako staff 352700 2 Mar 11:00 acorn_p2_1.json -rwxrwxrwx 1 hosako staff 352700 2 Mar 11:33 acorn_p2_2.json
Also, nexus.log shows it received "304 Not Modified"
2022-03-02 01:00:35,094+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 >> GET /repository/npm-test-proxy/acorn HTTP/1.1 2022-03-02 01:00:35,094+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 >> If-Modified-Since: Mon, 27 Dec 2021 10:33:31 GMT 2022-03-02 01:00:35,095+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 >> If-None-Match: W/"0c60fe55ed4ab74a25931ce134755a2c" ... (snip) ... 2022-03-02 01:00:35,100+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 << HTTP/1.1 304 Not Modified 2022-03-02 01:00:35,101+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 << date: Wed, 02 Mar 2022 01:00:35 GMT 2022-03-02 01:00:35,101+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 << server: Nexus/3.37.0-01 (PRO) ... (snip) ... 2022-03-02 01:00:35,102+0000 DEBUG [qtp444422638-900] *UNKNOWN org.apache.http.headers - http-outgoing-189 << etag: W/"0c60fe55ed4ab74a25931ce134755a2c"
NOTE:
Also, after unquarantine from IQ web UI, the metadata is correctly updated but the last_modified and etag were not changed, so that the chained proxy still get the old metadata JSON.
EXPECTATION:
- The chained proxy (eg: npm-proxy-proxy) should be able to get the updated metadata from the remote Nexus. To do so, it would need to update at least attributes.content.etag field.
- If unquarantined on the remote Nexus, the chained proxy should get the latest metadata.
- Also, if no specific reason, I expect other attributes such as the last_modified, Blob updated etc. should be updated.
Attachments
Issue Links
- depends on
-
NEXUS-32575 SPIKE for Remove Quarantined Versions does not update asset attributes.content.etag
-
- New
-