Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-31363

Remove Quarantined Versions does not update asset attributes.content.etag

    Details

    • Type: Bug
    • Status: Waiting for Review
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.37.0
    • Fix Version/s: None
    • Component/s: IQ Integration, NPM
    • Labels:
    • Story Points:
      5
    • Sprint:
      NXRM MadMax Sprint 31, NXRM MadMax Sprint 32, NXRM MadMax Sprint 39
    • Notability:
      2

      Description

      SYMPTOM:

      A npm proxy repository, which remote URL is another Nexus npm proxy repo, does not get the latest metadata if the remote Nexus uses "Remove Quarantined Versions" feature.

      REPRODUCE STEPS:

      1. Create "npm-test-proxy" with https://registry.npmjs.org/ and enabling "Remove Quarantined Versions"
      2. Create "npm-proxy-proxy" which remote URL uses above 'npm-test-proxy', and for testing purpose, type "0" for Maximum metadata age.
      3. Populate some data:
        _BASE_URL1="http://localhost:8081/repository/npm-test-proxy"
        _BASE_URL2="http://localhost:8081/repository/npm-proxy-proxy"
        curl -I ${_BASE_URL1%/}/acorn/-/acorn-3.3.0.tgz 
        curl -I ${_BASE_URL1%/}/acorn/-/acorn-8.7.0.tgz
        curl -I ${_BASE_URL2%/}/acorn/-/acorn-3.3.0.tgz 
        curl -I ${_BASE_URL2%/}/acorn/-/acorn-8.7.0.tgz 
        
      4. Download current metadata
        $ curl -D- -s -o acorn_p1_1.json ${_BASE_URL1%/}/acorn 
        HTTP/1.1 200 OK
        date: Wed, 02 Mar 2022 00:50:30 GMT
        server: Nexus/3.37.0-01 (PRO)
        x-content-type-options: nosniff
        content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
        x-xss-protection: 1; mode=block
        last-modified: Mon, 27 Dec 2021 10:33:31 GMT  <<<
        etag: W/"0c60fe55ed4ab74a25931ce134755a2c"  <<<
        content-type: application/json
        transfer-encoding: chunked
        
        $ curl -D- -s -o acorn_p2_1.json ${_BASE_URL2%/}/acorn 
        HTTP/1.1 200 OK
        date: Wed, 02 Mar 2022 00:56:23 GMT
        server: Nexus/3.37.0-01 (PRO)
        x-content-type-options: nosniff
        content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
        x-xss-protection: 1; mode=block
        last-modified: Mon, 27 Dec 2021 10:33:31 GMT  <<<
        etag: W/"0c60fe55ed4ab74a25931ce134755a2c"  <<<
        content-type: application/json
        transfer-encoding: chunked
        

        Also, check the other attributes from the Browse page, such as Blob created, Blob updated, (two) last_modified (http://localhost:8081/#browse/browse:npm-test-proxy:acorn)

      5. Enable org.apache.http (or org.apache.http.headers) DEBUG logging.
      6. Enable IQ Audit and Quarantine capability on "npm-test-proxy"
      7. Re-download the npm metadata from "npm-test-proxy"
        $ curl -D- -s -o acorn_p1_2.json ${_BASE_URL1%/}/acorn 
        HTTP/1.1 200 OK
        date: Wed, 02 Mar 2022 01:31:44 GMT
        server: Nexus/3.37.0-01 (PRO)
        x-content-type-options: nosniff
        content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
        x-xss-protection: 1; mode=block
        last-modified: Mon, 27 Dec 2021 10:33:31 GMT  <<<
        etag: W/"0c60fe55ed4ab74a25931ce134755a2c"  <<<
        content-type: application/json
        transfer-encoding: chunked
        

        No change in the last-modified and etag but the file size is different:

        $ ls -l acorn_p1_*.json
        -rwxrwxrwx  1 hosako  staff  352579  2 Mar 10:50 acorn_p1_1.json
        -rwxrwxrwx  1 hosako  staff  258483  2 Mar 11:31 acorn_p1_2.json
        
      8. Re-download the metadata from "npm-proxy-proxy"
        $ curl -D- -s -o acorn_p2_2.json ${_BASE_URL2%/}/acorn 
        HTTP/1.1 200 OK
        date: Wed, 02 Mar 2022 01:33:22 GMT
        server: Nexus/3.37.0-01 (PRO)
        x-content-type-options: nosniff
        content-security-policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
        x-xss-protection: 1; mode=block
        last-modified: Mon, 27 Dec 2021 10:33:31 GMT  <<<
        etag: W/"0c60fe55ed4ab74a25931ce134755a2c"  <<<
        content-type: application/json
        transfer-encoding: chunked
        

        and downloaded exact same json file...

        $ ls -l acorn_p2_*.json
        -rwxrwxrwx  1 hosako  staff  352700  2 Mar 11:00 acorn_p2_1.json
        -rwxrwxrwx  1 hosako  staff  352700  2 Mar 11:33 acorn_p2_2.json
        

        Also, nexus.log shows it received "304 Not Modified"

        2022-03-02 01:00:35,094+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 >> GET /repository/npm-test-proxy/acorn HTTP/1.1
        2022-03-02 01:00:35,094+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 >> If-Modified-Since: Mon, 27 Dec 2021 10:33:31 GMT
        2022-03-02 01:00:35,095+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 >> If-None-Match: W/"0c60fe55ed4ab74a25931ce134755a2c"
        ... (snip) ...
        2022-03-02 01:00:35,100+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 << HTTP/1.1 304 Not Modified
        2022-03-02 01:00:35,101+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 << date: Wed, 02 Mar 2022 01:00:35 GMT
        2022-03-02 01:00:35,101+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 << server: Nexus/3.37.0-01 (PRO)
        ... (snip) ...
        2022-03-02 01:00:35,102+0000 DEBUG [qtp444422638-900]  *UNKNOWN org.apache.http.headers - http-outgoing-189 << etag: W/"0c60fe55ed4ab74a25931ce134755a2c"
        

      NOTE:

      Also, after unquarantine from IQ web UI, the metadata is correctly updated but the last_modified and etag were not changed, so that the chained proxy still get the old metadata JSON.

      EXPECTATION:

      • The chained proxy (eg: npm-proxy-proxy) should be able to get the updated metadata from the remote Nexus. To do so, it would need to update at least attributes.content.etag field.
      • If unquarantined on the remote Nexus, the chained proxy should get the latest metadata.
      • Also, if no specific reason, I expect other attributes such as the last_modified, Blob updated etc. should be updated.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              hosako Hajime Osako
              Last Updated By:
              Grace Lee Grace Lee
              Team:
              NXRM - Mad Max
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Date of First Response:

                  tigCommentSecurity.panel-title