[NEXUS-26037] outbound docker requests do not implement docker api v2 tags/list pagination via RFC5988 Link header - Sonatype JIRA

XML

Word

Printable

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 3.28.0
Fix Version/s: 3.37.0
Component/s: Docker
Labels:
- oncall
- supportant

Story Points:
2
Sprint:
NXRM MadMax Sprint 19
Notability:
2

Description

The Docker API spec v2 cites pagination as supported feature for the tags/list responses

https://docs.docker.com/registry/spec/api/#pagination-1

PAGINATION
Paginated tag results can be retrieved by adding the appropriate parameters to the request URL described above. The behavior of tag pagination is identical to that specified for catalog pagination. We cover a simple flow to highlight any differences.

Starting a paginated flow may begin as follows:
GET /v2/<name>/tags/list?n=<integer>
The above specifies that a tags response should be returned, from the start of the result set, ordered lexically, limiting the number of results to n. The response to such a request would look as follows:
200 OK
Content-Type: application/json
Link: <<url>?n=<n from the request>&last=<last tag value from previous response>>; rel="next"
{
  "name": <name>,
  "tags": [
    <tag>,
    ...
  ]
}
To get the next result set, a client would issue the request as follows, using the value encoded in the RFC5988 Link header:
GET /v2/<name>/tags/list?n=<n from the request>&last=<last tag value from previous response>
The above process should then be repeated until the Link header is no longer set in the response. The behavior of the last parameter, the provided response result, lexical ordering and encoding of the Link header are identical to that of catalog pagination.

NXRM outbound docker requests do not seem to support pagination, and as such when asking NXRM to list all docker tags for an image that a remote provides via pagination, NXRM will return only a partial list of tags, those on the first page.

Example Reproduce

This lists all tags, because skopeo will follow every Link header until quay.io does not return any more:

skopeo list-tags docker://quay.io/kubernetes-ingress-controller/nginx-ingress-controller
{
    "Repository": "quay.io/kubernetes-ingress-controller/nginx-ingress-controller",
    "Tags": [
        "0.21.0",
        "0.22.0",
        "0",
        "0.19.0",
        "0.8",
        "0.8.1",
        "0.8.2",
        "0.8.3",
        "0.9.0-beta.1",
        "0.9.0-beta.10",
        "0.9.0-beta.11",
        "0.9.0-beta.12",
        "0.9.0-beta.13",
        "0.9.0-beta.14",
        "0.9.0-beta.15",
        "0.9.0-beta.2",
        "0.9.0-beta.3",
        "0.9.0-beta.4",
        "0.9.0-beta.5",
        "0.9.0-beta.6",
        "0.9.0-beta.7",
        "0.9.0-beta.8",
        "0.4",
        "0.5",
        "0.6",
        "0.61",
        "0.62",
        "0.9.0",
        "0.7",
        "0.9.0-beta.16",
        "0.18.0",
        "0.9.0-beta.19",
        "0.14.0",
        "0.20.0",
        "0.12.0",
        "0.11.0",
        "0.16.2",
        "0.10.1",
        "0.13.0",
        "0.9.0-beta.18",
        "0.10.0",
        "0.9.0-beta.17",
        "0.10.2",
        "0.16.1",
        "0.16.0",
        "0.15.0",
        "0.17.0",
        "0.17.1",
        "0.23.0",
        "0.24.0",
        "0.24.1",
        "0.25.0",
        "0.25.1",
        "0.26.0",
        "0.26.1",
        "0.26.2",
        "0.27.0",
        "0.27.1",
        "master",
        "0.28.0",
        "0.29.0",
        "0.30.0",
        "0.31.0",
        "0.31.1",
        "0.32.0",
        "0.33.0"
    ]
}

However if NXRM proxies https://quay.io, and skopeo uses NXRM, then only a partial list of tags is returned to the client:

skopeo list-tags docker://192.168.2.73:15751/kubernetes-ingress-controller/nginx-ingress-controller
{
    "Repository": "192.168.2.73:15751/kubernetes-ingress-controller/nginx-ingress-controller",
    "Tags": [
        "0",
        "0.10.0",
        "0.10.1",
        "0.10.2",
        "0.11.0",
        "0.12.0",
        "0.13.0",
        "0.14.0",
        "0.15.0",
        "0.16.0",
        "0.16.1",
        "0.16.2",
        "0.17.0",
        "0.17.1",
        "0.18.0",
        "0.19.0",
        "0.20.0",
        "0.21.0",
        "0.22.0",
        "0.23.0",
        "0.24.0",
        "0.4",
        "0.5",
        "0.6",
        "0.61",
        "0.62",
        "0.7",
        "0.8",
        "0.8.1",
        "0.8.2",
        "0.8.3",
        "0.9.0",
        "0.9.0-beta.1",
        "0.9.0-beta.10",
        "0.9.0-beta.11",
        "0.9.0-beta.12",
        "0.9.0-beta.13",
        "0.9.0-beta.14",
        "0.9.0-beta.15",
        "0.9.0-beta.16",
        "0.9.0-beta.17",
        "0.9.0-beta.18",
        "0.9.0-beta.19",
        "0.9.0-beta.2",
        "0.9.0-beta.3",
        "0.9.0-beta.4",
        "0.9.0-beta.5",
        "0.9.0-beta.6",
        "0.9.0-beta.7",
        "0.9.0-beta.8"
    ]
}

Expected

Provide a safe mechanism for NXRM as a Docker client to follow RFC5988 Link headers a remote may return to it as part of Docker API V2 spec, thus providing an equivalent experience that other Docker Clients will have if they implement Pagination talking directly to the remote registry.

Attachments

Issue Links

causes

NEXUS-31322 docker tags/list outbound paged request can consume all available outbound connections for the proxy repository

Closed

is related to

NEXUS-35956 docker group does not return Link header (pagination feature)

mentioned in: Page Loading...

Activity

People

Assignee:: Igor Udovika

Reporter:: Peter Lynch

Last Updated By:: Jerome Garec

Team:: NXRM - Mad Max

Votes:: 0 Vote for this issue

Watchers:: 13 Start watching this issue

Dates

Created:: 12/07/20 07:14 PM

Updated:: 10/31/22 05:14 PM

Resolved:: 10/25/21 09:19 AM

Date of First Response:: 08/Dec/20 2:22 PM