Uploaded image for project: 'Dev - Nexus Repo'
  1. Dev - Nexus Repo
  2. NEXUS-28391

Failed to install docker plugins with registry-mirror

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: New
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.32.0
    • Fix Version/s: None
    • Component/s: Docker
    • Labels:

      Description

      We have configured docker hub proxy repo.

      Image pulling works ok.

      But we have problem with setting up plugins.

      [root@localhost docker]# docker plugin install grafana/loki-docker-driver:latest
      Error response from daemon: error resolving plugin reference: unexpected status code [manifests latest]: 502 Bad Gateway
      

      If i remove registry-mirror from daemon.json (or use

      docker plugin install registry-1.docker.io/grafana/loki-docker-driver:latest), it will install plugin.

       

      Debug log without proxy

      Jul 16 00:31:14 localhost dockerd: time="2021-07-16T00:31:14.177649305-04:00" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/20.10.7 go/go1.13.15 git-commit/b0f5bc3 kernel/3.10.0-1160.21.1.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.7 \\(linux\\))" request.method=HEAD url="https://registry-1.docker.io/v2/grafana/loki-docker-driver/manifests/latest"
      Jul 16 00:31:14 localhost dockerd: time="2021-07-16T00:31:14.690262484-04:00" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=169 response.header.content-type=application/json response.header.date="Fri, 16 Jul 2021 04:31:14 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:grafana/loki-docker-driver:pull\"" response.status="401 Unauthorized" url="https://registry-1.docker.io/v2/grafana/loki-docker-driver/manifests/latest"
      Jul 16 00:31:14 localhost dockerd: time="2021-07-16T00:31:14.690328309-04:00" level=debug msg=Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:grafana/loki-docker-driver:pull\"" host=registry-1.docker.io
      Jul 16 00:31:14 localhost dockerd: time="2021-07-16T00:31:14.690378363-04:00" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/20.10.7 go/go1.13.15 git-commit/b0f5bc3 kernel/3.10.0-1160.21.1.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.7 \\(linux\\))" request.method=HEAD url="https://registry-1.docker.io/v2/grafana/loki-docker-driver/manifests/latest"
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.317191169-04:00" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=519 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Fri, 16 Jul 2021 04:31:15 GMT" response.header.docker-content-digest="sha256:63435d7ee80a2734cc270aadd637f129f54a8cd97f21033b4aa6def50bd5f666" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:63435d7ee80a2734cc270aadd637f129f54a8cd97f21033b4aa6def50bd5f666\"" response.header.ratelimit-limit="100;w=21600" response.header.ratelimit-remaining="62;w=21600" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/grafana/loki-docker-driver/manifests/latest"
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.317260706-04:00" level=debug msg=resolved desc.digest="sha256:63435d7ee80a2734cc270aadd637f129f54a8cd97f21033b4aa6def50bd5f666" host=registry-1.docker.io
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.320151507-04:00" level=debug msg="hostDir: /etc/docker/certs.d/registry-1.docker.io"
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.320199200-04:00" level=debug msg="Resolved registry hosts" hosts="[{0xc000fd62a0 0xc000fd6300 registry-1.docker.io https /v2 7 map[]}]" registryHost=registry-1.docker.io
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.320304823-04:00" level=debug msg=fetch digest="sha256:63435d7ee80a2734cc270aadd637f129f54a8cd97f21033b4aa6def50bd5f666" mediatype=application/vnd.docker.distribution.manifest.v2+json size=519
      Jul 16 00:31:15 localhost dockerd: time="2021-07-16T00:31:15.320599664-04:00" level=debug msg=fetch digest="sha256:3365f75c342dd674212df9648d334502b69fc7f144a5dd1cd15485c2442bc083" mediatype=application/vnd.docker.plugin.v1+json size=830
      

      debug with mirror

      Jul 16 00:42:54 localhost dockerd: time="2021-07-16T00:42:54.989960616-04:00" level=debug msg="Resolved registry hosts" hosts="[{0xc00070c900 0xc00070c960 docker-proxy.example.com https /v2 3 map[]} {0xc00070c990 0xc00070c9f0 registry-1.docker.io https /v2 7 map[]}]" registryHost=docker.io
      Jul 16 00:42:54 localhost dockerd: time="2021-07-16T00:42:54.990013022-04:00" level=debug msg=resolving host=docker-proxy.example.com
      Jul 16 00:42:54 localhost dockerd: time="2021-07-16T00:42:54.990032474-04:00" level=debug msg="do request" host=docker-proxy.example.com request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent="docker/20.10.7 go/go1.13.15 git-commit/b0f5bc3 kernel/3.10.0-1160.21.1.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.7 \\(linux\\))" request.method=HEAD url="https://docker-proxy.example.com/v2/grafana/loki-docker-driver/manifests/latest?ns=docker.io"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.356357373-04:00" level=debug msg="fetch response received" host=docker-proxy.example.com response.header.connection=keep-alive response.header.content-length=168 response.header.content-security-policy="sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation" response.header.content-type=application/json response.header.date="Fri, 16 Jul 2021 04:42:55 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server=nginx/1.19.6 response.header.x-content-type-options=nosniff response.header.x-xss-protection="1; mode=block" response.status="502 Bad Gateway" url="https://docker-proxy.example.com/v2/grafana/loki-docker-driver/manifests/latest?ns=docker.io"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.356444317-04:00" level=debug msg="Error while resolving reference, falling back to backwards compatible accept header format" error="unexpected status code [manifests latest]: 502 Bad Gateway" ref="docker.io/grafana/loki-docker-driver:latest"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.356533978-04:00" level=debug msg="Resolved registry hosts" hosts="[{0xc00070d530 0xc00070d5c0 docker-proxy.example.com https /v2 3 map[]} {0xc00070d5f0 0xc00070d650 registry-1.docker.io https /v2 7 map[]}]" registryHost=docker.io
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.356583820-04:00" level=debug msg=resolving host=docker-proxy.example.com
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.356604667-04:00" level=debug msg="do request" host=docker-proxy.example.com request.header.accept=application/vnd.docker.distribution.manifest.v2+json request.header.accept.1=application/vnd.docker.distribution.manifest.list.v2+json request.header.accept.2=application/vnd.oci.image.manifest.v1+json request.header.accept.3=application/vnd.oci.image.index.v1+json request.header.user-agent="docker/20.10.7 go/go1.13.15 git-commit/b0f5bc3 kernel/3.10.0-1160.21.1.el7.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.7 \\(linux\\))" request.method=HEAD url="https://docker-proxy.example.com/v2/grafana/loki-docker-driver/manifests/latest?ns=docker.io"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.722295686-04:00" level=debug msg="fetch response received" host=docker-proxy.example.com response.header.connection=keep-alive response.header.content-length=168 response.header.content-security-policy="sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation" response.header.content-type=application/json response.header.date="Fri, 16 Jul 2021 04:42:55 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.server=nginx/1.19.6 response.header.x-content-type-options=nosniff response.header.x-xss-protection="1; mode=block" response.status="502 Bad Gateway" url="https://docker-proxy.example.com/v2/grafana/loki-docker-driver/manifests/latest?ns=docker.io"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.722390917-04:00" level=debug msg="Failed to resolve reference after falling back to backwards compatible accept header format" error="unexpected status code [manifests latest]: 502 Bad Gateway" ref="docker.io/grafana/loki-docker-driver:latest"
      Jul 16 00:42:55 localhost dockerd: time="2021-07-16T00:42:55.722691598-04:00" level=debug msg="FIXME: Got an API for which error does not match any expected type!!!: unexp.....
      

      Looks like a bug.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            e.prichinin Evgeniy P
            Last Updated By:
            Dawid Sawa Dawid Sawa
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Date of First Response:

                tigCommentSecurity.panel-title