Tag / authentication

Update 2024-02-03: I just learned this week that index.docker.io is another DNS used for access to the Docker Hub registry. The behavior of index.docker.io is the same as registry-1.docker.io, so no changes to the commands are added. The list of DNS names is updated in the post below.

As part of my role in the Open Containers Initiative’s (OCI) Authentication Working Group, I started looking at the current state of authentication across registries. It is in my plan to document the current implementation in a list of registries including Docker Hub, GitHub Container Registry, ACR, ECR, GCR, NVIDIA, JFrog Artifactory, Harbor, and Zot. The goal is to identify differences and work on a proposal to unify those in a OCI registry authentication. For the purpose of this investigation, I will use only the APIs as specified in the OCI Distribution specification and the token authentication specified in Distribution Registry Authentication. For this post, I will concentrate on how Docker Hub currently (i.e. Jan 2024) handle authentication. This can be used as a baseline for comparison with other registries.

Registry Authentication Scenarios

For each of the registries above, I will look at the following authentication scenarios.

• Authentication for the root of the registry (i.e https://myregistry.provider.com/)
• Authentication flow for the /v2/ endpoint
• Authentication flow for the /v2/_catalog endpoint
• Authentication flow for any /v2/<repository> endpoint

Note that the distribution registry authentication flows also authorize the user for certain action on the repositories. I will look at how this is done and what permissions the users receive after each call.

For my tests, I will use the Visual Studio Code REST Client and will post the actual API calls below. You can easily reproduce those calls with curl.

Docker Hub Root URL Authentication

Before I look at the authentication for the root URL, there are a few different DNS entries you need to keep in mind when working with Docker Hub:

• hub.docker.com is the DNS used to access the Docker Hub’s web interface
• registry-1.docker.io is the DNS used to access the registry APIs
• index.docker.io is used to access the registry API (same as registry-1.docker.io)
• registry.docker.io is a placeholder DNS used to issue the authentication token for
• auth.docker.io is the authentication endpoint

This set up can be a bit confusing for novice users of the APIs and it also poses some challenges with the flows that I will explain below. From user experience point of view, the assumption may be to use the web site DNS for the APIs, which will be wrong. For Docker Hub, the main DNS that API calls should be made (based on my investigation) is registry-1.docker.io.

Hitting the root of registry-1.docker.io as follows does not return authentication challenge but 404 error.

### Get the registry root
### REQUEST 
GET https://registry-1.docker.io/

### RESPONSE
HTTP/1.1 404 Not Found
content-type: text/plain; charset=utf-8
docker-distribution-api-version: registry/2.0
x-content-type-options: nosniff
date: Tue, 30 Jan 2024 00:48:25 GMT
content-length: 19
strict-transport-security: max-age=31536000
connection: close

404 page not found

Docker Hub automatically switches to HTTPS if the root is requested via HTTP. I am mentioning this because not every registry implements this redirection (as I recall seeing in the past). The behavior of the root is not specified in any specification and it is left to the vendors to implement as they find convenient. There shouldn’t be any expectation for consistency across registries.

Docker Hub /v2/ Endpoint Authentication

According to the specifications /v2/ endpoint should return authentication challenge. Note that the trailing slash is required according to specification. Here the result of a simple GET request on /v2 (without trailing slash):

### Get the /v2 endpoint
### REQUEST 
GET https://registry-1.docker.io/v2

### RESPONSE
HTTP/1.1 401 Unauthorized
content-type: application/json
docker-distribution-api-version: registry/2.0
www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io"
date: Tue, 30 Jan 2024 00:58:22 GMT
content-length: 87
strict-transport-security: max-age=31536000
connection: close

{
  "errors": [
    {
      "code": "UNAUTHORIZED",
      "message": "authentication required",
      "detail": null
    }
  ]
}

HEAD request returns the same headers without the body. POST, PUT and DELETE requests return 301 redirect to /v2/ (with a slash), and I would expect that GET and HEAD do the same. It is a separate question if /v2/ (with a slash) is considered a resource or a path. I would normally consider this a path and not a resource. Nevertheless, GET and HEAD requests behave differently from POST, PUT and DELETE – I would expect all of them to have a permanent redirect from /v2 to /v2/.

When requesting /v2/ all methods return the same response:

### RESPONSE
HTTP/1.1 401 Unauthorized
content-type: application/json
docker-distribution-api-version: registry/2.0
www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io"
date: Tue, 30 Jan 2024 03:15:13 GMT
content-length: 87
strict-transport-security: max-age=31536000
connection: close

{
  "errors": [
    {
      "code": "UNAUTHORIZED",
      "message": "authentication required",
      "detail": null
    }
  ]
}

Note that the OCI spec does not specify any behavior for HEAD, POST, PUT, and DELETE for /v2/ endpoint. This means the implementation is left to the vendor. I would expect to see 405 (Method not supported) for anything else than GET. Current behavior on Docker Hub leaves me with the impression that I may be able to modify or delete the /v2/ (resource). OPTIONS request also works on the /v2/ endpoint with the same result as above.

Authentication Flow for /v2/ Endpoint on Docker Hub

Now that I know what is the behavior of the /v2/ endpoint, let’s go over the authentication flow. As you can see from the www-authenticate header above, I need to make a request to realm="https://auth.docker.io/token" with parameter service="registry.docker.io". Here is where things become confusing with the DNS names that Docker Hub returns. About that a bit later though. Here is the call to complete the flow:

### Authenticate for the /v2/ endpoint
### REQUEST
GET https://auth.docker.io/token?service=registry.docker.io

### RESPONSE
HTTP/1.1 200 OK
content-type: application/json
x-trace-id: 30f17bf77f44bfe9593e7ad8ccff8d1a
date: Tue, 30 Jan 2024 03:35:39 GMT
transfer-encoding: chunked
strict-transport-security: max-age=31536000
connection: close

{
  "token": "eyJhbGciOiJSUzI1N<redacted>",
  "access_token": "eyJhbGciOiJSUzI1N<redacted>",
  "expires_in": 300,
  "issued_at": "2024-01-30T03:35:39.896023447Z"
}

Using the retrieved access-token (which is the same as the token in the response body), I can call the /v2/ endpoint and get the expected response:

### Get the /v2/ endpoint with auth token
### REQUEST
GET https://registry-1.docker.io/v2/
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.1 200 OK
content-length: 2
content-type: application/json
docker-distribution-api-version: registry/2.0
date: Tue, 30 Jan 2024 03:39:05 GMT
strict-transport-security: max-age=31536000
connection: close

{}

Docker Hub Registry DNS

Now, what is the story with the Docker Hub registry DNS? The confusing part is that I interact with registry-1.docker.io but the authentication challenge service and the token’s audience are issued for registry.docker.io. I would expect the challenge service to be the same as the DNS I interact with. Here is what happens if you try to access /v2/ endpoint on registry.docker.io:

### Get the /v2/ endpoint with auth token on registry.docker.io
### REQUEST
GET https://registry.docker.io/v2/
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.0 503 Service Unavailable
cache-control: no-cache
content-type: text/html

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

This means that in my code I will need to have a special handling for Docker Hub to ignore the service from the authentication challenge (and aud , i.e. “Audience” in the token) and always use registry-1.docker.io.

/v2/ Endpoint Token Use

If you are wondering if you can use the token you received for the /v2/ endpoint for other purposes, the answer is “No”. Here is what happens if I try to use that token for requesting the  /v2/_catalog:

### Get the /v2/_catalog with /v2/ issues auth token
### REQUEST
GET https://registry-1.docker.io/v2/_catalog
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.1 401 Unauthorized
content-type: application/json
docker-distribution-api-version: registry/2.0
www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="registry:catalog:*",error="insufficient_scope"
date: Tue, 30 Jan 2024 03:53:37 GMT
content-length: 145
strict-transport-security: max-age=31536000
connection: close

{
  "errors": [
    {
      "code": "UNAUTHORIZED",
      "message": "authentication required",
      "detail": [
        {
          "Type": "registry",
          "Class": "",
          "Name": "catalog",
          "Action": "*"
        }
      ]
    }
  ]
}

Or eventually pull the manifest for any image:

### Get the /v2/library/python/manifests/3.10 with /v2/ issues auth token
### REQUEST
GET https://registry-1.docker.io/v2/library/python/manifests/3.10
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.1 401 Unauthorized
content-type: application/json
docker-distribution-api-version: registry/2.0
www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/python:pull",error="insufficient_scope"
date: Tue, 30 Jan 2024 03:55:44 GMT
content-length: 157
strict-transport-security: max-age=31536000
docker-ratelimit-source: 50.46.250.96
connection: close

{
  "errors": [
    {
      "code": "UNAUTHORIZED",
      "message": "authentication required",
      "detail": [
        {
          "Type": "repository",
          "Class": "",
          "Name": "library/python",
          "Action": "pull"
        }
      ]
    }
  ]
}

In both cases I get 401 (Unauthorized) error with “insufficient_scope”. I will need to request a new token with the respective scope for each one of the calls above.

Docker Hub _catalog Authentication Flow

Retrieving the catalog of the registry maybe a common operation that you may want to do. Interestingly the /v2/_catalog endpoint is not specified by OCI (and the page doesn’t exist anymore on Docker documentation). This means that the behavior of the /v2/_catalog endpoint is defined by the respective vendor. I should not expect any consistency across vendors. Based on the response I got above for the /v2/_catalog endpoint:

www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="registry:catalog:*",error="insufficient_scope"

I should make a request to realm="https://auth.docker.io/token" with parameters service=registry.docker.io&scope=registry:catalog:*. Trying to retrieve a token for this request returns the following:

### Authenticate for the /v2/_catalog endpoint
### REQUEST
GET https://auth.docker.io/token?service=registry.docker.io&scope=registry:catalog:*

### RESPONSE
HTTP/1.1 400 Bad Request
content-type: application/json
x-trace-id: ee3b9e9dfb535dee0f4980ffc007e62c
date: Tue, 30 Jan 2024 04:14:25 GMT
content-length: 36
strict-transport-security: max-age=31536000
connection: close

{
  "details": "unknown resource type"
}

Although the challenge tells you that you can (theoretically) request the catalog, the authentication call responds with “unknown resource type”. It is obvious that Docker doesn’t want to return the whole catalog of thousands of images but I would expect to receive a 404 error and save a request in this case.

Docker Hub Repository Authentication Flow

For this scenario, I will do two different tests. The first one will be to a common public repository, while the second one to my repository on Docker Hub.

Authentication Flow for Pulling from Public Repositories on Docker Hub

As you saw from the above request to pull the Python image manifest, I need to make a request to realm="https://auth.docker.io/token" with parameters service=registry.docker.io&scope=repository:library/python:pull. Here is what I can do with the token I retrieve from this authentication request:

### Authenticate for the /v2/library/python/manifests/3.10 endpoint
### REQUEST
GET https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/python:pull

### RESPONSE
HTTP/1.1 200 OK
content-type: application/json
x-trace-id: f4abf5cfc5be3e7d4d1ddb4e1954e55c
date: Tue, 30 Jan 2024 04:24:21 GMT
transfer-encoding: chunked
strict-transport-security: max-age=31536000
connection: close

{
  "token": "eyJhbGciOiJSUzI1N<redacted>",
  "access_token": "eyJhbGciOiJSUzI1N<redacted>",
  "expires_in": 300,
  "issued_at": "2024-01-30T04:24:21.670237722Z"
}

First, I can pull the manifest for the image. Of course, this is expected, because this is the original endpoint that I requested:

### Pull the manifest for the python:3.10 image
### REQUEST
GET https://registry-1.docker.io/v2/library/python/manifests/3.10
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.1 200 OK
content-length: 13886
content-type: application/vnd.docker.distribution.manifest.v1+prettyjws
docker-content-digest: sha256:10f8e59b0b74e09797ea8bcebb7fb20ce8afb1ac303a1c4fcd334f7599befa57
docker-distribution-api-version: registry/2.0
etag: "sha256:10f8e59b0b74e09797ea8bcebb7fb20ce8afb1ac303a1c4fcd334f7599befa57"
date: Tue, 30 Jan 2024 04:27:05 GMT
strict-transport-security: max-age=31536000
ratelimit-limit: 100;w=21600
ratelimit-remaining: 100;w=21600
docker-ratelimit-source: 50.46.250.96
connection: close

{
  "schemaVersion": 1,
  "name": "library/python",
  "tag": "3.10",
  "architecture": "amd64",
  ...
}

I can also list the tags:

### List the tags for the python repository
### REQUEST
GET https://registry-1.docker.io/v2/library/python/tags/list
Authorization: Bearer eyJhbGciOiJSUzI1N<redacted>

### RESPONSE
HTTP/1.1 200 OK
content-type: application/json
docker-distribution-api-version: registry/2.0
date: Tue, 30 Jan 2024 04:29:52 GMT
transfer-encoding: chunked
strict-transport-security: max-age=31536000
connection: close

{
  "name": "library/python",
  "tags": [
    "2",
    "2-alpine",
    "2-alpine3.10",
    "2-alpine3.11",
    "2-alpine3.4",
    ...
  ]
}

And eventually pull a blob for the image. It seems all pull operations are allowed with the token. Being curious, I decided to see what would happen if I request pull and push permissions.

### Authenticate for pull and push for the /v2/library/python/manifests/3.10 endpoint
### REQUEST
GET https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/python:pull,push

### RESPONSE
HTTP/1.1 200 OK
content-type: application/json
x-trace-id: 271b6fa7fb60fd7fcb546d9fb32a3d35
date: Tue, 30 Jan 2024 04:55:20 GMT
transfer-encoding: chunked
strict-transport-security: max-age=31536000
connection: close

{
  "token": "eyJhbGciOiJSUzI1N<redacted>",
  "access_token": "eyJhbGciOiJSUzI1NiI<redacted>",
  "expires_in": 300,
  "issued_at": "2024-01-30T04:55:20.734558819Z"
}

The requests succeeds but if I decode the token, I see that the allowed actions are only for pull:

{
      "actions": [
        "pull"
      ],
      "name": "library/python",
      "parameters": {
        "pull_limit": "100",
        "pull_limit_interval": "21600"
      },
      "type": "repository"
    }
  ],
  "aud": "registry.docker.io",
  "exp": 1706590717,
  "iat": 1706590417,
  "iss": "auth.docker.io",
  "jti": "dckr_jti_Q6sjQ6BF5_1o8LJJFE3qCkFsZlo=",
  "nbf": 1706590117,
  "sub": ""
}

Trying to POST, PUT, and DELETE results in 401 (Unauthorized) error. Of course, HEAD requests are possible with this token.

Authentication Flow for Pull, Push, and Delete from a Private Repositories on Docker Hub

All of the requests above were made without specifying user credentials. To test the full permissions, I will use my own repository on Docker Hub. I have also push and delete permissions on this repository. First requesting the token is done with basic authentication:

### Authenticate with username and password for the /v2/toddysm/python
### REQUEST
GET https://auth.docker.io/token?service=registry.docker.io&scope=repository:toddysm/python:pull,push,delete
Authorization: Basic <base64_encoded_username_and_password>

### RESPONSE
HTTP/1.1 200 OK
content-type: application/json
x-trace-id: 409339cb64a91bdfcbaa72da4ce35219
date: Tue, 30 Jan 2024 05:16:37 GMT
transfer-encoding: chunked
strict-transport-security: max-age=31536000
connection: close

{
  "token": "eyJhbGciOiJSUzI1N<redacted>",
  "access_token": "eyJhbGciOiJSUzI1NiI<redacted>",
  "expires_in": 300,
  "issued_at": "2024-01-30T05:16:37.110643594Z"
}

And with the token, you can push data or delete data:

### Delete a manifest with the auth token
### REQUEST
DELETE https://registry-1.docker.io/v2/toddysm/python/manifests/3.10
Authorization: Bearer eyJhbGciOiJSUzI1NiI<redacted>

### RESPONSE
HTTP/1.1 202 Accepted
docker-distribution-api-version: registry/2.0
date: Tue, 30 Jan 2024 05:18:33 GMT
content-length: 0
strict-transport-security: max-age=31536000
docker-ratelimit-source: 50.46.250.96
connection: close

Summary

Here is a quick summary of the main points above:

• Even for anonymous access, Docker Hub requires you to request a token to access its APIs.
• Docker Hub registry uses different DNS names for the actual registry endpoint and the token audience. This may have some impact on how you implement the API calls and the authentication header parsing.
• Docker Hub registry doesn’t support _catalog despite the fact that it responds with proper authentication challenge.
• When requesting a token for Docker Hub registry, do not rely that you have received the full permissions you requested. You may need to parse the token to discover the access you are given.
• Some of the request method responses may be confusing and not consistent with other registries.

This completes my investigation on the Docker Hub registry authentication. All commands are also available as a Gist.