401 Unauthorized on some API endpoints when using OIDC

[SimonAlling]

Expected behavior and actual behavior

I expect to be able to access all API endpoints that my user has access to via e.g. curl, using my CLI secret for authentication.

However, when using an OIDC user account, I cannot access some endpoints:

• I can GET the list of projects, but I cannot GET a specific project.
• I can GET the list of repositories in a project, but I cannot GET a specific repository.
• I can GET the list of artifacts in a repository, but I cannot GET a specific artifact (or e.g. its vulnerabilities).

Specifically, when using curl, I get a 401 response with this body:

{"errors":[{"code":"UNAUTHORIZED","message":"unauthorized"}]}

I can access all endpoints with curl using a robot account or the admin account (which do not use OIDC).

I can also access all endpoints in the web GUI and the API Explorer (/devcenter-api-2.0) using my OIDC account.

Steps to reproduce the problem

HARBOR_USERNAME="bob"
 HARBOR_PASSWORD="…"
HARBOR_URL="https://example.com"
HARBOR_PROJECT="foo"
HARBOR_REPO="bar"
HARBOR_TAG="baz"
API_PATHS=(
    "/projects"
    "/projects/${HARBOR_PROJECT:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts/${HARBOR_TAG:?}"
    "/projects/${HARBOR_PROJECT:?}/repositories/${HARBOR_REPO:?}/artifacts/${HARBOR_TAG:?}/additions/vulnerabilities"
)

for apiPath in "${API_PATHS[@]}"; do
    curl --silent --output /dev/null --write-out "%{http_code} %{url_effective}\n" -u ${HARBOR_USERNAME:?}:${HARBOR_PASSWORD:?} "${HARBOR_URL:?}/api/v2.0${apiPath}"
done

Output:

200 https://example.com/api/v2.0/projects
401 https://example.com/api/v2.0/projects/foo
200 https://example.com/api/v2.0/projects/foo/repositories
401 https://example.com/api/v2.0/projects/foo/repositories/bar
200 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts
401 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts/baz
401 https://example.com/api/v2.0/projects/foo/repositories/bar/artifacts/baz/additions/vulnerabilities

Lines like this one show up in the log:

2022-04-04T14:00:00Z [ERROR] [/server/middleware/security/basic_auth.go:40][requestID="…"]: failed to authenticate bob: not supported

The error is thrown here.

Additional information

Notably, it turns out that in the web GUI and the API Explorer, a cookie named sid is included in each request. That cookie alone turns out to be enough for authentication and authorization – the Authorization header (i.e. curl's -u flag) is not even needed:

 HARBOR_SID="deadbeefdeadbeefdeadbeefdeadbeef"
curl --silent --output /dev/null --write-out "%{http_code} %{url_effective}\n" --cookie "sid=${HARBOR_SID:?}" "${HARBOR_URL:?}/api/v2.0/projects/${HARBOR_PROJECT:?}"

As previously stated, the cookie is necessary for some endpoints (when accessed by an OIDC user). However, the API Explorer generates commands like this one, without any cookie:

curl -X 'GET' \
  'https://example.com/api/v2.0/projects/foo' \
  -H 'accept: application/json' \
  -H 'X-Is-Resource-Name: false' \
  -H 'authorization: Basic Ym9iOm15IENMSSBzZWNyZXQK'

All in all, there seems to be some inconsistency regarding OIDC authentication:

• When the API is accessed by an OIDC user, some endpoints (e.g. /projects/foo) require an OIDC cookie, while others (e.g. /projects) do not.
• In the API Explorer, commands without necessary authentication information are displayed.

Versions

• Harbor version: 2.4.1
• docker engine version: N/A
• docker-compose version: N/A

[SimonAlling]

Probably related: #16398

[MinerYang]

Please refer to this #16398 (comment)
Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

[lindhe]

Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

For anyone else getting confused by this: search for just "OIDC", not "API OIDC", on that Wiki page.

[lindhe]

3. My Harbor is configured to use OIDC for authentication, how do I access Harbor's API?

[A] In such case, you have to use the OIDC token as bearer token to access Harbor's API.

That's clearly not true. You don't have to use the OIDC token. That is only required sometimes, depending on which API endpoint is used for the call.

[MinerYang]

Correction: search for OIDC on https://github.com/goharbor/harbor/wiki/Harbor-FAQs.
Appreciate!

[SimonAlling]

Please refer to this #16398 (comment) Or refer to FAQ https://github.com/goharbor/harbor/wiki/Harbor-FAQs in API OIDC part

Why has this issue been closed when neither point of concern below has been addressed?

• When the API is accessed by an OIDC user, some endpoints (e.g. /projects/foo) require an OIDC cookie, while others (e.g. /projects) do not.
• In the API Explorer, commands without necessary authentication information are displayed.

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[lindhe]

Please keep this open.

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[cesarb1392]

This issue is still relevant, please keep it open

[j14s]

Still an issue on 2.5.1

[lindhe]

When I ran into this yesterday, it was on a Docker API. Can that be a clue?

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[lindhe]

Ping

[till]

Same error, is this a bug or no bug?

When I log into Harbor via browser and go to API docs, it gives me an option to authorize, and I do this using basic auth. Once authorized, I can run example requests on the Swagger part. When I try something similar with a command line client or with Go, I am stuck with UnAuthorize instead?

[till]

So this seems like a bug:

I was trying to do a /users/current (with curl/go) — that one is broken with my (OIDC) user (used username and CLI secret), but a /projects works fine. The request works (as others have mentioned) on the website (Swagger).

[lindhe]

I think this is clearly a bug.

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[Ketec]

Same issue. After an hour of debugging why requesting artifacts works fine but I cannot get tags of a specific one I end up here.

I have a user in Jenkins credential storage I use to make the curl requests.

This is not for just people to login and play around with postman or something.

This is programmatic use in pipeline. It should have a single credential/api key you can use to access API, stored securely with credentials.

That excess requirement here makes ZERO sense - you can access ALL artifacts with user:password but not individual?

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.

[till]

Please un-stale.

[SimonAlling]

@majusmisiak:

Did you find any workaround for this issue?

IIRC, unfortunately not. I changed jobs a few weeks after creating this issue and haven't used Harbor since.

[naveenb29]

still an issue - v2.7.1-6015b3ef

[Ketec]

YEt again i end up on this very same issue trying to deal with getting actual image version for latest tag from harbor.

All i can do is query ALL artifacts and loop through every time just to find the 'latest' tag.

But it looks like noone in Harbor team cares at all. Zero response.

[ryanmac8]

Still an issue

[lindhe]

Would be nice to try this now after #8723 was purportedly resolved. I don't have a Harbor instance available right now, but if someone could test it that would be nice.

[Forbzy]

This is still broken on harbor 2.10.0

[tculpepp]

Still a problem! Please address

[og-mbenz]

still an issue

[fflambers]

still a problem!

[Ketec]

Another year and again i end up on the same problem.

[reasonerjt]

There are misunderstandings in the discussion on this issue and I can't answer each of them. But I'll try to clarify in general:

When Harbor is configured to authenticate via OIDC provider, it is by design that you should use the ID token to access the API have the same permission as the authenticated user. Therefore, the suggestion in this comment remains valid.

CLI secret was designed to help CLI tools to push and pull artifacts so we allow using CLI secret to access a few API endpoints to fulfill the workflow for artifact push and pull, but not other APIs.

There is a proposal to extend the scope of CLI secret which may solve the problem, but in internal discussion, there are other ideas like introducing a concept "API-token" that is not restricted in OIDC mode. We feel we may not reach a quick decision before "feature freeze" in v2.13.0, so we are moving it out, but I'll follow up with the discussion in the proposal.