Support to bearer tokens

[jesusjuansalgado]

Unfortunately, I think we need to support bearer tokens, as it looks like this is the method used for many data access methods in new software. For example, for the SRCNet we use the token version of Rucio (from CERN) to access the data for all the clients (including command line ones)

I imagine the problem is that a bearer token cannot easily be obtained in command line so the reactive approach is difficult, doesn't it? However, I think it is better to add support and explain the problems if the consensus is that this is not a preferred method. Something like:

Bearer Token Authentication
The ivoa_bearer authentication scheme allows VO services to request authentication via Bearer tokens, aligning with OAuth 2.0 standards. This scheme is compatible with automated clients and supports token expiration and refresh workflows.
ivoa_bearer
Scheme Parameters (in WWW-Authenticate header)
When a protected resource is accessed without valid authentication, the service may respond:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: ivoa_bearer
  access_url="https://example.org/auth/token",
  standard_id="ivo://ivoa.net/sso#oauth2-password",
  refresh_url="https://example.org/auth/refresh"

• access_url (required): URL to obtain a Bearer token using the method described in standard_id
• standard_id (required): Identifies the grant type or login protocol used
• refresh_url (optional): URL to use a refresh_token to get a new access_token
• token_type (optional): Defaults to Bearer; can be omitted

Authentication Response
A successful token acquisition response SHOULD follow OAuth2 conventions:

{
  "access_token": "abc123",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "xyz789"
}

Permit Usage
Once the client has the access token, it must include it in an Authorisation header for protected resources:

Authorization: Bearer abc123

Token Expiry and Refresh
Tokens SHOULD include expires_in so clients can anticipate expiry.

If a refresh_token is returned, the client can use it at the refresh_url:

curl -X POST https://example.org/auth/refresh \
  -d 'grant_type=refresh_token&refresh_token=xyz789'

If a token is used after expiration, the server SHOULD return a 401 or 403 with:

WWW-Authenticate: ivoa_bearer
  error="expired_token",
  access_url="https://example.org/auth/token",
  standard_id="ivo://ivoa.net/sso#oauth2-password",
  refresh_url="https://example.org/auth/refresh"

Optionally, the server MAY also include:

X-VO-Auth-Error: expired_token
This allows clients to distinguish an expired token from an invalid one and take appropriate action.

CLI and Non-Browser Client Support
To avoid reliance on browser-based OAuth2 flows, services that use ivoa_bearer SHOULD support at least one of the following standard_id methods:

• ivo://ivoa.net/sso#oauth2-client-credentials: for client ID + secret auth (non-user)
• ivo://ivoa.net/sso#oauth2-password: user provides username + password
• ivo://ivoa.net/sso#oauth2-device-code: device login with browser-prompted user action

Clients can discover the correct flow from the standard_id in the challenge.

Example CLI Flow (Password Grant)
Receive challenge:

WWW-Authenticate: ivoa_bearer access_url="https://example.org/auth/token",
  standard_id="ivo://ivoa.net/sso#oauth2-password"

Get token:

curl -X POST https://example.org/auth/token \
  -d 'grant_type=password&username=gertrude&password=xxxx'

Use token:
curl -H "Authorization: Bearer abc123" https://example.org/data/securefile.fits
Refresh when expired:

curl -X POST https://example.org/auth/refresh \
  -d 'grant_type=refresh_token&refresh_token=xyz789'

If you agree with the approach, I can update the text accordingly

[mbtaylor]

@jesusjuansalgado you're right that we will certainly need to support bearer tokens in some way.

A major problem that we need to solve for tokens is about usage domain - when the token is issued, it must tell the client for which resources it is valid. For security reasons this information has to be provided by the Authorization Server (the endpoint that issues the token), and not acquired along with the challenge from the Resource Server. As far as I can tell none of the OAuth2-related standards provide a standard way to do this. RFC 6749 sec 3.3 does define a response parameter named "scope", but it means something different. It's not so hard to do (e.g. add another response parameter to the JSON that's a list of Origins?) but we'd need to come up with a concrete proposal.

@aragilar has already proposed a solution for OAuth2 in this context, which is more complicated to use but more flexible than the suggestion above (it uses RFC 8628 Device Authorization Grants), see ivoa-std/SSO#22. That uses a parameter called allowed_domains which seems OK (though we'd need to firm up syntax), I don't know if that's in use elsewhere.

Either James's suggestion or yours, or allowing both, would be OK with me. My suggestion would be to get a working implementation of one or both (or something else) and if it seems to satisfy the requirements then write it up in the document. If somebody comes up with a server-side implementation of some way to work with Bearer Tokens, I'll try to prototype a client-side implementation.

[jesusjuansalgado]

Thanks Mark. I will look at the domain problem. I think a simple protocol implementation could be included at the SRCNet as one of the reference implementations

[jesusjuansalgado]

Is this mixing both proposals?

Bearer Token Authentication
The ivoa_bearer authentication scheme allows VO services to request authentication via Bearer tokens, aligning with OAuth 2.0 standards. This scheme is compatible with automated clients and supports token expiration, refresh workflows, and secure domain scoping via the allowed_origins parameter.

ivoa_bearer
Scheme Parameters (in WWW-Authenticate header):

When a protected resource is accessed without valid authentication, the service may respond:

HTTP/1.1 401 Unauthorized  
WWW-Authenticate: ivoa_bearer  
  access_url="https://example.org/auth/token",  
  standard_id="ivo://ivoa.net/sso#oauth2-password",  
  refresh_url="https://example.org/auth/refresh"

• access_url (required): URL to obtain a Bearer token using the method described in standard_id
• standard_id (required): Identifies the grant type or login protocol used
• refresh_url (optional): URL to use a refresh_token to get a new access_token
• token_type (optional): Defaults to Bearer; can be omitted

Authentication Response
A successful token acquisition response SHOULD follow OAuth2 conventions, and MUST include a domain scope:

{
  "access_token": "abc123",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "xyz789",
  "allowed_origins": ["https://example.org", "https://data.example.org"]
}

• allowed_origins (recommended): A list of HTTP origins (scheme + host [+ port], as defined in RFC 9110 Section 4.3.1) to which the token may be presented.
• Clients MUST NOT use the token for requests to origins not listed here.

Permit Usage
Once the client has the access token, it must include it in an Authorization header for protected resources:
Authorization: Bearer abc123
Before use, the client MUST verify that the protected resource's origin matches one in allowed_origins (*).

Token Expiry and Refresh
Tokens SHOULD include expires_in so clients can anticipate expiry.

If a refresh_token is returned, the client can use it at the refresh_url:

curl -X POST https://example.org/auth/refresh \
  -d 'grant_type=refresh_token&refresh_token=xyz789'

The refresh response SHOULD repeat the allowed_origins field, and clients MUST update their scoping accordingly.

If a token is used after expiration, the server SHOULD return a 401 or 403 with:

WWW-Authenticate: ivoa_bearer  
  error="expired_token",  
  access_url="https://example.org/auth/token",  
  standard_id="ivo://ivoa.net/sso#oauth2-password",  
  refresh_url="https://example.org/auth/refresh"

Optionally, the server MAY also include:

X-VO-Auth-Error: expired_token
This allows clients to distinguish an expired token from an invalid one and take appropriate action.

CLI and Non-Browser Client Support
To avoid reliance on browser-based OAuth2 flows, services that use ivoa_bearer SHOULD support at least one of the following standard_id methods:

• ivo://ivoa.net/sso#oauth2-client-credentials: for client ID + secret auth (non-user)
• ivo://ivoa.net/sso#oauth2-password: user provides username + password
• ivo://ivoa.net/sso#oauth2-device-code: device login with browser-prompted user action

Clients can discover the correct flow from the standard_id in the challenge.

Example CLI Flow (Password Grant)
Receive challenge:

WWW-Authenticate: ivoa_bearer  
  access_url="https://example.org/auth/token",  
  standard_id="ivo://ivoa.net/sso#oauth2-password"

Get token:

curl -X POST https://example.org/auth/token \
  -d 'grant_type=password&username=gertrude&password=xxxx'

Response:

{
  "access_token": "abc123",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "xyz789",
  "allowed_origins": ["https://example.org"]
}

Use token:
curl -H "Authorization: Bearer abc123" https://example.org/data/securefile.fits Refresh when expired:

curl -X POST https://example.org/auth/refresh \
  -d 'grant_type=refresh_token&refresh_token=xyz789'

(*) note: I have used allowed_origins as, at least, there is a definition in RFC 9110. allowed_domains seems to be a close concept to it but I have not found syntax definition so, I think, it is better to use something more standard (but totally fine if I have not found allowed_domains but it is defined somewhere)

[mbtaylor]

@jesusjuansalgado this looks quite reasonable to me, though I have a couple of comments:

• would it make more sense to have the refresh_url delivered in the auth response rather than the challenge?
• I'd say allowed_origins in the original authentication response has to be MANDATORY not RECOMMENDED, since otherwise there is no way to use the returned token securely.
• The details of the suggested standard_id options oauth2-client-credentials and oauth2-device-code have to be spelled out (at least, a pointer to the relevant RFC or whatever). If they have the form "ivo://ivoa.net/sso#..." it means they should be defined in the ivo://ivoa.net/sso registry record which I suppose is outside of this standard and would mean revising SSO. And I've just noticed another problem with that... I've opened an issue standard_id value namespace #7 for discussing this.

But I don't have an informed opinion on how this compares to @aragilar's suggestion in terms of satisfying the requirements of OAuth2 service implementors, I'll leave that to others to discuss.

[jesusjuansalgado]

• would it make more sense to have the refresh_url delivered in the auth response rather than the challenge?
  I see the refresh_url as a description of the service to be invoked (similar to the access_url) but I think we can put it whenever makes more sense
• I'd say allowed_origins in the original authentication response has to be MANDATORY not RECOMMENDED, since otherwise there is no way to use the returned token securely.
  Yes, fine
• The details of the suggested standard_id options oauth2-client-credentials and oauth2-device-code ...
  fully true. I think we need to define at least #oauth2-client-credentials at least (maybe others from RFC6749 as the password one is quite deprecated) but #oauth2-device-code could be a better candidate for human logging without browser

[andamian]

My impression is that the OIDC and Auth2.0 define the refresh flow themselves. Mentioning it in this document only duplicates that info.
For example, I use idc-* commands to manage SKA IAM tokens:

oidc-gen --iss=https://example.org/auth/token --scope max --flow=device example-client

This, if I'm not mistaken returns an id and a refresh token (and I think an access token as well). Then all I have to do is to repeatedly call SKA_TOKEN=$(oidc-token example-client) to get valid access tokens (until the refresh token expires). I can probably implement that in my Python client itself but I would need to safe guard the refresh token. The bottom line is that all that idc-gen required was the iss and it figured out the rest (refresh tokens endpoints and the like).

[andamian]

My issue, (and I don't know if I open a new one) is that the document doesn't show how to advertise supported external IdP. At that CADC we have our login endpoint but we are planning on supporting bearer tokens from external trusted IdPs (or Proxy IdPs) such as SKA IAM and Rubin CILogon. Should the service list multiple www-authenticate in this case or just one that acts as a proxy for all the others? First option seems prone to phishing attacks while the second requires users to login to a local proxy login service which is against the SSO principles. This is a big stumbling block that we are struggling with.

[mbtaylor]

The service can present multiple challenges, but it doesn't need to have a challenge for each source of bearer tokens. In the case of multiple challenges the client can choose any one it understands. So you could just offer a standard:

   www-authenticate: Bearer

challenge to support use of tokens in general, but a

   www-authenticate: ivoa_bearer access_url="https://example.org/login", ...

challenge that points clients to your login service if they don't already know where to get a token. Does that help? Or are you wondering how you can get clients to offer users the choice of multiple different login endpoints?

[andamian]

I'm wondering how can a client (tool or service) determine 1. what kind of permits work securely with the service and 2. where to get them if they don't have them handy. I suspect that in the case of OIDC tokens, the access_url of the www-authenticate is the same with OIDC token issuer (iss claim) so with multiple options, the client might be able to produce an appropriate token if it happens to already have one. While this works, it doesn't address the problem of trust - any rogue service can claim that they accept SKA tokens to trick the user into sending them and then use them for other purposes.

[jesusjuansalgado]

My impression is that the OIDC and Auth2.0 define the refresh flow themselves. Mentioning it in this document only duplicates that info. For example, I use idc-* commands to manage SKA IAM tokens:

oidc-gen --iss=https://example.org/auth/token --scope max --flow=device example-client

This, if I'm not mistaken returns an id and a refresh token (and I think an access token as well). Then all I have to do is to repeatedly call SKA_TOKEN=$(oidc-token example-client) to get valid access tokens (until the refresh token expires). I can probably implement that in my Python client itself but I would need to safe guard the refresh token. The bottom line is that all that idc-gen required was the iss and it figured out the rest (refresh tokens endpoints and the like).

It is true that some of the workflow is implicit on the OIDC/Auth2.0. The reason to express all this metadata is to mimic the description of other authentication methods in the document. There are things like the authentication method (password, device_code, client_credentials..), access_urls, etc simplify the implementation of general purpose clients (so the compatibility is added for non-aware command line clients). I think with this and with a proper update at SSO (as Mark is flagging) the use of tokens for general astronomical command line clients will not be intimidating

[jesusjuansalgado]

I'm wondering how can a client (tool or service) determine 1. what kind of permits work securely with the service and 2. where to get them if they don't have them handy. I suspect that in the case of OIDC tokens, the access_url of the www-authenticate is the same with OIDC token issuer (iss claim) so with multiple options, the client might be able to produce an appropriate token if it happens to already have one. While this works, it doesn't address the problem of trust - any rogue service can claim that they accept SKA tokens to trick the user into sending them and then use them for other purposes.

This is a fair comment. We need to ensure that clients check the trusted IdPs so tokens are sent not because the server just says that they accept them but because the are endorse to receive them. We could add something like:

Clients MUST NOT use authentication challenges alone to determine whether it is safe to send a token or credential. Token use MUST be constrained to trusted issuers and bound domains, and MUST be validated according to client-side policies (e.g., iss, aud, or configured trust anchors)

but we need to investigate what that means and how this could be obtained so we can add an explanation too.

[aragilar]

The scheme in #6 (comment) is not valid OAuth 2.0 (nor by reference OIDC), as it is missing the client ID and (sometimes optional) client secret.

The two problems my ivoa_oauth design (i.e. http://mail.ivoa.net/pipermail/dsp/2024-October/001972.html) aimed to solve was getting the client ID and client secret (by using RFC 7591) and by having an initial endpoint which listed the origins that the token could be used against (along with a pointer to the RFC 8414 endpoint which standard off-the-shelf OAuth 2.0 clients can be given alongside the client ID and secret to self-configure).

It also mandated the RFC 8628 (Device Authorization Grant) because the password grant is at best deprecated (it has been removed from OAuth 2.1), and its use will be flagged as a security issue.

I don't see the value in writing our own protocol from scratch, as then no-one can use their existing OAuth 2.0/OIDC infrastructure.

[aragilar]

@andamian What I suggest is running a proxy service which forwards on the authentication request (I think most decently featured IdPs can do this, I know Apereo CAS (which Data Central runs) can do this, and I'm 90% sure Keycloak can), and so you would issue out client IDs/secrets from your proxy (for VO clients like TOPCAT), and you've receive separate CILogon client ID/secret to do the proxying (or if they want to run the VO discovery service from ivoa_oauth, then you can give them your origins and point to their discovery service).

[mbtaylor]

I'm wondering how can a client (tool or service) determine 1. what kind of permits work securely with the service and 2. where to get them if they don't have them handy.

@andamian the whole point of the document is to answer that question: the challenges are constructed so that the client can find out from them what permits work and where to get one. Some clients, e.g. web applications that have up-front knowledge about the service they are talking to, may not need this.

I suspect that in the case of OIDC tokens, the access_url of the www-authenticate is the same with OIDC token issuer (iss claim) so with multiple options, the client might be able to produce an appropriate token if it happens to already have one. While this works, it doesn't address the problem of trust - any rogue service can claim that they accept SKA tokens to trick the user into sending them and then use them for other purposes.

The specification must be written so that the domain of the token is issued with the token itself, and not implicitly/explicitly claimed by the challenge. So even if a rogue service challenge says "I want tokens from an authentication server X", the domain of the tokens obtained from X will not include the rogue endpoint, so the client will never reveal such tokens to it. That is the point of @aragilar's allowed_domains and @jesusjuansalgado's allowed_origins.

[jesusjuansalgado]

The scheme in #6 (comment) is not valid OAuth 2.0 (nor by reference OIDC), as it is missing the client ID and (sometimes optional) client secret.

The two problems my ivoa_oauth design (i.e. http://mail.ivoa.net/pipermail/dsp/2024-October/001972.html) aimed to solve was getting the client ID and client secret (by using RFC 7591) and by having an initial endpoint which listed the origins that the token could be used against (along with a pointer to the RFC 8414 endpoint which standard off-the-shelf OAuth 2.0 clients can be given alongside the client ID and secret to self-configure).

It also mandated the RFC 8628 (Device Authorization Grant) because the password grant is at best deprecated (it has been removed from OAuth 2.1), and its use will be flagged as a security issue.

I don't see the value in writing our own protocol from scratch, as then no-one can use their existing OAuth 2.0/OIDC infrastructure.

I think this is not what we are trying to do. The ivoa_bearer scheme does not aim to reimplement OAuth 2.0, nor to define a new authentication protocol. Instead, its purpose is to allow a VO service to advertise, via a WWW-Authenticate challenge, that it expects authentication via a Bearer token, and to provide sufficient metadata (like access_url, standard_id, refresh_url, etc.) to guide a client in obtaining and using such a token.
This is why it is very simplified because it just provide metadata to the clients needed to start the communication, not defining the communication itself. A client should load a specialised library to handle the oAuth communication