config.json access on sites that require auth but return 400 without auth

[rbtcollins]

Problem

This is a usability papercut on the new registry authentication feature in nightly.

The merged code depends on a GET -> 401 -> GET-with-auth -> 200 sequence for config.json.

That doesn't work (today) for JFrog Artifactory cargo registries: they return 400 for the first request.

Obviously this is a thing that they'll fix eventually, but perhaps an option to allow prior-knowledge of auth tokens being required would be helpful, since I doubt they'll be the only site to do this weirdly.

Steps

No response

Possible Solution(s)

No response

Notes

No response

Version

No response

[Eh2406]

I am reluctant to add this, as it provides an easy way for a user to accidentally send tokens to a url/registry that knows nothing about this RFC. That being said, if this over time turns out to be a real user need we can design and add it.

[rbtcollins]

Fair enough. I am a bit confused about that easy way thing - I mean, any HTTP server that provides an auth challenge can harvest the token IF our configuration language permits tokens to be send to their URL, so I'm not sure how prior knowledge permission adds to that risk.

[Eh2406]

I think a website has to be actively malicious to return 401 while keeping track of Auth headers. Whereas someone only needs to be careless to throw together a read-only registry (or for that matter any other website someone accidentally puts in their config.toml) that logs all inputs.

[sassman]

I know the RFC 3139 does not really specify what I'm asking now for, but IMO it is very important to have.

As a user I would like to control the behavior of cargo, so that I can force-overwrite the flag "auth-required": true that comes right now from the registry index (see config.json).

The background for this is, that some of the commercial vendors, like JFrog / Artifactory support registries that are protected by an Auth-token, but they lack the flexibility to change the config.json in a way that this new feature can be turned on for cargo.

Additionally as a user I might want to be in control of wether my Auth-token is not send over to a specific registry or not.

In order to empower the user to force overwrite the very flag that is dictated by the registry it would require a to extend the config file ($CARGO_HOME/.cargo/config) in such / similar ways:

# $CARGO_HOME/.cargo/config

[registries.my-corporate-registry]
index = "https://my-corporate.jfrog.io/artifactory/git/my-corporate-registry.git"
# allowing the user to have explicit control
auth-required = true

It was already mentioned that other package managers in other languages offer such control to users by changing local configuration.

[sassman]

@Eh2406 I would even contribute if that is going to be accepted, just let me know.

[Eh2406]

So if I'm reading the discussion correctly there are two arguments for a auth-required in .cargo/config.toml. "Registries do not follow the RFC" and "users want more control".

As to 1. I don't yet feel justified in adding functionality to cargo to work around bugs in registries. I might be talked into a separate permanently unstable flag, if it really is the only way to get testing on the feature. It certainly true that if we require asymmetric tokens we cannot accommodate all the ways servers could implement it wrong.

As to 2. I don't fully understand the design you have intended. Can you specify the full 3x3 matrix of configurations and what should happen in each one?

Registry	Config	Behavior Read	Behavior Mutation
Yes	Yes	Send token	Send token
Unset	Yes	Send token	Send token
No	Yes	?	?
Yes	Unset	Send token?	Send token
Unset	Unset	Don't send token	Send token
No	Unset	Don't send token	Error?
Yes	No	Send token?	Send token
Unset	No	Don't send token	Send token
No	No	Don't send token	Error?

Also can you give some information on what use cases the user gets to opt into by having this new flag? What practical "more control" does this provide?

[sassman]

As to 1. I don't yet feel justified in adding functionality to cargo to work around bugs in registries. I might be talked into a separate permanently unstable flag, if it really is the only way to get testing on the feature. It certainly true that if we require asymmetric tokens we cannot accommodate all the ways servers could implement it wrong

The suggestion with the separate unstable flag, to send an auth header always sounds fair enough to me.

As to 2. I don't fully understand the design you have intended. Can you specify the full 3x3 matrix of configurations and what should happen in each one?

What I had in mind was like this:

Registry	Config	Behavior Read	Behavior Mutation
Yes	Yes	Send token	Send token
Unset	Yes	Send token	Send token
No	Yes	Send token	Send token
Yes	Unset	Send token	Send token
Unset	Unset	Don't send token	Send token
No	Unset	Don't send token	Send token (same as of today)
Yes	No	Don't send token	Error with hint that the config says no
Unset	No	Don't send token	Error with hint that the config says no
No	No	Don't send token	Error with hint that the config says no

But to be honest, the more I'm thinking about it I see the potential confusion that it could cause.

So maybe your suggestion with a separate unstable flag would be a less confusing thing then.

[rbtcollins]

FWIW I've worked around this by implementing a small HTTPS proxy to transform the Jfrog responses while JFrog work on the bug. I note that https://orth.uk/cargo-registry-authentication/ is another way to workaround this, just using different tooling.

Also going to ping @AsafZalcman-jfrog for visibility.

[nadav-yo]

That doesn't work (today) for JFrog Artifactory cargo registries: they return 400 for the first request.

Which endpoint returns 400?
I'm familiar with an issue we already addressed of auth-required=true always being sent on sparse-index enabled repos, even if anonymous access was enabled. this can configurable by the repo anonymous flag.

(Nadav from JFrog)