Add domains on the fly

[Wicpar]

Hi,
I am implemententing an auto-whitelabel fature on my service based on the host. For this i use wildcard dns resolution, and allow clients to register custom domains on the same service that will change based on host, at least one subdomain is provided.
Is this crate capable of adding new domains on the fly ?

[FlorianUekermann]

Adding domains on the fly isn't supported yet, but could probably be added. Could you describe your usecase a little more?

If I understood your use case correctly you would like to add (and presumably remove) additional domains, which may not share the same TLD, on the fly. Not sure I understood the parts about wildcard dns and the subdomain, so I'm not sure if that is relevant here.

I think there is one key consideration. Do you want one or a few certs for all domains (iirc Let's Encrypt allows about a hundred), or many certs for many sets of domains. Different rate limits may apply to each case. I suspect the latter is more robust and scalable, so I'll focus on that in the following.

I can think of three ways to go about this:

1. Allow the low level API to construct AcmeAccept from futures_rustls::StartHandshake instead of the TCP stream. This would allow you to inspect the host before handing the connection over to the respective AcmeAcceptor instance for the domain.
2. Generalize all components to handle multiple independently maintained certs with different domain sets (this affects mostly resolver and state structs).
3. Do (1) and add a new high level type specifically to provide a high-level API to do manage the state collection, inspection and matching.

(1) seems like a low-hanging fruit and generally useful. (3) is a nice composition of existing features (after (1) is implemented).
However (2) is quite appealing in terms of resource sharing and while composing atomic features is nice, it may be too easy to generalize here to pass that up for introducing something more complex like (3), which may be something better done by the users of this crate if desired.

Do you think (some/all of) these options would match your requirements?

Also, this sound like you may be working on a commercial service. If that is the case, feel free to contact me at florian@code.green if professional support or accelerated development could be valuable to you.

[Wicpar]

Thank you for the description, i'm not too familiar with rustls and how certificates work but i indeed envisioned the latter where i store lots of certs in the database and choose one as needed (with a local cache of course).
I'll try the different options, but my ideal high level api would be to just have an arc dashmap that maps the domains to certificates and you can just add a new certificate manually, or request a new one be generated for a domain (that does a dns check if it has the right IP before challenging acme). I suppose that would imply solution 2.

[FlorianUekermann]

To prevent misunderstandings: You can't do this at the moment with rustls-acme (well, you might, but it's not obvious how). I'm trying to understand your use case better to come up with a solution that makes sense. The 3 options above were ideas for implementations.

I'm a bit confused what exactly you want to accomplish, so here are a few questions/notes:

1. Your last comment, sounds like you may want to use a mix of certs from other sources and let's encrypt. If that is the case, that would be good to know.
2. You mention a certificate container (dashmap). I think that may be based on a misunderstanding of how rustls-acme works. rustls-acme is not primarily a library to execute acme orders and get a certificate in return to use as you please. You can do that with the stuff in the rustls_acme::acme module. I'll happily support changes that make the stuff in that module useful to more people, but high-level helpers for manual order workflows would have to live in another crate.
   Instead rustls-acme focuses on a convenient implementation of the tls-alpn-01 validation, which implies, that it has access to all incoming TLS handshakes. The benefit is that you can set up a TLS listener with Let's Encrypt support in about 5 lines of code (see high-level example in the docs), with certificate acquisition and renewal taken care of under the hood. This means:
   a. If you need more control than adding/removing which domains you need certs for, this is not the crate to use. DNS checks are of course something you may do on your own before adding/removing domains (which isn't implemented yet).
   b. If you want to keep rustls-acme separate from your regular TLS handshaking code, that's not an option either, because that isn't compatible with how the tls-alpn-01 validation procedure works.
   c. If you want manual control when/how certificate acquisition/renewal is done, this isn't the right crate either, because taking that pain away is the point of the crate. Use another crate or build your own solution using the rustls_acme::acme module.
3. Persistent caching is done by user provided Cache trait implementations, so that is likely already solved in a way that works for you. It only assumes you can store some bytes keyed by domain and directory string in some key-value-store like a DB or a folder. Does that work for your use case?
4. Do you want one certificate with all domains or multiple certificates with one or a few domains each?

If you don't want to deal with certificate and rustls details, the high-level API is probably what you want. Please have a look at it and let me know if that sufficient for you if a way to add/remove domains is added.

[Wicpar]

1. If possible yes, some clients may want to provide their own certificates for https, bit it's clearly a nice to have and not absolutely necessary. i suppose if you inject a cert from the cache it should work ?
2. i'm not familiar with tls-alpn-01 so i might be mistaken on how this works. the dashmap would simply be the cache that allows adding certificates on the fly, either by the challenge system or user provided (like a DB to store them long term)
3. that solves 2 for me :) i just need to know that it checks the cache every time
4. anything that works, i don't know all the nuances it entails.

I know it's not yet possible, but i was thinking of forking it to make it work, sounds like it should be feasible.
From what you are saying a method to add/remove a domain on the fly should be all i need.

[FlorianUekermann]

1. If possible yes, some clients may want to provide their own certificates for https, bit it's clearly a nice to have and not absolutely necessary. i suppose if you inject a cert from the cache it should work ?

It's ugly, but should work. Only if you inject before starting the state management though. I'm considering implementing a pinning compatible mechanism where you can specify a cert instead of a new one being generated. That would have the side effect of covering this usecase.

2. i'm not familiar with tls-alpn-01 so i might be mistaken on how this works. the dashmap would simply be the cache that allows adding certificates on the fly, either by the challenge system or user provided (like a DB to store them long term)

I don't see any overlap between how this crate works and the dashmap idea. I think there may still be some fundamental misconceptions about how this crate is used. I would recommend that you start using the crate with a fixed set of domains to figure out how the existing interface could be extended to add more domains in a convenient way.

3. that solves 2 for me :) i just need to know that it checks the cache every time

Currently it does when a certifcate is missing (so currently at startup), but not for renewals.

4. anything that works, i don't know all the nuances it entails.

I know it's not yet possible, but i was thinking of forking it to make it work, sounds like it should be feasible. From what you are saying a method to add/remove a domain on the fly should be all i need.

Not quite. You would have to touch multiple places. Most importantly the underlying state machine has to be generalized to mutliple sets of certificates if you want to be able to scale beyond 100 domains.

I suspect that this will become possible quite soon by using the lower level API and simply running multiple instances of rustls_acme::AcmeState as issue #46 is resolved. Then you would simply receive the "client hello" yourself, and fetch the certificate from the respective resolver yourself.

[Wicpar]

nice :)

[Wicpar]

I started reworking the crate pretty fundamentally based around a new ResolvesServerCert implementation.
I already added multi certificate support and x509 based domain verification (so there is no need to save anything but the certificates)
I also did a few refactors, do you want me to keep them in my PR or should i just keep to the new implementation ?

[Wicpar]

with the latest push it is almost ready, StreamlinedResolver::new_with_updater can be given directly as the certificate provider for rustls, and existing pems can be added to it. The updater can be driven from anywhere.

it just needs the caching and the acceptors.

[flamestro]

Hey, a small fyi for everyone reading this in the future. This example: https://github.com/FlorianUekermann/rustls-acme/blob/main/examples/low_level_tokio.rs actually handles this usecase imo. It is (as indicated) more low level, but resolves most of the ownership issues that come with the high level solution in terms of adding domains / certs on the fly without reloading.