Refresh docs, remove references to old conversion scripts

Author: Laurent

docs/getting-started/advanced.md

@@ -10,4 +10,6 @@ We use Rye to maintain and release Refiners but it conforms to the standard Pyth
 
 ## Using stable releases from PyPI
 
-Although we recommend using our development branch, we do [publish more stable releases to PyPI](https://pypi.org/project/refiners/) and you are welcome to use them in your project. However, note that the format of weights can be different from the current state of the development branch, so you will need the conversion scripts from the corresponding tag in GitHub, for instance [here for v0.2.0](https://github.com/finegrain-ai/refiners/tree/v0.2.0).
+Although we recommend using our development branch, we do publish more stable releases to [PyPI](https://pypi.org/project/refiners/) and you are welcome to use them in your project.
+They are also available directly on the [GitHub releases page](https://github.com/finegrain-ai/refiners/releases).
+However, beware that the format of weights can be different from the current state of the development branch.

docs/getting-started/recommended.md

@@ -6,55 +6,100 @@ icon: material/star-outline
 
 Refiners is still a young project and development is active, so to use the latest and greatest version of the framework we recommend you use the `main` branch from our development repository.
 
-Moreover, we recommend using [Rye](https://rye-up.com) which simplifies several things related to Python package management, so start by following the instructions to install it on your system.
+Moreover, we recommend using [Rye](https://rye.astral.sh/) which simplifies several things related to Python package management, so start by following the instructions to install it on your system.
 
 ## Installing
 
 To try Refiners, clone the GitHub repository and install it with all optional features:
 
 ```bash
-git clone "git@github.com:finegrain-ai/refiners.git"
+git clone git@github.com:finegrain-ai/refiners.git
 cd refiners
 rye sync --all-features
 ```
 
 ## Converting weights
 
-The format of state dicts used by Refiners is custom and we do not redistribute model weights, but we provide conversion tools and working scripts for popular models. For instance, let us convert the autoencoder from Stable Diffusion 1.5:
+The format of state dicts used by Refiners is custom, so to use pretrained models you will need to convert weights.
+We provide conversion tools and pre-converted weights on our [HuggingFace organization](https://huggingface.co/refiners) for popular models.
 
-```bash
-python "scripts/conversion/convert_diffusers_autoencoder_kl.py" --to "lda.safetensors"
+For instance, to use the autoencoder from Stable Diffusion 1.5:
+
+### Use pre-converted weights
+
+```py
+from huggingface_hub import hf_hub_download
+from refiners.foundationals.latent_diffusion.stable_diffusion_1.model import SD1Autoencoder
+
+# download the pre-converted weights from the hub
+safetensors_path = hf_hub_download(
+    repo_id="refiners/sd15.autoencoder",
+    filename="model.safetensors",
+    revision="9ce6af42e21fce64d74b1cab57a65aea82fd40ea",  # optional
+)
+
+# initialize the model
+model = SD1Autoencoder()
+
+# load the pre-converted weights
+model.load_from_safetensors(safetensors_path)
+```
+
+### Convert the weights yourself
+
+If you want to convert the weights yourself, you can use the conversion tools we provide.
+
+```py
+from refiners.conversion import autoencoder_sd15
+
+# This function will:
+#   - download the original weights from the internet, and save them to disk at a known location
+#     (e.g. tests/weights/stable-diffusion-v1-5/stable-diffusion-v1-5/vae/diffusion_pytorch_model.safetensors)
+#   - convert them to the refiners format, and save them to disk at a known location
+#     (e.g. tests/weights/refiners/sd15.autoencoder/model.safetensors)
+autoencoder_sd15.runwayml.convert()
+
+# get the path to the converted weights
+safetensors_path = autoencoder_sd15.runwayml.converted.local_path
+
+# initialize the model
+model = SD1Autoencoder()
+
+# load the converted weights
+model.load_from_safetensors(safetensors_path)
 ```
 
-If you need to convert weights for all models, check out `script/prepare_test_weights.py`.
+!!! note
+    If you need to convert more model weights or all of them, check out the `refiners.conversion` module.
+
+!!! warning
+    Converting all the weights requires a lot of disk space and CPU time, so be prepared.
+    Currently downloading all the original weights takes around ~100GB of disk space,
+    and converting them all takes around ~70GB of disk space.
 
 !!! warning
-    Using `script/prepare_test_weights.py` requires a GPU with significant VRAM and a lot of disk space.
+    Some conversion scripts may also require quite a bit of RAM, since they load the entire weights in memory,
+    ~16GB of RAM should be enough for most models, but some models may require more.
 
-Now to check that it works copy your favorite 512x512 picture in the current directory as `input.png` and create `ldatest.py` with this content:
+
+### Testing the conversion
+
+To quickly check that the weights you got from the hub or converted yourself are correct, you can run the following snippet:
 
 ```py
 from PIL import Image
 from refiners.fluxion.utils import no_grad
-from refiners.foundationals.latent_diffusion.stable_diffusion_1.model import SD1Autoencoder
-
-with no_grad():
-    lda = SD1Autoencoder()
-    lda.load_from_safetensors("lda.safetensors")
 
-    image = Image.open("input.png")
-    latents = lda.image_to_latents(image)
-    decoded = lda.latents_to_image(latents)
-    decoded.save("output.png")
-```
+image = Image.open("input.png")
 
-Run it:
+with no_grad():
+    latents = model.image_to_latents(image)
+    decoded = model.latents_to_image(latents)
 
-```bash
-python ldatest.py
+decoded.save("output.png")
 ```
 
-Inspect `output.png`: it should be similar to `input.png` but have a few differences. Latent Autoencoders are good compressors!
+Inspect `output.png`, if the converted weights are correct, it should be similar to `input.png` (but have a few differences).
 
 ## Using Refiners in your own project
 
@@ -63,20 +108,28 @@ So far you used Refiners as a standalone package, but if you want to create your
 ```bash
 rye init --py "3.11" myproject
 cd myproject
-rye add --git "git@github.com:finegrain-ai/refiners.git" --features training refiners
+rye add refiners@git+https://github.com/finegrain-ai/refiners
 rye sync
 ```
 
-If you only intend to do inference and no training, you can drop `--features training`.
+If you intend to use Refiners for training, you can install the `training` feature:
 
-To convert weights, you can either use a copy of the `refiners` repository as described above or add the `conversion` feature as a development dependency:
+```bash
+rye add refiners[training]@git+https://github.com/finegrain-ai/refiners
+```
+
+Similarly, if you need to use the conversion tools we provide, you install the `conversion` feature:
 
 ```bash
-rye add --dev --git "git@github.com:finegrain-ai/refiners.git" --features conversion refiners
+rye add refiners[conversion]@git+https://github.com/finegrain-ai/refiners
 ```
 
 !!! note
-    You will still need to download the conversion scripts independently if you go that route.
+    You can install multiple features at once by separating them with a comma:
+
+    ```bash
+    rye add refiners[training,conversion]@git+https://github.com/finegrain-ai/refiners
+    ```
 
 ## What's next?