We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent db96d97 commit 109e978Copy full SHA for 109e978
vllm/config/load.py
@@ -40,6 +40,8 @@ class LoadConfig:
40
more information.\n
41
- "runai_streamer" will load the Safetensors weights using Run:ai Model
42
Streamer.\n
43
+ - "runai_streamer_sharded" will load weights from pre-sharded checkpoint
44
+ files using Run:ai Model Streamer.\n
45
- "bitsandbytes" will load the weights using bitsandbytes quantization.\n
46
- "sharded_state" will load weights from pre-sharded checkpoint files,
47
supporting efficient loading of tensor-parallel models.\n
0 commit comments