Skip to content

Commit 8082d60

Browse files
committed
docs: Add example scripts for serving Eagle models
- Add Eagle1 example scripts for Llama models - Add Eagle3 example scripts for Llama and Qwen models - Include both regular and float16 serving examples
1 parent cb0f182 commit 8082d60

File tree

4 files changed

+5
-0
lines changed

4 files changed

+5
-0
lines changed
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
VLLM_USE_V1=1 vllm serve nm-testing/eagle-llama3.1-8b-instruct-converted-0717 --enforce-eager --port 5000 >output.txt
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
VLLM_USE_V1=1 vllm serve nm-testing/eagle-llama3.1-8b-instruct-converted-FC-0717 --enforce-eager --port 5000 >output.txt
2+
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
VLLM_USE_V1=1 vllm serve nm-testing/SpeculatorLlama3-1-8B-Eagle3-converted-0717 >output_speculators_llama.txt
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
VLLM_USE_V1=1 vllm serve nm-testing/Speculator-Qwen3-8B-Eagle3-converted-0717 >output_speculators_qwen.txt

0 commit comments

Comments
 (0)