Skip to content

Commit eb2f7a5

Browse files
committed
Mention installing promptsource from source
1 parent 2ca3287 commit eb2f7a5

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,8 @@ library.
105105

106106
1. Pull the `lm-evaluation-harness` repo by `git submodule update --init
107107
--recursive`. We use the `big-refactor` branch.
108-
2. Install `lm-evaluation-harness`: `pip install -e 3rdparty/lm-evaluation-harness`
108+
2. Install `lm-evaluation-harness`: `pip install -e 3rdparty/lm-evaluation-harness`.
109+
On Python 3.10 you might need to manually install the latest version of `promptsource`: `pip install git+https://github.com/bigscience-workshop/promptsource.git`.
109110
3. Run evaluation with (more documentation at the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/big-refactor) repo):
110111
```
111112
python evals/lm_harness_eval.py --model mamba --model_args pretrained=state-spaces/mamba-130m --tasks lambada_openai,hellaswag,piqa,arc_easy,arc_challenge,winogrande --device cuda --batch_size 64

0 commit comments

Comments
 (0)