Skip to content

Commit bda9665

Browse files
authored
Results with number of parameters + full fine tuning (#2602)
This change updates all results with their respective number of parameters (trained + absolute) and adds the newly introduced full-finetuning. In addition to these results there was also an issue with the Makefile as it didn't consider the possibility of having experiments that don't have an adapter config (e.g., full fine-tuning).
1 parent d67d034 commit bda9665

27 files changed

+4248
-4340
lines changed

method_comparison/MetaMathQA/Makefile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ RESULTS_DIR := results
1010

1111
# 1. Find all experiment directories by looking for adapter_config.json files.
1212
# This gives us a list like: experiments/lora/llama-3.2-3B-rank32 ...
13-
EXPERIMENT_PATHS := $(patsubst %/adapter_config.json,%,$(shell find $(EXPERIMENTS_DIR) -name "adapter_config.json"))
13+
EXPERIMENT_PATHS := $(shell find $(EXPERIMENTS_DIR) \
14+
-name "adapter_config.json" -or \
15+
-name "training_params.json" | xargs dirname | sort -u)
1416

1517
# 2. Define a function to replace all occurrences of a character in a string.
1618
# This is needed to replicate the result naming logic from run.py (e.g., "lora/foo" -> "lora-foo").
@@ -44,7 +46,7 @@ define EXPERIMENT_template
4446
# The dependencies are its config files, code changes need to be audited manually since they can
4547
# vary in degree of importance. Note that we explicitly ignore when the script fails to run
4648
# so that the other experiments still have a chance to run.
47-
$(call exp_to_res,$(1)): $(1)/adapter_config.json $(wildcard $(1)/training_params.json)
49+
$(call exp_to_res,$(1)): $(wildcard $(1)/adapter_config.json) $(wildcard $(1)/training_params.json)
4850
@echo "---"
4951
@echo "Running experiment: $(1)"
5052
-$(PYTHON) $(RUN_SCRIPT) -v $(1)

0 commit comments

Comments
 (0)