Skip to content

Commit d21256c

Browse files
guangy10Guang YangmichaelbenayounGithub Executorch
authored
Export to ExecuTorch: Initial Integration (#2090)
Co-authored-by: Guang Yang <guangyang@fb.com> Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com> Co-authored-by: Github Executorch <github_executorch@arm.com>
1 parent 9840121 commit d21256c

36 files changed

+2090
-4
lines changed

‎.github/workflows/test_executorch_export.yml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: ExecuTorch Export / Python - Test
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
build:
15+
strategy:
16+
fail-fast: false
17+
matrix:
18+
python-version: ['3.10', '3.11', '3.12']
19+
os: [macos-15]
20+
21+
runs-on: ${{ matrix.os }}
22+
steps:
23+
- uses: actions/checkout@v2
24+
- name: Setup Python ${{ matrix.python-version }}
25+
uses: actions/setup-python@v2
26+
with:
27+
python-version: ${{ matrix.python-version }}
28+
- name: Install dependencies for ExecuTorch
29+
run: |
30+
pip install .[tests,exporters-executorch]
31+
pip list
32+
- name: Run tests
33+
working-directory: tests
34+
run: |
35+
RUN_SLOW=1 pytest executorch/export/test_*.py -s -vvvv --durations=0

‎.github/workflows/test_executorch_runtime.yml

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: ExecuTorch Runtime / Python - Test
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
build:
15+
strategy:
16+
fail-fast: false
17+
matrix:
18+
python-version: ['3.10', '3.11', '3.12']
19+
os: [macos-15]
20+
test-modeling:
21+
- test_modeling_gemma2.py
22+
- test_modeling_gemma.py
23+
- test_modeling_llama.py
24+
- test_modeling_olmo.py
25+
- test_modeling.py
26+
- test_modeling_qwen2.py
27+
28+
runs-on: ${{ matrix.os }}
29+
steps:
30+
- uses: actions/checkout@v2
31+
- name: Setup Python ${{ matrix.python-version }}
32+
uses: actions/setup-python@v2
33+
with:
34+
python-version: ${{ matrix.python-version }}
35+
- name: Install dependencies for ExecuTorch
36+
run: |
37+
pip install .[tests,exporters-executorch]
38+
pip list
39+
- name: Run tests
40+
working-directory: tests
41+
run: |
42+
RUN_SLOW=1 pytest executorch/runtime/${{ matrix.test-modeling }} -s -vvvv --durations=0

‎docs/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM nikolaik/python-nodejs:python3.9-nodejs18
1+
FROM nikolaik/python-nodejs:python3.11-nodejs23
22

33
ARG commit_sha
44
ARG clone_url
@@ -8,4 +8,4 @@ RUN python3 -m pip install --no-cache-dir --upgrade pip
88
RUN python3 -m pip install --no-cache-dir git+https://github.com/huggingface/doc-builder.git
99

1010
RUN git clone $clone_url && cd optimum && git checkout $commit_sha
11-
RUN python3 -m pip install --no-cache-dir ./optimum[onnxruntime,benchmark,quality,exporters-tf,doc-build,diffusers]
11+
RUN python3 -m pip install --no-cache-dir ./optimum[onnxruntime,benchmark,quality,exporters-executorch,doc-build,diffusers]

‎docs/source/_toctree.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,23 @@
8181
title: Reference
8282
isExpanded: false
8383
title: "ONNX"
84+
- sections:
85+
- local: exporters/executorch/overview
86+
title: Overview
87+
- sections:
88+
- local: exporters/executorch/usage_guides/export_a_model
89+
title: Export a model to ExecuTorch
90+
- local: exporters/executorch/usage_guides/contribute
91+
title: Add support for exporting an architecture to ExecuTorch
92+
title: How-to guides
93+
- sections:
94+
- local: exporters/executorch/package_reference/configuration
95+
title: ExecuTorch configurations
96+
- local: exporters/executorch/package_reference/export
97+
title: Export functions
98+
title: Reference
99+
isExpanded: false
100+
title: "ExecuTorch"
84101
- sections:
85102
- local: exporters/tflite/overview
86103
title: Overview

‎docs/source/exporters/executorch/overview.mdx

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Overview
14+
15+
🤗 Optimum handles the export of PyTorch to ExecuTorch in the `exporters.executorch` module. It provides classes, functions, and a command line interface to perform the export easily.
16+
17+
Supported architectures from [🤗 Transformers](https://huggingface.co/docs/transformers/index):
18+
19+
- Gemma
20+
- Gemma2
21+
- Llama2
22+
- Llama3(Llama3.2)
23+
- OLMo
24+
- Qwen2(Qwen2.5)
25+
26+
There are many more models are supported by ExecuTorch, we will add those models to Optimum over time. Read more at [pytorch/executorch/examples/](https://github.com/pytorch/executorch/tree/main/examples)

‎docs/source/exporters/executorch/package_reference/configuration.mdx

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Configuration for ExecuTorch Export
14+
15+
ExecuTorch export provides a flexible configuration mechanism through dynamic registration, enabling users to have
16+
complete control over the export process. The configuration system is divided into task configurations and recipe
17+
configurations, each addressing specific aspects of the export pipeline.
18+
19+
20+
## Task Configurations
21+
22+
Task configurations determine how a Hugging Face model should be loaded and prepared for export, tailored to specific tasks.
23+
24+
For instance, when exporting a model for a text generation task, the provided configuration utilizes **static caching** and
25+
**SDPA (Scaled Dot-Product Attention)** for inference optimization.
26+
27+
By leveraging task configurations, users can ensure that their models are appropriately prepared for efficient execution on
28+
the ExecuTorch backend.
29+
30+
[[autodoc]] exporters.executorch.task_registry.discover_tasks
31+
32+
[[autodoc]] exporters.executorch.task_registry.register_task
33+
34+
[[autodoc]] exporters.executorch.tasks.causal_lm.load_causal_lm_model
35+
36+
37+
## Recipe Configurations
38+
39+
Recipe configurations control the specifics of lowering an eager PyTorch module to the ExecuTorch backend. These
40+
configurations allow users to:
41+
42+
- Specify whether and how to **quantize** the model.
43+
- Delegate computation to various accelerators, such as **CPU**, **GPU**, **NPU**, **DSP**, and others.
44+
- Define **custom transformation passes**.
45+
- Implement advanced techniques like memory planning algorithms to optimize resource utilization.
46+
47+
[[autodoc]] exporters.executorch.recipe_registry.discover_recipes
48+
49+
[[autodoc]] exporters.executorch.recipe_registry.register_recipe
50+
51+
[[autodoc]] exporters.executorch.recipes.xnnpack.export_to_executorch_with_xnnpack
52+
53+
The combination of task and recipe configurations ensures that users can customize both the high-level task setup
54+
and the low-level export details to suit their deployment requirements.

‎docs/source/exporters/executorch/package_reference/export.mdx

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Export functions
14+
15+
## Main functions
16+
17+
[[autodoc]] exporters.executorch.convert.export_to_executorch
18+
19+
The primary export function is designed to be **model- and task-independent** as well as **optimization-agnostic**, providing a
20+
highly flexible and modular interface for exporting Hugging Face models to the ExecuTorch backend.
21+
22+
This approach highlights the **composability** of ExecuTorch export pipeline, where dynamically registered **task configurations**
23+
specify how a :hug model is prepared, and **recipe configurations** encapsulate device-specific optimizations during export. This
24+
separation allows users to customize the export process without altering the core function.
25+
26+
For more details on task and recipe configurations, see the [Configuration for ExecuTorch Export](./configuration.mdx).

‎docs/source/exporters/executorch/usage_guides/contribute.mdx

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
-->
12+
13+
# Adding support for an unsupported architecture
14+
15+
We welcome contributions to extend the functionality of ExecuTorch export. This guide provides high-level instructions for contributors who want to:
16+
17+
1. Export a new model that is not currently supported.
18+
2. Add new recipes or support a new task for export.
19+
20+
---
21+
22+
## Exporting a New Model
23+
24+
If you want to export a model that is not already supported by the library, follow these steps:
25+
26+
### Step 1: Export and Test the Model
27+
1. Attempt to export and lower the model using an existing task and recipe. On success, it will store the exported model in a `.pte` file.
28+
2. Add a test case for the model in the appropriate test suite.
29+
- For example, you can make sure tests pass for the new `my_new_model` by running:
30+
```bash
31+
pytest tests/executorch/export/test_*.py -k "test_my_new_model" # doctest: +SKIP
32+
pytest tests/executorch/runtime/test_*.py -k "test_my_new_model" # doctest: +SKIP
33+
```
34+
35+
### Step 2: Handle Export Failures
36+
1. If the export fails in Step 1, report the issue by opening a GitHub issue.
37+
2. If the issue requires changes to the model’s architecture or its Hugging Face implementation, these modifications may be made upstream in the Hugging Face Transformers library.
38+
39+
---
40+
41+
## Adding New Recipes or Tasks
42+
43+
To extend ExecuTorch with new recipes or tasks, follow these guidelines:
44+
45+
### Registering a New Recipe
46+
You can add a custom recipe to define specific optimizations or configurations for exporting models. Below is an example:
47+
48+
```python
49+
from exporters.executorch import register_recipe
50+
51+
@register_recipe("my_custom_recipe")
52+
def export_with_custom_recipe(model, config, *args, **kwargs):
53+
# Example: Apply a custom quantization
54+
```
55+
56+
### Registering a Task
57+
The task registration process is same as adding a recipe. Besides that you may need to implement a new `ExecuTorchModelForXXX` class.

0 commit comments

Comments
 (0)