Skip to content

Commit 962db1b

Browse files
Update documents for buffer allocator (#11467)
### Summary 1. Update MediaTek backend documents for the decoupled buffer allocator. 2. Follow backend template. 3. Remove unnecessary instructions. Fixes #8532 @pytorchbot label "partner: mediatek"
1 parent 20ea230 commit 962db1b

File tree

2 files changed

+64
-90
lines changed

2 files changed

+64
-90
lines changed

backends/mediatek/README.md

Lines changed: 15 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -14,23 +14,11 @@ The examples provided in this repository are tested and supported on the followi
1414

1515
Before you begin, ensure you have the following prerequisites installed and configured:
1616

17-
#### 1. Buck2 Build Tool
18-
19-
- **Download Buck2**: Obtain Buck2 from the official [releases page](https://github.com/facebook/buck2/releases/tag/2024-02-01).
20-
- **Add to PATH**: Extract the downloaded file and add the directory to your system's `$PATH` environment variable.
21-
```bash
22-
export PATH=<path_to_buck>:$PATH
23-
```
24-
25-
#### 2. Android NDK
17+
#### 1. Android NDK
2618

2719
- **Download Android NDK**: Acquire the Android NDK version 26.3.11579264 from the [Android developer site](https://developer.android.com/ndk/downloads).
28-
- **Set NDK Path**: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located.
29-
```bash
30-
export ANDROID_NDK=<path_to_android_ndk>
31-
```
3220

33-
#### 3. MediaTek ExecuTorch Libraries
21+
#### 2. MediaTek ExecuTorch Libraries
3422

3523
To get started with MediaTek's ExecuTorch libraries, download the [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) from MediaTek's NeuroPilot portal. The SDK includes the following components:
3624

@@ -60,26 +48,28 @@ Follow the steps below to setup your build environment:
6048
pip3 install mtk_neuron-8.2.19-py3-none-linux_x86_64.whl
6149
pip3 install mtk_converter-8.13.0+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
6250
```
63-
- Set evironment variables for building backend
64-
```bash
65-
export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator>
66-
```
6751

6852
### Build
69-
1. Navigate to `scripts/` directory.
70-
71-
2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process, MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`
53+
1. Copy `NeuronAdapter.h` to `backends/mediatek/runtime/include/api/`
7254

55+
2. Set NDK Path: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located.
7356
```bash
74-
./mtk_build.sh
57+
export ANDROID_NDK=<path_to_android_ndk>
7558
```
7659

77-
### Run
60+
3. Build the backend library `libneuron_backend.so`:
61+
```bash
62+
cd backends/mediatek/scripts/
63+
./mtk_build.sh
64+
```
65+
The output is `libneuron_backend.so` in `cmake-android-out/backends/mediatek/`.
7866

79-
1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend.
67+
### Run
8068

69+
1. Push `libneuron_backend.so`, `libneuronusdk_adapter.mtk.so` and `libneuron_buffer_allocator.so` to the device.
70+
2. Set the library path before running ExecuTorch:
8171
```bash
82-
export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH
72+
export LD_LIBRARY_PATH=<path_to_neuron_backend>:<path_to_usdk>:<path_to_buffer_allocator>:$LD_LIBRARY_PATH
8373
```
8474

8575
Please refer to `executorch/examples/mediatek/` for export and execution examples of various of models.

docs/source/backends-mediatek.md

Lines changed: 49 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,95 +1,79 @@
11
# MediaTek Backend
22

3-
MediaTek backend empowers ExecuTorch to speed up PyTorch models on edge devices that equips with MediaTek Neuron Processing Unit (NPU). This document offers a step-by-step guide to set up the build environment for the MediaTek ExecuTorch libraries.
4-
5-
::::{grid} 2
6-
:::{grid-item-card} What you will learn in this tutorial:
7-
:class-card: card-prerequisites
8-
* How to export and lower a PyTorch model ahead of time with ExecuTorch for MediaTek devices.
9-
* How to build MediaTek backend and examples.
10-
* How to deploy the exported models on device with ExecuTorch runtime.
11-
:::
12-
:::{grid-item-card} Tutorials we recommend you complete before this:
13-
:class-card: card-prerequisites
14-
* [Introduction to ExecuTorch](intro-how-it-works.md)
15-
* [Getting Started](getting-started.md)
16-
* [Building ExecuTorch with CMake](using-executorch-building-from-source.md)
17-
:::
18-
::::
19-
20-
21-
## Prerequisites (Hardware and Software)
22-
23-
### Host OS
24-
- Linux operating system
25-
26-
### Supported Chips:
27-
- MediaTek Dimensity 9300 (D9300)
28-
- MediaTek Dimensity 9400 (D9400)
3+
The MediaTek backend enables acceleration of PyTorch models on edge devices with MediaTek Neuron Processing Units (NPUs). This backend provides tools for exporting, building, and deploying models to leverage MediaTek hardware.
294

30-
### Software:
5+
## Features
316

32-
- [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress) is a lightweight SDK for deploying AI applications on MediaTek SOC devices.
7+
- Acceleration of PyTorch models on MediaTek NPUs
8+
- Tools for model export and lowering
9+
- Example scripts for model deployment and execution
3310

34-
## Setting up your developer environment
11+
## Target Requirements
3512

36-
Follow the steps below to setup your build environment:
13+
- **Hardware:** MediaTek Dimensity 9300 (D9300), Dimensity 9400 (D9400)
14+
- **Host OS:** Linux
15+
- **SDK:** [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress)
3716

38-
1. **Setup ExecuTorch Environment**: Refer to the [Getting Started](getting-started.md) guide for detailed instructions on setting up the ExecuTorch environment.
17+
## Development Requirements
3918

40-
2. **Setup MediaTek Backend Environment**
41-
```bash
42-
pip3 install -r requirements.txt
43-
```
44-
- Install the two .whl downloaded from NeuroPilot Portal
45-
```bash
46-
pip3 install mtk_neuron-8.2.19-py3-none-linux_x86_64.whl
47-
pip3 install mtk_converter-8.13.0+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
48-
```
49-
- Set evironment variables for building backend
50-
```bash
51-
export NEURON_BUFFER_ALLOCATOR_LIB=<path_to_buffer_allocator>
52-
```
53-
Additionally, make sure to copy `NeuronAdapter.h` to the following directory: `backends/mediatek/runtime/include/api/`.
19+
- Linux operating system
20+
- Python dependencies:
21+
```bash
22+
pip3 install -r requirements.txt
23+
```
24+
- NeuroPilot SDK Python wheels (download from [NeuroPilot Express SDK](https://neuropilot.mediatek.com/resources/public/npexpress/en/docs/npexpress)):
25+
```bash
26+
pip3 install mtk_neuron-8.2.19-py3-none-linux_x86_64.whl
27+
pip3 install mtk_converter-8.13.0+public-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
28+
```
5429

55-
## Build
30+
## Using the MediaTek Backend
5631

57-
### Ahead of time:
32+
### Exporting and Lowering a Model
5833

59-
**Exporting a PyTorch Model for MediaTek Backend**:
60-
1. Lower and export the `.pte` file for on-device execution. The export script samples are povided under `example/mediatek/`. For example, the following commnad exports the `.pte` using the scripts provided.
34+
To export and lower a model for the MediaTek backend, use the provided shell script:
6135
```bash
6236
cd executorch
63-
6437
./examples/mediatek/shell_scripts/export_oss.sh mobilenetv3
6538
```
39+
The exported `.pte` file is saved in a directory named after the model.
6640

67-
2. Find the `.pte` files under the directory named as same as the model.
41+
### Partitioner API
6842

69-
### Runtime:
43+
A list of CompileSpec is suppported by MediaTek backend:
44+
- `platform-config`: Specifies the targeted MediaTek platform name to compile for.
7045

71-
**Build MediaTek Backend for ExecuTorch Runtime**
72-
1. Navigate to `backends/mediatek/scripts/` directory.
46+
## Runtime Integration
7347

74-
2. **Build MediaTek Backend**: Once the prerequisites are in place, run the `mtk_build.sh` script to start the build process:
75-
```bash
76-
./mtk_build.sh
77-
```
48+
This section presents an example of exporting and deploying a model. Please refer to `executorch/examples/mediatek/` for export and execution examples of various of models.
7849

79-
3. MediaTek backend will be built under `cmake-android-out/backends/` as `libneuron_backend.so`.
50+
### Building Example Runners
8051

81-
**Build a runner to execute the model on the device**:
82-
1. Build the runners and the backend by exedcuting the script:
52+
Build example runners:
8353
```bash
8454
./mtk_build_examples.sh
8555
```
56+
Runners are located in `cmake-android-out/examples/mediatek/`.
8657

87-
2. The runners will be built under `cmake-android-out/examples/`
58+
### Deploying to Device
8859

89-
## Deploying and running on a device
60+
1. Push `libneuron_backend.so`, `libneuronusdk_adapter.mtk.so` and `libneuron_buffer_allocator.so` to the device.
61+
2. Set the library path before running ExecuTorch:
62+
```bash
63+
export LD_LIBRARY_PATH=<path_to_neuron_backend>:<path_to_usdk>:<path_to_buffer_allocator>:$LD_LIBRARY_PATH
64+
```
9065

91-
1. **Push MediaTek universal SDK and MediaTek backend to the device**: push `libneuronusdk_adapter.mtk.so` and `libneuron_backend.so` to the phone and export it to the `$LD_LIBRARY_PATH` environment variable before executing ExecuTorch with MediaTek backend.
66+
### Building the Backend from Source
67+
1. Copy `NeuronAdapter.h` to `backends/mediatek/runtime/include/api/`
9268

69+
2. Set NDK Path: Ensure that the `$ANDROID_NDK` environment variable is set to the path where the NDK is located.
9370
```bash
94-
export LD_LIBRARY_PATH=<path_to_usdk>:<path_to_neuron_backend>:$LD_LIBRARY_PATH
71+
export ANDROID_NDK=<path_to_android_ndk>
9572
```
73+
74+
3. Build the backend library `libneuron_backend.so`:
75+
```bash
76+
cd backends/mediatek/scripts/
77+
./mtk_build.sh
78+
```
79+
The output is `libneuron_backend.so` in `cmake-android-out/backends/mediatek/`.

0 commit comments

Comments
 (0)