Apple M1 - Rosetta 2 (workaround)

[raymondlo84]

I've just tested this repository on Apple M1 (with Rosetta 2) and it works (partially)

Here's what I have done to get it running.

Step 1.
Install Python 3.7.x with Brew (https://brew.sh/2020/12/01/homebrew-2.6.0/) with Rosetta 2.

Step 2.
Follow the README.me to check out the repository and setup the python virtual environment

Step 3.
Do not run the pip install -r requirement.txt, instead run these separately! and make sure you update the pip to the latest.

python -m pip install --upgrade pip 
pip install jupyterlab
pip install ipywidgets
pip install tensorflow
pip install openvino-dev
pip install opencv-python
pip install matplotlib
#this one is a must or it will break the ONNX export
pip install geffnet==0.9.8 
pip install onnx
pip install torch
pip install torchvision 

KNOWN issues:
All of the example codes with Model Optimizer are not working... However, one with pre-trained model works! So you can try the monodepth one and the hello-world one! ...

Enjoy!

[ryanloney]

Very cool. If you skip the Model Optimizer part of the 102-pytorch-onnx notebook, does the ONNX model run in OpenVINO?

[helena-intel]

Could you try if pip install -r requirements.txt --use-deprecated=legacy-resolver also works?

[raymondlo84]

=) Have to try it out next when I have my hands on the Apple M1!

[raymondlo84]

... It seems that option is removed?

(openvino_env) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % pip install -r requirements.txt --use-deprecated=legacy-resolver

Usage:
pip install [options] [package-index-options] ...
pip install [options] -r [package-index-options] ...
pip install [options] [-e] ...
pip install [options] [-e] ...
pip install [options] <archive url/path> ...

option --use-deprecated: invalid choice: 'legacy-resolver' (choose from )

[raymondlo84]

(openvino_env) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % pip install -r requirements.txt 
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Ignoring torch: markers 'sys_platform == "linux" or platform_system == "Windows"' don't match your environment
Ignoring torchvision: markers 'sys_platform == "linux" or platform_system == "Windows"' don't match your environment
Collecting openvino-dev
  Using cached openvino_dev-2021.3.0-2807-py3-none-any.whl (2.3 MB)
Processing /Users/raymondlo/Library/Caches/pip/wheels/60/c9/8d/698815f2b2d3b561e631cd8e46ea1edce585dac8624aa8bd75/matplotlib-3.3.4-cp37-cp37m-macosx_11_0_x86_64.whl
Collecting gdown
  Downloading gdown-3.12.2.tar.gz (8.2 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting geffnet==0.9.8
  Using cached geffnet-0.9.8-py3-none-any.whl (37 kB)
Collecting fastseg
  Using cached fastseg-0.1.2-py3-none-any.whl (13 kB)
Collecting onnx
  Downloading onnx-1.9.0.tar.gz (9.8 MB)
     |████████████████████████████████| 9.8 MB 4.2 MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting ipywidgets
  Using cached ipywidgets-7.6.3-py2.py3-none-any.whl (121 kB)
Collecting torch<=1.7.1,>=1.5.1
  Using cached https://download.pytorch.org/whl/cpu/torch-1.7.0-cp37-none-macosx_11_0_x86_64.whl (107.7 MB)
Collecting torchvision<=0.8.2,>=0.6.1
  Using cached https://download.pytorch.org/whl/torchvision-0.8.1-cp37-cp37m-macosx_11_0_x86_64.whl (1.0 MB)
Collecting networkx>=1.11
  Using cached networkx-2.5.1-py3-none-any.whl (1.6 MB)
Collecting defusedxml>=0.5.0
  Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow<2.4,>=2.2 (from -r requirements.txt (line 20)) (from versions: none)
ERROR: No matching distribution found for tensorflow<2.4,>=2.2 (from -r requirements.txt (line 20))
WARNING: You are using pip version 20.2.4; however, version 21.1 is available.
You should consider upgrading via the '/Users/raymondlo/Documents/openvino_notebooks/openvino_env/bin/python -m pip install --upgrade pip' command.
(openvino_env) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % python -m pip install --upgrade pip==20.2.4

Requirement already up-to-date: pip==20.2.4 in ./openvino_env/lib/python3.7/site-packages (20.2.4)
WARNING: You are using pip version 20.2.4; however, version 21.1 is available.
You should consider upgrading via the '/Users/raymondlo/Documents/openvino_notebooks/openvino_env/bin/python -m pip install --upgrade pip' command.

[helena-intel]

@raymondlo84 it is not removed (yet), it doesn't exist yet in your version of pip :-) You should still upgrade pip to the latest version as in your original steps, the only thing that changes is that instead of installing the requirements step by step you now install them in one go with this --use-deprecated option.

This option may stop working at some point, but hopefully by that time we will not need it anymore. I verified that the option still exists in the most recent version of pip, 21.1. If pip does remove this option at some point in the future, you can install this pip version with python -m pip install 21.1

[raymondlo84]

I can confirm that the new method solved the installation issue. Model Optimizer is not working unfortunately. But the IE (inference engine) works and so we can actually run the hello world, monodepth and superresolution demo with the pre-trained models =).

Python executable: /Users/raymondlo/Documents/openvino_notebooks/openvino_env/bin/python
OpenVINO environment activated: OK
Python version: 3.7 OK
OpenVINO pip package installed: OK
OpenVINO import succeeds: OK
OpenVINO development tools installed: OK
OpenVINO not installed globally: OK
Numpy version: 1.18.5 OK

Everything looks good!

(openvino_env) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % python
Python 3.7.9 (default, Nov 20 2020, 23:58:42) 
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 

(openvino_env) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % uname -ar
Darwin Raymonds-MacBook-Pro.local 20.4.0 Darwin Kernel Version 20.4.0: Thu Apr 22 21:46:41 PDT 2021; root:xnu-7195.101.2~1/RELEASE_ARM64_T8101 arm64

[PTMMachineLearning]

@raymondlo84 Hello I am on a Facebook Pytorch team reaching out to understand the status and if everything is now working?

[raymondlo84]

@raymondlo84 Hello I am on a Facebook Pytorch team reaching out to understand the status and if everything is now working?

Hi, I have to double-check on the model optimizer again with Apple M1. There were dependencies issues related to Tensorflow last time. . .

[raymondlo84]

I've verified that the ONNX conversion works without any workaround needed! YAH!.

Here is what I did to get it running.

1. Create a terminal with Rosetta 2 enabled
2. Install brew (x86_64)!
3. Install python@3.7 with brew
4. Follow exact steps in the Github README, and make sure we are using Python3.7

The major bug now seems to be the Tensorflow -> IR conversion. It just failed without any warning or error messages. The ONNX one works, and I can run our 102-PyTorch tutorial successfully and many other ones without Tensorflow Conversion :)

Please let me know if you see the same on your M1?

Exporting ONNX model to IR... This may take a few minutes.
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/Users/raymondlo/Documents/openvino_notebooks/notebooks/102-pytorch-onnx-to-openvino/model/fastseg1024.onnx
	- Path for generated IR: 	/Users/raymondlo/Documents/openvino_notebooks/notebooks/102-pytorch-onnx-to-openvino/model
	- IR output name: 	fastseg1024
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	[1,3, 512, 1024]
	- Mean values: 	[123.675, 116.28 , 103.53]
	- Scale values: 	[58.395, 57.12 , 57.375]
	- Scale factor: 	Not specified
	- Precision of IR: 	FP16
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	None
	- Reverse input channels: 	False
ONNX specific parameters:
	- Inference Engine found in: 	/Users/raymondlo/Documents/openvino_env_3.7/lib/python3.7/site-packages/openvino
Inference Engine version: 	2021.4.0-3839-cd81789d294-releases/2021/4
Model Optimizer version: 	2021.4.0-3839-cd81789d294-releases/2021/4
[ WARNING ]  Const node 'Resize_753/Add_input_port_1/value829230260' returns shape values of 'float64' type but it must be integer or float32. During Elementwise type inference will attempt to cast to float32
[ WARNING ]  Const node 'Resize_774/Add_input_port_1/value832430262' returns shape values of 'float64' type but it must be integer or float32. During Elementwise type inference will attempt to cast to float32
[ WARNING ]  Const node 'Resize_797/Add_input_port_1/value835630264' returns shape values of 'float64' type but it must be integer or float32. During Elementwise type inference will attempt to cast to float32
[ WARNING ]  Const node 'Resize_821/Add_input_port_1/value838830266' returns shape values of 'float64' type but it must be integer or float32. During Elementwise type inference will attempt to cast to float32
[ WARNING ]  Changing Const node 'Resize_753/Add_input_port_1/value829231231' data type from float16 to <class 'numpy.float32'> for Elementwise operation
[ WARNING ]  Changing Const node 'Resize_774/Add_input_port_1/value832430544' data type from float16 to <class 'numpy.float32'> for Elementwise operation
[ WARNING ]  Changing Const node 'Resize_797/Add_input_port_1/value835630832' data type from float16 to <class 'numpy.float32'> for Elementwise operation
[ WARNING ]  Changing Const node 'Resize_821/Add_input_port_1/value838831060' data type from float16 to <class 'numpy.float32'> for Elementwise operation
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /Users/raymondlo/Documents/openvino_notebooks/notebooks/102-pytorch-onnx-to-openvino/model/fastseg1024.xml
[ SUCCESS ] BIN file: /Users/raymondlo/Documents/openvino_notebooks/notebooks/102-pytorch-onnx-to-openvino/model/fastseg1024.bin
[ SUCCESS ] Total execution time: 23.87 seconds. 
[ SUCCESS ] Memory consumed: 572 MB. 

[PTMMachineLearning]

Thank you for the update. Understand the ONNX conversion works. Going forward, if there is interest in accelerated ML workflow, training or inference, on MacOS M1, I would be interested in knowing more about the project. Thanks again.

[raymondlo84]

I don't know if I can share much here. But I think since it's open source, we can compile everything natively if needed. We do have many plugins including ARM, and so this is a good starter if you are interested in seeing M1 running natively with OpenVINO =).

[raymondlo84]

Just managed to compile the source and run the notebooks as well under Rosetta 2! :) Here is the cmake command I used and the instruction is the same as the official Mac OS one :) But Tensorflow still crashed so I don't know what to do with it still, and I think the next step is to compile everything under arm architecture and then also enable the ARM plugin.

https://github.com/openvinotoolkit/openvino/wiki/BuildingForMacOS#additional-build-options

cmake \                                                             
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX="${OPENVINO_BASEDIR}/openvino_dist" \
-DPYTHON_EXECUTABLE=$(which python3) \
-DENABLE_MYRIAD=OFF \
-DENABLE_VPU=OFF \
-DENABLE_PYTHON=ON \
-DNGRAPH_PYTHON_BUILD_ENABLE=ON -DENABLE_AVX512F=OFF \
-DPYTHON_EXECUTABLE=/usr/local/Cellar/python@3.7/3.7.11/Frameworks/Python.framework/Versions/3.7/bin/python3.7m \
-DPYTHON_LIBRARY=/usr/local/Cellar/python@3.7/3.7.11/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7m.dylib \
-DPYTHON_INCLUDE_DIR=/usr/local/Cellar/python@3.7/3.7.11/Frameworks/Python.framework/Versions/3.7/include/python3.7m ..

[raymondlo84]

Just tested Apple M1 with MacOS 12.2.

All scripts run properly straightly from the macOS wiki page, and no need to give any sort of workaround. However, the model optimizer still fails and also we have poor performance with FP16-INT8. FP16 seems to run 3x faster than the FP16-INT8.

[antoinebrl]

Hi! Thanks for sharing the outcome of your test. Could you describe the steps you took?
I tried to compile opencv and openvino with the cmake command you shared above (with -DTHREADING=SEQ). But sadly not device is detected:

python /opt/intel/openvino/deployment_tools/inference_engine/samples/python/hello_query_device/hello_query_device.py

[ INFO ] Creating Inference Engine
[ INFO ] Available devices:

[raymondlo84]

=) You need to compile the ARM plugin!

[raymondlo84]

https://github.com/openvinotoolkit/openvino_contrib/blob/master/modules/arm_plugin/README.md