This repository was archived by the owner on May 1, 2025. It is now read-only.

MPS support #40

Open

tmm1 opened this issue Jun 28, 2023 · 1 comment

tmm1 commented Jun 28, 2023

It would be nice to support Metal Performance Shaders backend for pytorch on macOS

the main changes for inference seem to be:

load_in_8bit must be False
device_map set to {"":"mps"}

see:

https://github.com/Birch-san/falcon-play/blob/cbf9b2aebe7eef3eea305a511d6cdda17282ca8a/scripts/chat_play.py#L154-L157

https://github.com/Birch-san/falcon-play/blob/cbf9b2aebe7eef3eea305a511d6cdda17282ca8a/scripts/chat_play.py#L175

cc @Birch-san are there any other good docs or resources for adding mps support?

The text was updated successfully, but these errors were encountered:

Contributor

bdqnghi commented Jul 4, 2023

this is great, can you help to propose any pull request that i can merge?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.