Skip to content

Conversation

@Meatfucker
Copy link

This includes changes to fix the precision issues in the training scripts, as well as a modification to the conda creation command so it pulls in a complete requirement list, including torch with CUDA.

@Meatfucker Meatfucker changed the title Changes to make this install cleanly on Windows. Changes to make this install and run cleanly on Windows. Jan 27, 2023
@OliverKulinski
Copy link

This worked for me. The old error no longer shows but now I get CUDA out of memory
RuntimeError: CUDA out of memory. Tried to allocate 642.00 MiB (GPU 0; 8.00 GiB total capacity; 7.21 GiB already allocated; 0 bytes free; 7.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

@Meatfucker
Copy link
Author

The program requires extremely high VRAM. I oom'd about two thirds of the way training on a 24gb card. They recommend a 36gb GPU and they seem to mean it. If someone can get it to work on lower precision, itll use less ram.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants