DEV Community

Craig Morten
Craig Morten

Posted on • Edited on

Setting up Stable Diffusion for MacOS

#python #ai

With the landscape quickly changing, this article is fast becoming outdated!

If you face issues with the tutorial below I recommend you checkout the latest advice here.

Stable Diffusion is a latent text-to-image diffusion model that was recently made open source.

For Linux users with dedicated NVDIA GPUs the instructions for setup and usage are relatively straight forward. However for MacOS users you can't use the project "out of the box". Not to worry! There are some steps to getting it working nevertheless!

Environment Setup

To begin you need Python, Conda, and a few other libraries set up:

# Install Python, Cmake, Git, and Protobuf
brew install python \
  cmake \
  git \
  protobuf

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install Conda:
## Either use this for older "pre-M1" Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash Miniconda3-latest-MacOSX-x86_64.sh

## Or use this for older M1 Macs:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
bash Miniconda3-latest-MacOSX-arm64.sh
Enter fullscreen mode Exit fullscreen mode

You may need to restart your terminal at this point for the changes for the new libraries to be picked up.

Clone this fork of the project and checkout the apple patch branch:

git clone https://github.com/magnusviri/stable-diffusion
cd stable-diffusion
git checkout apple-silicon-mps-support
Enter fullscreen mode Exit fullscreen mode

At this point you will need to make sure you're using Python 3, check out this article for different ways to make Python 3 the default version on your Mac.

Set up the Conda environment:

conda env create -f environment-mac.yaml
conda activate ldm
Enter fullscreen mode Exit fullscreen mode

And finally set the following environment variable:

export PYTORCH_ENABLE_MPS_FALLBACK=1
Enter fullscreen mode Exit fullscreen mode

Code Changes

Our environment is now set up, but we have a few tweaks that we need to allow the code to gracefully fallback to using the CPU (if required!).

Append .contiguous() at ldm/models/diffusion/plms.py#L27 resulting in:

-        attr = attr.to(torch.float32).to(torch.device(self.device_available))
+        attr = attr.to(torch.float32).to(torch.device(self.device_available)).contiguous()
Enter fullscreen mode Exit fullscreen mode

Similarly append a new line x = x.contiguous() after ldm/modules/attention.py#L211 so it looks something like:

def _forward(self, x, context=None):
+       x = x.contiguous()
        x = self.attn1(self.norm1(x)) + x
Enter fullscreen mode Exit fullscreen mode

Download Stable Diffusion Weights

Let's install our diffusion weights

curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt
Enter fullscreen mode Exit fullscreen mode

Create Images 🚀

You should now be ready to generate images on your MacOS device using Stable Diffusion! 🎉 🎉

python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
Enter fullscreen mode Exit fullscreen mode

A drawing of web developers

Tricks and hacks gleamed from https://github.com/CompVis/stable-diffusion/issues/25 - credit to all the folks in that thread for figuring out how to get things working!

Top comments (41)

Collapse
 
magnusviri profile image
James Reynolds • • Edited

Hey all, I'm magnusviri. The instructions here need to be modified like this.

git checkout https://github.com/lstein/stable-diffusion

Don't check out anything. After conda activate ldm run this.

python scripts/preload_models.py

The latest instructions are here.

I haven't updated them with python scripts/preload_models.py. Anyone can get a github account, fork lstein's repo, update the readme with the latest info, and do a pull request to get it updated for everyone. This is the beauty of open source.

This stuff is moving extremely fast and is extremely complex. It's moving so fast that since it's moved from magnusviri/stable-diffusion to lstein/stable-diffusion I haven't even had time to double check that the readme is even accurate or update it with the latest list of errors plaguing people. While people are trying to make it as easy as possible this is nowhere near ready for the masses and non-power users.

Collapse
 
craigmorten profile image
Craig Morten •

Nice one @magnusviri! I've put a notice at the top as my personal notes from having a play are fast becoming unfit for the enthusiasm of the community!

Collapse
 
asmang profile image
Greg Asman • • Edited

Thanks for the detailed instructions! Unfortunately, I'm getting stuck at environment creation where pip is failing with the message below. Any thoughts on how to get over this hurdle? Thanks again!

(Note: It appears to be an issue with onnx. I tried a pip install onnx and it went through every version unsucessfully.)

Pip subprocess error:
  Running command git fetch -q --tags
  Running command git reset --hard -q 24268930bf1dce879235a7fddd0b2355b84d7ea6
  Running command git fetch -q --tags
  Running command git reset --hard -q d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      fatal: not a git repository (or any of the parent directories): .git
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/kk/qyr1f5dj017b338m860tt9dm0000gn/T/pip-install-5of30ges/onnx_8af078f05ebe4736b1b65355c7b7406b/setup.py", line 81, in <module>
          assert CMAKE, "Could not find cmake executable!"
      AssertionError: Could not find cmake executable!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

failed

CondaEnvException: Pip failed
Enter fullscreen mode Exit fullscreen mode
Collapse
 
careuno profile image
Carlos Merchán •

im having the same issue, any clue?

Collapse
 
craigmorten profile image
Craig Morten • • Edited

I’ll give my own instructions another go and see if can reproduce…

What version of python/pip are you using? IIRC you need to use python 3

Updates:

  1. If you followed the steps verbatim the conda installation step was incorrect - I have amended!
  2. Just noticed your error is regarding not having cmake, adding some env setup steps for this now! You need to install this brew install cmake
Thread Thread
 
careuno profile image
Carlos Merchán •

I’ll check again installing cmake
Question: why are you using /Miniconda3-latest-MacOSX-x86_64.sh instead of /Miniconda3-latest-MacOSX-arm64.sh ?

Thread Thread
 
craigmorten profile image
Craig Morten •

Miniconda3-latest-MacOSX-x86_64.sh is for "pre-M1" macs, e.g. I use a MacBook Air 2017 which uses an Intel chip.

Miniconda3-latest-MacOSX-arm64.sh is for Apple M1 😄

Thread Thread
 
careuno profile image
Carlos Merchán •

I was confused because you use apple-silicon-mps-support branch and I thought you had a M1 cpu. Anyway now it's working for me, thanks

Collapse
 
tangsirongit profile image
laimj • • Edited

Thanks for the detailed instructions!

after set up the enviroment.

the last step:
python scripts/orig_scripts/txt2img.py --prompt "An Armenia girl with curly hair goes to Senior school with her mum in Shanghai, and she carries a dark red shoulder bag" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1

the error:

Loading model from sd-v1-4.ckpt
Global Step: 470000
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Traceback (most recent call last):
  File "/Users/tangsir/Downloads/tools/stable-diffusion/scripts/orig_scripts/txt2img.py", line 327, in <module>
    main()
  File "/Users/tangsir/Downloads/tools/stable-diffusion/scripts/orig_scripts/txt2img.py", line 203, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/Users/tangsir/Downloads/tools/stable-diffusion/scripts/orig_scripts/txt2img.py", line 42, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/tangsir/Downloads/tools/stable-diffusion/ldm/util.py", line 89, in instantiate_from_config
    return get_obj_from_str(config['target'])(
  File "/Users/tangsir/Downloads/tools/stable-diffusion/ldm/models/diffusion/ddpm.py", line 657, in __init__
    self.instantiate_cond_stage(cond_stage_config)
  File "/Users/tangsir/Downloads/tools/stable-diffusion/ldm/models/diffusion/ddpm.py", line 768, in instantiate_cond_stage
    model = instantiate_from_config(config)
  File "/Users/tangsir/Downloads/tools/stable-diffusion/ldm/util.py", line 89, in instantiate_from_config
    return get_obj_from_str(config['target'])(
  File "/Users/tangsir/Downloads/tools/stable-diffusion/ldm/modules/encoders/modules.py", line 253, in __init__
    self.tokenizer = CLIPTokenizer.from_pretrained(
  File "/Users/tangsir/opt/anaconda3/envs/ldm/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1768, in from_pretrained
    raise EnvironmentError(
OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'openai/clip-vit-large-patch14' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.
Enter fullscreen mode Exit fullscreen mode
Collapse
 
tangsirongit profile image
laimj •

fixed by removing local_files_only=True:

diff --git a/ldm/modules/encoders/modules.py b/ldm/modules/encoders/modules.py
index 07e1da8..f7f5abb 100644
--- a/ldm/modules/encoders/modules.py
+++ b/ldm/modules/encoders/modules.py
@@ -118,7 +118,7 @@ class BERTTokenizer(AbstractEncoder):
         #   BertTokenizerFast.from_pretrained("bert-base-uncased")
         try:
             self.tokenizer = BertTokenizerFast.from_pretrained(
-                'bert-base-uncased', local_files_only=True
+                'bert-base-uncased'
             )
         except OSError:
             raise SystemExit(
@@ -251,10 +251,10 @@ class FrozenCLIPEmbedder(AbstractEncoder):
     ):
         super().__init__()
         self.tokenizer = CLIPTokenizer.from_pretrained(
-            version, local_files_only=True
+            version
         )
         self.transformer = CLIPTextModel.from_pretrained(
-            version, local_files_only=True
+            version
         )
         self.device = device
         self.max_length = max_length
Enter fullscreen mode Exit fullscreen mode
Collapse
 
sjoerdw_bijleveld_38e54 profile image
Sjoerd W. Bijleveld •

I had the same problem and your method solved it, but the next error crept up:

Traceback (most recent call last):
Traceback (most recent call last):
File "/Users/SWB/stable-diffusion/scripts/orig_scripts/txt2img.py", line 327, in <module>
main()
File "/Users/SWB/stable-diffusion/scripts/orig_scripts/txt2img.py", line 277, in main
samples_ddim, _ = sampler.sample(S=opt.ddim_steps,
File "/Users/SWB/miniconda3/envs/ldm/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/Users/SWB/stable-diffusion/ldm/models/diffusion/plms.py", line 156, in sample
self.make_schedule(ddim_num_steps=S, ddim_eta=eta, verbose=verbose)
File "/Users/SWB/stable-diffusion/ldm/models/diffusion/plms.py", line 108, in make_schedule
(1 - self.alphas_cumprod_prev)
TypeError: unsupported operand type(s) for -: 'int' and 'builtin_function_or_method'
(ldm) SWB@SWB-iMac stable-diffusion %

For me it is hard to find the reason for this error. Please help!

Collapse
 
frederik_list_e55eb0bd135 profile image
Frederik List • • Edited

Hi, thanks for the tutorial, haven't managed to get it to work yet. I got this error:

FileNotFoundError: [Errno 2] No such file or directory: 'sd-v1-4.ckpt'
Enter fullscreen mode Exit fullscreen mode

Edit:
I don't know what I did, but now the error is

FileNotFoundError: [Errno 2] No such file or directory: 'models/ldm/stable-diffusion-v1/model.ckpt'
Enter fullscreen mode Exit fullscreen mode

Edit:
I found the file somewhere else on my mac and put it in the correct folder and renamed it model.ckpt, but it resulted in the following error :

OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'openai/clip-vit-large-patch14' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.
Enter fullscreen mode Exit fullscreen mode

Not really sure how I can proceed from here to try and make it work.

Some other issues I came across along the way:

  • The branch name seems to be apple-mps-support and not apple-silicon-mps-support
  • There is no script named txt2img.py in the /scripts, but there is in /scripts/orig_scripts, however, it doesn't work for me. Is there a new script to replace it? Which one?
Collapse
 
magnusviri profile image
James Reynolds •

github.com/lstein/stable-diffusion...

Collapse
 
nizu profile image
Nicola Zuliani •

Thanks for the instructions, but I found some problems while trying to make this to work.
Many modules looks missing on my MacOS (with M1), that I think I managed to install, but now I'm stucked.
The error I get is:

ImportError: dlopen(/Users/nicol/miniconda3/lib/python3.9/site-packages/_dlib_pybind11.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_png_do_expand_palette_rgb8_neon'
Enter fullscreen mode Exit fullscreen mode

Any idea on how to solve it? I tried to install libpng but things don't change.

Thanks again :)

Collapse
 
sjoerdw_bijleveld_38e54 profile image
Sjoerd W. Bijleveld •

Hi Craig,
I'm stuck at the point of installing Conda. I get the message: 'zsh: command not found: wget'. It seems wget is a Linux command, but I'm on a iMac (Monterey). Any help is much appreciated!

Sjoerd

Collapse
 
kaolin profile image
ʞɐoןıu ıɯɐƃo ɟıɹǝ •

you can install wget with brew, or switch to using curl to get the file also

curl -O URLHERE

Collapse
 
sjoerdw_bijleveld_38e54 profile image
Sjoerd W. Bijleveld •

Thanks! ATM I'm not able to try it, but also further investigation revealed that probably the path to the zsh shell is not valid. So I will check that also asap.

Thread Thread
 
sjoerdw_bijleveld_38e54 profile image
Sjoerd W. Bijleveld •

It turned out I indeed had to install 'wget' with brew. It does now recognise the command. So no problems with the shell path. Thanks so much!

Collapse
 
gravitide profile image
Aaron Smith • • Edited

Are other people having issues with grpcio? I have never delved into the python world much, but tried installing an earlier version into the conda environment and cleared the caches. But it will still download the latest version which fails on my M1 mac.

conda list grpcio
Enter fullscreen mode Exit fullscreen mode

Shows the installed version as 1.42.0 but I still get this error

Collecting grpcio>=1.24.3
  Downloading grpcio-1.47.0.tar.gz (21.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.9/21.9 MB 40.0 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

Pip subprocess error:
  Running command git fetch -q --tags
  Running command git reset --hard -q 24268930bf1dce879235a7fddd0b2355b84d7ea6
  Running command git fetch -q --tags
  Running command git reset --hard -q d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/3s/zhffdtdn241f_m_jvjz9qsxw0000gn/T/pip-install-gizrp4u0/grpcio_e54571b9c9134f0a854bb64e8c0605c2/setup.py", line 263, in <module>
          if check_linker_need_libatomic():
        File "/private/var/folders/3s/zhffdtdn241f_m_jvjz9qsxw0000gn/T/pip-install-gizrp4u0/grpcio_e54571b9c9134f0a854bb64e8c0605c2/setup.py", line 219, in check_linker_need_libatomic
          cpp_test = subprocess.Popen(
        File "~/opt/miniconda3/envs/ldm/lib/python3.10/subprocess.py", line 966, in __init__
          self._execute_child(args, executable, preexec_fn, close_fds,
        File "~/opt/miniconda3/envs/ldm/lib/python3.10/subprocess.py", line 1717, in _execute_child
          and os.path.dirname(executable)
        File "~/opt/miniconda3/envs/ldm/lib/python3.10/posixpath.py", line 152, in dirname
          p = os.fspath(p)
      TypeError: expected str, bytes or os.PathLike object, not list
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

failed

CondaEnvException: Pip failed
Enter fullscreen mode Exit fullscreen mode
Collapse
 
000fff profile image
000fff •

I am getting a weird error when I try and run the python script.

I have an apple Macbook Pro M1 with 64GB of ram.


(ldm) thomaspetersen@MacBook-Pro stable-diffusion % curl "https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media" > sd-v1-4.ckpt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4067M  100 4067M    0     0  63.4M      0  0:01:04  0:01:04 --:--:-- 64.5M
(ldm) thomaspetersen@MacBook-Pro stable-diffusion % python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
Intel MKL FATAL ERROR: This system does not meet the minimum requirements for use of the Intel(R) Math Kernel Library.
The processor must support the Intel(R) Supplemental Streaming SIMD Extensions 3 (Intel(R) SSSE3) instructions.
The processor must support the Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions.
The processor must support the Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
(ldm) thomaspetersen@MacBook-Pro stable-diffusion % wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
Enter fullscreen mode Exit fullscreen mode
Collapse
 
constantinez profile image
Constantinez • • Edited

thx for this tutorial
but I got error on my MacBook pro (intel):

(ldm) kos@macbook stable-diffusion % python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckpt sd-v1-4.ckpt --skip_grid --n_samples 1
zsh: segmentation fault python scripts/txt2img.py --prompt "a drawing of web developers" --plms --ckp

Collapse
 
kaolin profile image
ʞɐoןıu ıɯɐƃo ɟıɹǝ •

The only time I got a segfault was when I set KMP_DUPLICATE_LIB_OK ... I posted a couple other solutions above (recommend the "delete the conflicting library" one over the "rebuild with nomkl" but whichever you feel more comfortable with ^_^ )

Collapse
 
kaolin profile image
ʞɐoןıu ıɯɐƃo ɟıɹǝ •

FWIW [2022-08-29] —
on an intel mac

1)
ldm/modules/attention.py#L211 already had the patch

2)
ran into issues with:

Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized OMP:

I wound up redoing the environment to include nomkl and it seems to have worked, if ... so ... SO ... slow [4 minutes!] . But I'll take functional over not (thanks!) ... helps me prep for doing this on a real machine, and I can kick things off overnight. ;)

dependencies:
  - nomkl
...
Enter fullscreen mode Exit fullscreen mode
Collapse
 
kaolin profile image
ʞɐoןıu ıɯɐƃo ɟıɹǝ • • Edited

Alternately, with the info that libiomp5.dylib and libomp.dylib were likely conflicts:

 mv /Users/fire/miniconda3/envs/ldm/lib/python3.10/site-packages/torch/lib/libiomp5.dylib /Users/fire/miniconda3/envs/ldm/lib/python3.10/site-packages/torch/lib/libiomp5.bak
Enter fullscreen mode Exit fullscreen mode

seems to have worked, ignoring the nomkl path.

The KMP_DUPLICATE_LIB_OK solution only led to segfaults.

The LD_PRELOAD did nothing, possibly because I was targeting the wrong one, but possibly because the names were actually different?

View full discussion (41 comments)

Read next

mikeyoung44 profile image

100x Faster Diffusion Planning with DiffuserLite's Refinement Approach

Mike Young -

thisdotmedia_staff profile image

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

This Dot Media -

yassinesallami profile image

How to create an image authentication system with python streamlit and canva!

Yassine Sallami -

ayush7614 profile image

How to deploy Granite MOE 1B and 3B in the Cloud?

Ayush kumar -