What is ComfyUI? ComfyUI, according to author, "lets you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart based interface". Basically, we can play with stable diffusion pipelines by just moving around some nodes.
So what is stable diffusion? Its a GenAI (Generative Artificial Intelligence) which can generate images from text prompts, images or both.
Before moving on, let's understand what is Runpod. Execution of an AI model need to take place in GPU (Graphic Processing Unit) as it allows millions of tiny processing unit, working at the same time. And GPUs are costly. So rather than buying GPU and running the models locally, we can rent out the GPU service from services like RunPod. RunPod provides us the cloud service for developing and executing AI models.
RunPod allows us 2 types of services, Pods and Serverless. Pods are the services that keep on, no matter whether the required service / API is being used or not. Where as in serverless provide workers, which only works when needed, and hence charged less (only for the time worker is).
Now let's dive into deploying ComfyUI with RunPod.
Runpod Worker Comfy
We are going to use runpod-worker-comfy for the deployment.
Although the git repo has detailed explanation for Comfy deployment, I had to face some issues which may stuck you if you doing this for the first time.
We do not need to clone this repo, rather we are going to use the docker images provided by runpod worker comfy.
Runpod worker comfy provides us 5 images as follow:
- timpietruskyblibla/runpod-worker-comfy:3.1.0-base: doesn't contain anything, just a clean ComfyUI
- timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-schnell: contains the checkpoint, text encoders and VAE for FLUX.1 schnell
- timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-dev: contains the checkpoint, text encoders and VAE for FLUX.1 dev
- timpietruskyblibla/runpod-worker-comfy:3.1.0-sdxl: contains the checkpoint and VAE for Stable Diffusion XL
- timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3: contains the checkpoint for Stable Diffusion 3 medium
Dockerfile
Based on your requirement, select one of the image.
We can use the selected image as raw, or we can add some customised layers, like adding packages or custom nodes (custom nodes can be added in the network volume, discussed later). In my case, I had to use a package rembg[gpu]
, need to remove the background from a given image. So here is my updated Dockerfile:
FROM timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3
RUN pip install rembg[gpu]
CMD /start.sh
For adding custom nodes, you can do:
...
RUN git clone <custom_node_repo.git> /comfyui/custom_nodes
...
Now build an image by docker build <dockerhub-username>/<your-image-name>
. Once done, push the image to docker hub by docker image push <dockerhub-username>/<your-image-name>
. You can use Docker GUI to push your image to docker hub.
You may skip the above steps if you do not need any customisation.
Runpod
Now we need to configure our runpod.
Storage
Let's make a Network Volume, which will have our output images, custom nodes (if we haven't added those using Dockerfile) and custom models.
Go to Storage
in sidebar. Click on New Network Volume
, select the data center, volume name and the storage volume it holds.
You can select any data center, usually the one which supports the cheapest (and medium - highly available) GPU, as GPU would only be required to deploy the instance. Won't affect the image processing.
For storage volume, it depends upon your requirements. How much space is required by custom models and custom nodes. If you are not using any custom models/nodes, I think 10GB is fine.
Once the network volume is created, we need to deploy it temporarily to make it accessible so we can add the custom things.
Select any of the GPU, select GPU count as 1, and select instance pricing as Spot
Once network volume is deployed, go to Pods
in side bar and you will a see a newly listed Pod. Click on it and connect the terminal through web.
Now you will see the terminal interface in web. We need to place everything in workspace
directory because its the directory of volume and contains persistent data. Anything not in this directory will be removed as soon as we delete the deployed instance of Network Volume.
So go to /workspace by cd workspace
and make respective directories i.e for custom nodes, make custom_nodes
by command mkdir custom_nodes
. Move into custom_nodes
by cd custom_nodes
and clone your desired nodes from git by git clone <repo url>
.
Once you are done with adding custom nodes, exit the terminal by exit
and close your web terminal tab.
Serverless Deployment of Comfy
Now head over to Serverless
in sidebar and click on New Endpoint
. The below screen will shown
We have to provide our configurations.
- Select GPU
- Enter you
Endpoint Name
, this could be anything - For
Worker Configuration
section, you should go through these docs - For container configuration, leave
Template
andContainer Registry Credentials
as empty. Enter the container image as the image you chose from the runpod-worker-comfytimpietruskyblibla/runpod-worker-comfy:3.1.0-<your model>
liketimpietruskyblibla/runpod-worker-comfy:3.1.0-sd3
ForContainer Start Command
, write:sh -c "ln -sf /runpod-volume/custom_nodes/* /comfyui/custom_nodes && /start.sh"
. This command makes a symlink between comfyui directory and network volume workspace. You may also link other directories too. Note: You can skip the above container start command if you haven't installed any custom nodes, models or any other custom stuff. If so, leave this field empty. - For Container Disk, it should be more than 20GB.
- Leave HTTP and TCP ports empty
- For Environment Variables, enter these:
BUCKET_ENDPOINT_URL --- for endpoint of s3 i.e https://bucket.s3.region.amazonaws.com
BUCKET_ACCESS_KEY_ID --- for access key of s3
BUCKET_SECRET_ACCESS_KEY --- for secret key of s3
- Keep the advance sections as itself.
- Finally deploy your serverless instance.
Creating API Key
Now to access your deployed instance, you would require an API key. Which you can get by:
- Go to
Settings
in sidebar - Create new API Key, enter any desired name, select its access i.e
only read
orread & write
. - Copy the generated API Key and store it somewhere safe.
Using comfy
Let's test if our comfy works as expected. But before moving on, we need the API endpoint where we are going to request.
Go to Serverless
in sidebar and click on the instance you just deployed. It will be like this:
You can access the logs in case of debugging. For now, we need API endpoint which you will see at the bottom of image. We got 5 different routes. You may read the guide here.
You can either go with run
or runsync
to get result by accessing status
or getting result immediately, respectively. In most cases, comfy takes some time so its better to go with run
and check the status
after some interval.
API Request
You may read the docs of API Request Format here
Once workflow is created you can request the above mentioned API Endpoint with API Key in header by Bearer <API Key>
API Response for /run
It will return id
of the job, at the root of response body. You will use this id to check the status of of job by /status/
API Response for /status
It will return the status
of the job, at the root of response body, along with output
if status is COMPLETED
All Done, now you can make another layer of API which will may only take desired inputs and return the output of comfy by checking the status
at some intervals.
That's all. Happy learning❤️
Top comments (0)