How to Set Up and Run ComfyUI on Lightning AI

Table of Contents

If you've been playing around with Stable Diffusion, the AI image generation model, you might've heard of ComfyUI - its powerful interface that's actually more "confusing UI" than comfy.

Don't let that scare you off, though! ComfyUI is actually one of the BEST ways to run Stable Diffusion, offering flexibility, control, and speed. Its node-based interface is optimized for efficiency, making image generation quicker than other UIs like AUTOMATIC1111.

But what if you don't have the GPU power or prefer not to clutter your system with multiple installations? That's where Lightning AI comes in - a platform that lets you access up to 22 free GPU hours per month.

This guide will walk you through the process of setting up and running ComfyUI on Lightning AI, allowing you to harness cloud-based GPU resources for efficient AI image generation.

If you're more of a visual learner or simply want to see how it's done, I recommend checking out my YouTube tutorial. It covers everything step-by-step.

Want to Take a Shortcut?

If you want to speed things up, I’ve got a ComfyUI template on Lightning AI that’ll have you generating images in no time. It’s ready to go with ComfyUI and the SDXL 1.0 model—just duplicate it and make it your own.

If you'd rather set up ComfyUI yourself from scratch, stick around and follow my step-by-step guide.

Setting Up Lightning AI

Sign up for Lightning AI at their website.
Create a new studio by selecting the "Code" environment.
Delete the default main.py file in your new studio.

For more information on lightning.ai, checkout my latest post Forget Colab: Lightning AI Is the FREE Dev Tool I Wish I Knew Sooner!

Installing ComfyUI

Open a new Terminal window in your studio.

Clone the ComfyUI repository:

git clone https://github.com/comfyanonymous/ComfyUI.git

Install the required dependencies:
```
pip install -r ComfyUI/requirements.txt
```

Installing ComfyUI Manager (Optional but Recommended)

Clone the ComfyUI Manager repository:

git clone https://github.com/ltdrdata/ComfyUI-Manager.git ComfyUI/custom_nodes/ComfyUI-Manager

Install the manager's dependencies:

pip install -r ComfyUI/custom_nodes/ComfyUI-Manager/requirements.txt

If you don't see the custom_nodes folder, it might be due to a setting in Lightning AI. To fix this, go to Visual Studio Code's settings (found at the bottom left corner of your screen). Search for "gitignore", and ensure that the "Explorer: Exclude Git Ignore" setting has been unchecked. This should reveal all hidden folders, including custom_nodes.

Running ComfyUI

Switch to GPU mode in Lightning AI:
- Click on the Studio Environment button.
- Select "Switch to GPU" and choose a GPU server.
- Check "Remember last used machine" for future sessions.
Run ComfyUI:
```
python ComfyUI/main.py
```

Connecting to ComfyUI

Install the Port Viewer plugin in Lightning AI.
Create a new port:
- Label it "Stable Diffusion on Comfy UI"
- Set the port number to 8188

Automating ComfyUI Startup

Add the ComfyUI start command to the on_start.sh script in your studio.

Downloading Checkpoint Files

There are three methods to download model checkpoint files:

Using ComfyUI Manager

In ComfyUI, click "Manager" then "Model Manager".
Search for "checkpoint" to filter results.
Find and install desired models (e.g., SDXL Turbo).
Click "Refresh" in ComfyUI to see new models.

Direct Download from Model Websites

The second way of downloading models is via the terminal. There are many models available out there, and you can find them on websites such as HuggingFace or CivitAI.

Go to the model page for the one you'd like to download - let's take the Juggernaut XL model as an example.

To download the checkpoint file, you'll need to run a specific command in the terminal within Lightning Studio – its format should look something like this:

wget -O ComfyUI/models/checkpoints/<MODEL_NAME>.safetensors "<DOWNLOAD_URL>&token=<API_TOKEN>"

Here's how to break down the command:

<MODEL_NAME> is where you'll enter the downloaded file's name, making it easy to recognize later. For example, we could call it Juggernaut_XL_Hyper
<DOWNLOAD_URL> comes from copying the link address of the specific model version's download button on civit.ai by right-clicking and selecting "Copy link address". The resulting URL is https://civitai.com/api/download/models/471120?type=Model&format=SafeTensor&size=full&fp=fp16.
<API_TOKEN> is your unique API key from civit.ai – you might not need this for every model, especially those that can be downloaded without logging in. For models requiring an account, sign up, create an API key under Account Settings > API keys, generate a new one, name it, and you're all set! Keep in mind to safeguard your API key. Once I've completed this tutorial, I'll revoke it straight away.

In our case example, here's what the command could look like:

wget -O ComfyUI/models/checkpoints/Juggernaut_XL_Hyper.safetensors "https://civitai.com/api/download/models/471120?type=Model&format=SafeTensor&size=full&fp=fp16&token=275dd0a429d0edff942e913565e4e63d"

Uploading Local Files

Navigate to the "comfyui/models/checkpoints" folder in your studio.
Right-click and choose "upload" to add local checkpoint files.

Using ComfyUI for Image Generation

Launch your studio and wait for it to load.
Open ComfyUI through the Port Viewer plugin.
In the node-based interface:
- Select your downloaded checkpoint.
- Adjust image settings (e.g., resolution for SDXL models).
- Modify prompts and other parameters as needed.
Click "Queue Prompt" to generate an image.
Right-click on the generated image to download it.

By following these steps, you can efficiently run ComfyUI on Lightning AI, leveraging cloud-based GPU resources for AI image generation without cluttering your local system.