Krita is one of my favorite open source programs. Krita is a raster graphics editor designed for digital art. This guide you’ll have to go to a few external websites on how to pair Stable Diffusion inside your Krita Application.
Step 1: Make sure you have downloaded and installed the latest version of Krita 5.2.0 or Newer
Website: https://krita.org/en/
Step 2: Download the Plugin
https://github.com/Acly/krita-ai-diffusion?tab=readme-ov-file
According to the official github. You will want to Unpack the archive into your pykrita folder.
Windows: Usually C:\Users\<user>\AppData\Roaming\krita\pykrita
Linux: Usually ~/.local/share/krita/pykrita
MacOS: Usually ~/Library/Application Support/krita/pykrita
Step 3: Enable The Plugin go to Settings tab and select Configure Krita.
On the left hand side scroll down to Python Plugin Manager and select the AI Image Diffusion and enable that then click Ok.
Now let’s create a new image hold CTRL-N to do that or go File New Image. And Select Create.
Now you will have a blank canvas. Now let’s enable the docker. Go to settings, dockers then AI Image Generation.
Step 4: Getting the API and adding it to Krita.Now we need to get the ComfyUI API. So go to Run Diffusion and let’s load up a Comfy server.
Make sure you are on the Beta Release as of writing this it is the most up to date.
Scroll down and turn on API then select Launch. Wait a few moments for the server to fire up.
When we are up and running we need to install a custom node. To do that we click on manager in the bottom right.
Click on Install Custom Nodes
In the searchbox type tooling and then install the tooling node that comes up.
You can then restart.
Now time to copy the API! Close those windows and click on the top right. A box will pop up saying API enabled and a link. Copy that link.
Step 5: Let’s test it out!
Head back over to Krita. The last thing we did was enable that docker so we should see (on my version) in the bottom right the new Plugin there. Click on the gear box that will bring you to the settings.
You should be on the connection tab. From here select Connect to External server. Then copy paste the API in that box and select connect. It should so a GREEN connection like below.
Important note: If you have problems connecting to the server and an error pops up, please do the following:
- Make sure your Krita install is on the latest version
- Make sure you have updated the Krita AI Generation extension to the latest version (download the latest or do a git pull on the directory via command line)
If it still fails, make a support ticket and let them know your Krita version and that you have updated to the latest, along with the error message and software version of the server you have launched.
Now that you are connected, click on Styles real quick (This is optional but what I did.) I made a new style with the Juggernaut7 Run Diffusion merge model. And took a look at the neg/positive prompts that each generation with this style will add to my generations. You can edit that as needed. You can also add Loras here if you want. We can go into more details on the ins and out later. Click OK when you are done.
Create a little sketch. I made a very ugly one on purpose just to show you what it can do.
You’ll want to write up a prompt. Here is mine: A hyperdetailed photograph of a cat next to a Sunflower in a Japanese ceramic vase on a table in a traditional japanese room with a window and sunbeams, 8k resolution, beautiful photograph
Now you can see I turned down the strength. That is like the denoising on IMG2IMG in Automatic1111 the lower it is the closer it will look like your sketch.
At 76% you can see it has a very similar layout to my image. But it is not photo realistic like my prompt so let’s turn that up more.
At 100% It follows my prompt more and still has a similar layout. Play with the strength and prompting to get the results you want.
If you are happy and want to keep drawing on top of this one click back on Layers tab and you can select which layer to generate with and merge layers and so forth.
Remember to enjoy the process.
Another example. Remember to have fun.
