LLM with Oobabooga

Let’s Learn how to Load a Large Language Model (LLM) on the WebUI Oobabooga on Run Diffusion.

Step 1: Fire Up a Medium Sized Server. This will do the trick for most of the Opensource LLMs. You will see this page load

image

I always click on the time top “hide my image gallery” because I’m working on a laptop. But if it doesn’t bother you feel free to keep it visible.

Step 2: Click on the Model Tab

image

Step 3: Select TheBloke_Mistral-7B-v0.1-GGUF on the Model Drop down. Wait a few moments and you will see it start to load up the settings. Model loader will change to llama.cpp and you will also notice under download on the far right it will say “Succesfully loaded.”

image

Step 4: Click on the Chat tab in the top left.

image

Step 5: Type in the box and hit Generate.

image

You can see that this created a basic assistant that I was able to have a little conversation with. What if we want a more unique interaction? We can load characters with preprogrammed personalities to tune the LLM to our interests or needs.

Loading a Character

Step 6: Click on the Parameters tab at the top.

image

Step 7: Click on the Character Tab

image

Here we can create our own character, load preloaded characters or upload our own. For now we will load one that comes with your Run Diffusion account. In future tutorials I’ll show you how to make your own or download some off the internet that others have made.

Step 8: In the Drop down select Example.

image

You can see that it has loaded up a character named Chiharu Yamada. Under context it shows a little bit about who she is and on the top right we can see her profile picture. It also has a place for us to upload a picture of ourselves if we want too. Let’s leave it as is for now.

Step 9: Once again select the Chat tab at the top.

image

You can see it loaded a little roleplay description and you are now able to chat with Chiharu.

Step 10: Chat and have fun.

image
About the author

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to .

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.