Stable Diffusion is a software that uses AI to create images. To guide it to create the image you want, you can use positive and negative prompts. Positive prompts are words or phrases that tell the software what you want to see in the image, while negative prompts tell it what to avoid.
Creating a good prompt is important. In fact, there are numerous websites where people seek ways to improve their prompt writing skills. This interest has given rise to a term called "prompt engineering.”
In this comprehensive guide, I'll show you how to write better prompts for Stable Diffusion. The tutorial is divided into two parts.
- Part 1: Mastering the Prompt Formula. We'll start by dissecting the anatomy of a well-crafted prompt, providing you with a clear understanding of the essential components that make up an effective prompt formula.
- Part 2: Advanced Techniques for Optimizing Your Prompts. In the second part, we'll take your newfound knowledge to the next level by exploring advanced techniques such as using weights, negative prompts, and more. You'll learn how to fine-tune your prompts to achieve even better results with Stable Diffusion.
If you prefer learning through a visual approach or want to gain additional insight into this topic, be sure to check out my YouTube video on this subject!
1. The Anatomy of a Prompt
In my experience, a prompt should incorporate specific elements to produce a high-quality image.
These include a subject, an artistic medium and style, the aesthetics of the image, and image quality. These elements collectively result in the following prompt formula:
<Artistic medium> <Adjective> Subject <Action or Accessory> <Location> <Artistic style> <Aesthetics> <Image quality>In this formula:
- All the elements enclosed in angle brackets <...> are optional. This means you can choose whether or not to include them in your prompt.
- The only mandatory element is the subject, so you must include it to describe the main focus of your image.
- The arrangement of elements is flexible, enabling you to position adjectives, actions, location, aesthetics, and image quality either before or after the subject. This lets you to tailor your prompt to align with the context or your personal style.
To gain a deeper understanding of each of these elements, we will gradually construct our prompt formula step-by-step:
- Step 1: The subject
- Step 2:Artistic Medium
- Step 3:Artistic Style
- Step 4:Aesthetics
- Step 5:Image quality
Step 1 in Building a Prompt Formula: The Subject
The subject is what you want to see in the generated image. Ask yourself: do I want an image of a person? An animal or perhaps a landscape?
For instance, let's focus on a cat as our subject.
Then, we need to consider the following questions:
- How do you characterize the cat? For instance, is the cat "adorable"?
- Are there particular characteristics you want for the cat? For example, the cat has "black fur and blue eyes."
- Where can the cat be found? For instance, "the cat is resting on a table."
- Are there any additional items the cat should have? For example, the cat is "wearing a red collar."
These questions collectively result in the following prompt formula:
<Adjective> Subject <Action or Accessory> <Location>- Subject: This is the central element you're addressing or emphasizing in your description. The subject can encompass a wide range of things, such as people, animals, objects, locations, plants, vehicles, food, weather conditions, natural occurrences, and abstract ideas. It's the only mandatory part of the formula, so you must include it.
- Adjective (Optional): You can use an adjective to add a descriptive word that helps paint a clearer picture of the subject. Adjectives can come before or after the Subject. This means you can use descriptive words to talk about the Subject, and you have the flexibility to place them before or after the main focus of your image.
- Action or Accessory (Optional): This is where you can include an action or an item that's related to the subject, but it's not required.
- Location (Optional): If you want to specify where the scene takes place, you can use this element. However, it's also optional.
Here are some examples.
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| A cute black cat with blue eyes wearing a red collar lying on a table |  |
|
| A cheerful young woman with long, curly hair holding a bouquet of colorful flowers in a garden |  |
|
| Enchanted forest with ancient trees |  |
|
| A curious cat exploring a mysterious cardboard box |  |
|
| Delicious pizza at a cozy Italian restaurant |  |
|
| Sparkling stars in the dark night sky |  |
|
| Tranquil waves lapping the sandy beach |  |
|
| Joyful children playing with colorful balloons in the park |  |
|
| Racing down the winding mountain road, a sleek sports car |  |
|
| An Owl |  |
|
| Busy city street, filled with rushing commuters |  |
|
Step 2 in Building a Prompt Formula: Artistic medium
"Artistic medium" refers to the materials and techniques an artist uses to create their work, like painting, photograph, or drawing. It's essentially the chosen form through which artistic expression takes shape. Ask yourself: do you want a photo, an oil painting, a graffiti, a cartoon, an illustration, or a 3D rendering?
Our can now be extended to:
<Artistic medium> <Adjective> Subject <Action or Accessory> <Location>Artistic mediums comes in many forms, and artists use different techniques. Here are some common ones:
-
Painting: Painting is the art of applying pigment, often in the form of paint, to a surface like canvas or paper to create visual artwork. This can include oil paintings, watercolors, acrylics, and more.
-
Drawing: Drawing is the creation of two-dimensional images or designs using various drawing techniques, such as pencil, charcoal, or ink drawings.
-
Illustration: An illustration is a visual representation often created with various artistic styles, such as drawing or digital art, to convey an idea, story, or concept.
-
3D-Render: A 3D render is a computer-generated image that creates a three-dimensional visual representation of objects, scenes, or environments using specialized software and techniques.
-
Photograph: A photograph is a visual representation of a real-life moment captured using a camera or similar device.
-
Digital Art: Digital art is a form of artistic expression created using digital tools and software, often on a computer or tablet, allowing for a wide range of styles and techniques such as pixel art, vector art and much more.
Examples
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| Graffiti of a cute black cat with blue eyes wearing a red collar lying on a table |  |
|
| Oil painting of a cheerful young woman with long, curly hair holding a bouquet of colorful flowers in a garden |  |
|
| illustration of a cute little fluffy alien |  |
|
| 3D render of a curious cat exploring a mysterious cardboard box |  |
|
| Black and white photograph of a bustling city street. |  |
|
| Watercolor painting of a delicious pizza at a cozy Italian restaurant |  |
|
| Photograph of an owl |  |
|
| Digital art representation of a busy city street, filled with rushing commuters |  |
|
| Ink drawing of a mountain landscape |  |
|
| Pixel art representation of a fairy |  |
|
Step 3 in Building a Prompt Formula: Artistic Style
In art, style is all about how an artist blends techniques, processes, and choices uniquely. It's the special way an image is represented. These styles are usually linked to certain periods, movements, or individual artists. Each artistic style has its own standout features, making it different from the rest and adding to the diverse and rich tapestry of the art world.
Our prompt formula can now be extended to:
<Artistic medium> <Adjective> Subject <Action or Accessory> <Location> <Artistic style>Artistic Styles for Diverse Art Forms
Some popular example for artistic styles are:
- Realism, Realistic style: A focus on mimicking the appearance and nature of the subject. It is recognizable by fine rendering, muted colors, intricate detail, and accurate fundamentals. Famous artists: Gustave Courbet, Jean-François Millet, Winslow Homer.
- Romanticism, Romantic style: It means capturing the subject in a favorable and exaggerated style.
Art in the romantic style is characterized by sharp contrast, bright highlights, strong colors, and a refined, almost
glossy finish. Famous artists: Caspar David Friedrich, Eugène Delacroix, William Blake. - Abstract art: A focus on capturing the subject in its simplest form. Think of abstract
as a scale rather than an absolute term. Essentially all artworks have some form of abstraction. Its characteristics are Reliance on shapes, colors, lines, and contrast. Famous artists: Wassily Kandinsky, Joan Miró, Piet Mondrian. - Impressionism: Captures the artist's impression of a subject. It is recognizable by light colors, weak shadows, energetic brushwork, broken color, and colorful accents. Famous artists: Claude Monet, Pierre-Auguste Renoir, Edgar Degas.
- Photorealism: It means capturing the subject with photographic accuracy. Photorealism is an art movement that seeks to replicate the look of real-life photographs through another medium. Charecteristics are meticulous drawing and detail. Famous artists: Chuck Close, Ralph Goings, Robert Bechtle.
- Surrealism: Surrealism is an art movement that explores the bizarre and dreamlike, creating images that defy reality and logic. It often features unexpected combinations, distorted forms, and a focus on the subconscious mind. Salvador Dalí and René Magritte are notable surrealistic artists.
- Pop Art: Pop art was a Western art movement that was popularized in The United States and Great Britain in the 1950s. Pop art – known for its bright, attention-catching colors – was used in advertising, merchandising, and graphic design during an era of rapid consumer growth. Famous artists: Andy Warhol, Roy Lichtenstein, Claes Oldenburg.
- Hyperrealism: Hyperrealism is an art style that creates extremely realistic artworks, often indistinguishable from high-resolution photographs. Artists meticulously capture details to achieve a heightened level of realism. Famous artists: Denis Peterson, Audrey Flack, Roberto Bernardi.
Yet, considering the significant variations within artistic styles, I've found that naming the artist is usually the wiser option. Instead of using "Abstract art" in your prompt, opt for an artist's name like "in the style of Kandinsky." The AI will then create an image mimicking the named artist's style. The artists' names don't have to exclusively be painters; they could include photographers like Ansel Adams or Helmut Newton, film directors such as Wes Anderson or Tim Burton, or even studios like Studio Ghibli, Disney, or Pixar.
You can combine and repeat elements, such as merging various artistic styles
Artistic Styles for Photography
Within the artistic medium of photography, there are distinct styles that deviate from the more general ones. Some prominent examples include:
- Color Splash: A technique where most of an image is in black and white, while a specific portion remains in color. This technique is used to draw attention to a particular subject or element within the photograph by isolating it in color against a monochromatic background. The term "color splash" is often associated with impactful and visually striking images where the selective use of color creates a focal point and emphasizes certain details.
- Tilt Shift: Tilt-shift is a photography technique that makes real scenes look like miniature models. It's achieved by adjusting the camera lens to control focus and perspective. This creates a shallow depth of field, making some parts blurry and giving the illusion of a tiny, toy-like world.
- Monochrome: Monochrome photography is a style where pictures are captured using shades of a single color, usually black and white or sepia. It creates a classic, timeless look by emphasizing light, shadow, and contrast without the distraction of multiple colors.
- High-Speed Photography: Captures rapidly occurring events, such as splashing liquids or fast-moving objects, by using high-speed shutter settings.
- Long Exposure: A photograph taken with a slow shutter speed, capturing motion blur in moving subjects or creating a smooth, flowing effect in water or clouds.
- Double Exposure: Double exposure is a photography technique where two different images are combined into a single frame. It creates a unique and artistic effect, blending elements from both images. This can result in surreal or ghostly visuals, as if two separate scenes are overlapping in one photograph.
Examples
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| Abstract art of a cute black cat with blue eyes wearing a red collar lying on a table |  |
|
| Oil painting of a cheerful young woman with long, curly hair holding a bouquet of colorful flowers in a garden, in the style of classicism |  |
|
| Hyperrealistic portrait of a beautiful woman wearing traditional dress qipao, floating in air with electricity surrounding, electricity in eyes, fire in background. Photograph |  |
|
| A lollipop in the style of Vincent van Gogh |  |
|
| A cute sparkly little unicorn, Banksy style |  |
|
| Donald Duck walking down the street, as a character in a Wes Anderson movie |  |
|
| A little fairy flying around in a beautiful tropical garden, in the styles of Alphonse Mucha and Picasso |  |
|
| Photo of construction site, workers, tilt shift effect |  |
|
| Color splash photo of a yellow tennis ball in the middle of an empty street |  |
|
| High speed photo of a strawberry falling into water |  |
|
| Photo of a busy city street with cars passing by, long exposure |  |
|
Step 4 in Building a Prompt Formula: Aesthetics
Aesthetics encompass the attractiveness and style of artwork, taking into account aspects such as perspective, lighting, contrast, and mood. It involves skillfully combining these elements to evoke emotions, convey messages, and offer a visually pleasing experience for the viewer.
Our prompt formula can now be extended to:
<Artistic medium> <Adjective> Subject <Action or Accessory> <Location> <Artistic style> <Aesthetics>Let's break down some of the aesthetic elements that can impact image generation.
Perspective
The perspective refers to the vantage point or viewpoint from which an image is observed or created. Here are some examples:
-
Linear Perspective: Linear perspective is a technique in art and photography that makes things look three-dimensional on a flat surface. It uses converging lines that meet at a point on the horizon, creating a sense of depth and distance.
-
Aerial Perspective (Atmospheric Perspective): Aerial perspective, also known as atmospheric perspective, is a technique in art and photography that shows depth by mimicking the way the atmosphere affects distant objects. It involves making distant objects appear lighter, less detailed, and bluer than closer ones. This technique creates a sense of distance and adds realism to the depiction of landscapes.
-
Fish-Eye view: Using a fish-eye lens to capture a scene, resulting in a distorted, spherical view that includes a wide field of view.
-
Worm's Eye View: Photographing or painting a subject from a low angle, as if seen from the ground, exaggerating the height and dominance of the subject.
-
Bird's Eye View: Capturing a scene from a high angle, as if seen from above, providing an overview and emphasizing patterns or relationships between elements.
Lighting
Lighting in images, whether in photography or paintings, refers to the deliberate use and manipulation of light to enhance visual impact. It shapes the mood, highlights details, and influences the overall atmosphere, contributing to the effectiveness and emotional resonance of the image. In paintings, artists use brushwork and color to simulate the effects of light, while in photography, the control of natural and artificial light sources is essential for achieving the desired aesthetic. These are some examples:
-
Natural Light / Ambient Light: Illuminating a subject using ambient sunlight, providing a soft and natural look, often ideal for portraits taken outdoors.
-
Backlighting: Positioning the light source behind the subject, creating a halo effect and emphasizing contours, often used for creating silhouettes.
-
Soft Light: Achieving a gentle and diffused illumination, commonly from overcast skies or through a diffuser, resulting in soft shadows and even tones.
-
Artificial Light: Using a direct and intense light source, such as the a neon or a spotlight, to create strong, defined shadows and highlights.
Contrast
Contrast in images is the degree of difference between elements, such as light and dark, colors, or textures. It adds visual interest, emphasizes focal points, and creates a dynamic composition. Here are some examples:
-
High Contrast: A photograph with stark black and white elements, creating strong visual impact and emphasizing sharp distinctions.
-
Low Contrast: A painting with muted tones and subtle transitions between light and shadow, creating a softer and more subdued visual effect.
Mood
The mood of an image refers to the overall emotional atmosphere it conveys. It can be described as:
-
Gloomy: Eliciting a sense of darkness, sadness, or somberness.
-
Bright: Radiating a cheerful and optimistic ambiance.
-
Vibrant: Bursting with lively and intense colors, creating energy.
-
Dynamic: Full of movement and action, conveying a sense of vitality.
-
Serene: Calm, peaceful, and tranquil, inducing a sense of relaxation.
-
Mysterious: Creating a sense of intrigue and curiosity, often with obscured or hidden elements.
-
Ethereal: Evoking a delicate, otherworldly, or dreamlike feeling.
-
Nostalgic: Stirring emotions associated with fond memories or the past.
-
Tense: Generating a feeling of unease, anticipation, or suspense.
-
Whimsical: Playful, lighthearted, and fanciful, often with a touch of fantasy.
-
Melancholic: Reflecting a deep, thoughtful sadness or pensiveness.
-
Empowering: Instilling a sense of strength, confidence, and inspiration.
-
Intimate: Creating a personal and close connection, often in portraiture or close-up shots.
-
Surreal: Exhibiting a dreamlike or fantastical quality that defies reality.
Examples
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| Photo of a busy city street with cars passing by, long exposure, birds eye view |  |
|
| Monkeys in the jungle, fish-eye perspective |  |
|
| Photo of an old man, soft lighting |  |
|
| photo of a cute cat on an empty floor, illuminated by a spotlight |  |
|
| A daisy, high contrast |  |
|
| A bar filled with people, neon light, gloomy atmosphere |  |
|
| A cute pink unicorn standing in a field of roses, soft lighting, whimsical, Studio Ghibli |  |
|
Step 5 in Building a Prompt Formula: Image Quality
Image quality pertains to the level of detail in a picture. Like many AI image generation tools, including Stable Diffusion, there are distinct parameters that impact quality. Specifically, you can determine the resolution and size of an image by adjusting them. Yet, based on my experience, there are specific "magic words" associated with image quality that can enhance what the AI produces. While these won't affect image size or resolution, they will enhance the visual appeal.
Our prompt formula now extends to:
<Artistic medium> <Adjective> Subject <Action or Accessory> <Location> <Artistic style> <Aesthetics> <Image quality>Some common magic words for image quality are:
-
HD (High Definition): Standard resolution for clear and detailed images, often 1920 x 1080 pixels.
-
HDR (High Dynamic Range): Enhances contrast and color range for more realistic and vibrant images.
-
UHD (Ultra High Definition): Higher resolution than HD, typically at least 3840 x 2160 pixels.
-
4K: A resolution standard with around 4000 pixels width, offering exceptionally sharp and detailed images.
-
Highly Detailed: Indicates an image with an abundance of intricate elements and clarity.
-
8K: Extremely high resolution, around 8000 pixels width, providing exceptional clarity and detail.
Example: A Landscape
To demonstrate how image quality influences the picture, I generated two images under identical conditions and using the same seed. The only difference was the incorporation of magic words.
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| A landscape |  |
|
| A landscape, 4k, 8k, UHD, highly detailed |  |
|
Examples
| Prompt | Image | Prompt Breakdown |
|---|---|---|
| Portrait of Morgan Freeman, studio lighting, UHD, highly detailed, HDR |  |
|
| The pyramids of Giza, 4k, 8k |  |
|
| A female cyborg in space, ultra detailed, 8k, UHD, HDR, HD |  |
|
Conclusion
In this first part of our tutorial, we've covered the essential components that make up an effective prompt formula. We've explored the key elements that help Stable Diffusion generate high-quality images based on your input.
By understanding and applying these principles, you'll be well-equipped to write better prompts for Stable Diffusion. However, as you experiment with different prompts and techniques, you may find that achieving specific results requires more than just a solid prompt formula.
That's where we're headed in Part 2: Advanced Techniques for Optimizing Your Prompts. In the next installment of this tutorial, we'll delve into advanced methods such as using weights, negative prompts, and more to take your image generation skills to the next level.
