AI Image Generation Explained

I often recall an experience from my childhood when I would watch a TV set that was not connected to any input. The screen displayed a chaotic mix of dots, accompanied by a hissing sound. Despite the randomness, I felt as though I could see images—like flocks of geese flying over a lake—emerging from the chaos, though I never understood why.

Now, with the help of AI, I am starting to piece together how this process functions. But how does AI actually create images?

To grasp this, we must first recognize that AI struggles with generating noise effectively.

Here's what occurs when AI attempts to produce noise:

Examining the Noise Generated by LeonardoAI

The noise generated by AI can be surprisingly decent. However, if you're familiar with image compression, you'll notice that it leads to the emergence of visual artifacts—patterns that appear due to the compression process aimed at reducing file sizes. Yet, since AI isn't focused on file size, why does it produce these artifacts?

AI is heavily influenced by bias, repeating similar patterns based on statistical likelihood. In fact, it may be even more susceptible to bias than humans, lacking the ability to evaluate its performance against real-world data. While some humans can think independently, AI merely absorbs information and replicates it in a statistical fashion.

This results in a patchwork of patterns scattered throughout the images.

Randomness in AI

Consider the challenge of selecting random numbers and then plotting them into a histogram. Humans are notoriously poor at generating random sequences, and the same applies to AI. The absence of a proper random number generator means that the numbers the AI generates will likely follow a predictable pattern.

White noise, characterized by a complete lack of repeating patterns, is akin to a series of perfectly random numbers. It's impossible to predict what follows in a truly random sequence, but algorithms often fall into the trap of creating patterns.

AI, unfortunately, does not know how to approximate white noise effectively. Instead, it merely shifts pixels around in an attempt to mimic noise, resulting in a visual output that lacks true randomness.

AI’s Grouping Mechanism

My hypothesis is that AI generates clusters of pixels to fill contextual spaces, similar to how snow forms in clouds where water molecules stick together. In its pursuit of making sense of noise, AI clumps the pixels together. During the upscaling process, it fails to retain the original prompt, merely rearranging the pixels.

Experimenting with Prompts

By placing this generated image into a real-time canvas, we can manipulate the output. The results are visually striking, demonstrating that the AI can synthesize something coherent from seemingly random data.

When we adjust the creativity strength, we find that it can generate images from nothingness. If a prompt is added, it can create something as vivid as a forest from a simple pile of noise.

As we continue adjusting the creativity settings, we start seeing urban landscapes intertwined with circuits, revealing the AI's unpredictable nature.

The Mechanism Behind AI Grouping

We've delved into how AI processes TV static and groups pixels. But what guides its decision on how to cluster them?

Terrain generation serves as an analogy here: it uses noise as input to create coherent landscapes by "smoothing" the noise and filling in details like marshes or mountains. AI likely employs a similar methodology, utilizing multiple variables of semi-random data across different frequencies to construct images.

The Role of Prompts

When a prompt is introduced, the LLM searches for relevant keywords and organizes the pixels that align with those keywords within the image. It essentially assigns locations for specific elements, which may lead to alterations in proportions and textures, even if those aspects were not explicitly defined in the prompt.

The surprising reality is that LeonardoAI will generate an image as long as the prompt passes through its filters, regardless of whether it makes logical sense.

Identifying Viable and Non-Viable Words

A "viable" word produces images consistently, drawing from a wide array of sources, including tags and descriptions attached to images in its training data. Conversely, "non-viable" words are those that lack consistent keys or analysis data, often resulting in nothing or blank outputs when used in prompts.

For example, a nonsensical string like "Copyn4rgfvycMLrHusfHO" has a negligible chance of appearing in training data, leading to unpredictable results.

In conclusion, while AI can generate images from both viable and non-viable prompts, the underlying mechanisms reveal a lack of true understanding and analysis.

Try these non-viable prompts:

&

$^ LeonardoAI

© 2023 Corbbin Goldsmith. All Rights Reserved.

This work, including all text, images, and other content, is the sole property of Corbbin Goldsmith. Unauthorized use, reproduction, modification, distribution, or incorporation into any AI/machine learning applications, databases, or any other digital or physical format, without prior written consent from Corbbin Goldsmith, is strictly prohibited.

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go: Be sure to clap and follow the writer ️👏️️ Follow us: X | LinkedIn | YouTube | Discord | Newsletter Visit our other platforms: Stackademic | CoFeed | Venture More content at PlainEnglish.io