Python and Pillow: How to Code Something DALL-E Cannot Do


Step 1

To understand the role of random variables in my code, consider the first step in our image creation process: forming a portrait-style rectangle, characterized by its greater height than its width. This rectangle, although seemingly simple, is an embodiment of random variables in action.

A rectangle can be dissected into four principal elements: a starting x and y coordinate, and an ending x and y coordinate. Now, these points, when chosen from a specific distribution, transform into random variables. But how do we decide the range of these points, or more specifically, the distribution they come from? The answer lies in one of the most common and crucial distributions in statistics: The Normal Distribution.

Defined by two parameters — the mean (μ) and standard deviation (σ), the Normal Distribution plays a pivotal role in our image generation process. The mean, μ, signifies the center of the distribution, thus acting as the point around which the values of our random variables gravitate. The standard deviation, σ, quantifies the degree of dispersion in the distribution. It decides the range of values the random variables could potentially take. In essence, a larger standard deviation would result in greater diversity in the images created.

import torch
canvas_height = 1000
canvas_width = 1500

#loop to show different values
for i in range(5):
#create normal distribution to sample from
start_y_dist = torch.distributions.Normal(canvas_height * 0.8, canvas_height * 0.05)
#sample from distribution
start_y = int(start_y_dist.sample())

#create normal distribution to sample height from
height_dist = torch.distributions.Normal(canvas_height * 0.2, canvas_height * 0.05)

height = int(height_dist.sample())
end_y = start_y + height

#start_x is fixed because of this being centered
start_x = canvas_width // 2
width_dist = torch.distributions.Normal(height * 0.5, height * 0.1)

width = int(width_dist.sample())
end_x = start_x + width

print(f"start_x: {start_x}, end_x: {end_x}, start_y: {start_y}, end_y: {end_y}, width: {width}, height: {height}")

start_x: 750, end_x: 942, start_y: 795, end_y: 1101, width: 192, height: 306
start_x: 750, end_x: 835, start_y: 838, end_y: 1023, width: 85, height: 185
start_x: 750, end_x: 871, start_y: 861, end_y: 1061, width: 121, height: 200
start_x: 750, end_x: 863, start_y: 728, end_y: 962, width: 113, height: 234
start_x: 750, end_x: 853, start_y: 812, end_y: 986, width: 103, height: 174

Sampling a square looks very similar we only have to sample the height or the width as they are the same. Sampling a circle is even easier as we only have to sample the radius.

Drawing a rectangle in Python is a straightforward process, especially when utilizing the Pillow library. Here’s how you can do it:

from PIL import Image, ImageDraw

# Create a new image with white background

# Loop to draw rectangles
for i in range(5):
img = Image.new('RGB', (canvas_width, canvas_height), 'white')

draw = ImageDraw.Draw(img)

# Creating normal distributions to sample from
start_y_dist = torch.distributions.Normal(canvas_height * 0.8, canvas_height * 0.05)
start_y = int(start_y_dist.sample())

height_dist = torch.distributions.Normal(canvas_height * 0.2, canvas_height * 0.05)
height = int(height_dist.sample())
end_y = start_y + height

start_x = canvas_width // 2
width_dist = torch.distributions.Normal(height * 0.5, height * 0.1)
width = int(width_dist.sample())
end_x = start_x + width

# Drawing the rectangle
draw.rectangle([(start_x, start_y), (end_x, end_y)], outline='black')

img.show()

Step 2

In the context of the vertical lines in these images, we consider three random variables, namely:

  1. The beginning y-coordinate of the line (y_start)
  2. The ending y-coordinate of the line (y_end)
  3. The x-coordinate of the line (x)

Since we are dealing with vertical lines, only one x-coordinate needs to be sampled for each line. The width of the line is constant, controlled by the size of the canvas.

Some additional logic was needed to ensure the lines didn’t intersect. To do this basically, we need to keep account of the image as a grid and keep track of the occupied positions. Let’s disregard that for the sake of simplicity.

Here’s an example of how this looks like in Python.

import torch
from PIL import Image, ImageDraw

# Setting the size of the canvas
canvas_size = 1000
# Number of lines
num_lines = 10
# Create distributions for start and end y-coordinates and x-coordinate
y_start_distribution = torch.distributions.Normal(canvas_size / 2, canvas_size / 4)
y_end_distribution = torch.distributions.Normal(canvas_size / 2, canvas_size / 4)
x_distribution = torch.distributions.Normal(canvas_size / 2, canvas_size / 4)
# Sample from the distributions for each line
y_start_points = y_start_distribution.sample((num_lines,))
y_end_points = y_end_distribution.sample((num_lines,))
x_points = x_distribution.sample((num_lines,))
# Create a white canvas
image = Image.new('RGB', (canvas_size, canvas_size), 'white')
draw = ImageDraw.Draw(image)
# Draw the lines
for i in range(num_lines):
draw.line([(x_points[i], y_start_points[i]), (x_points[i], y_end_points[i])], fill='black')
# Display the image
image.show()

This however only gives you lines. Another part of the cluster is the circles at the end of the lines, I called these adjacent circles. Random variables also determine their process. First, the fact that there will be an adjacent circle is sampled from a Bernoulli distribution, and the position (left, middle, right) of the shape is sampled from a uniform distribution.

A circle can be defined entirely by a single parameter: its radius. We can consider the length of a line as a condition that influences the radius of the circle. This forms a conditional probability model where the radius (R) of the circle is dependent on the length of the line (L). We use a conditional Gaussian distribution. The mean (μ) of this distribution is a function of the square root of the line length, while the standard deviation (σ) is a constant.

We initially suggest that the radius R, given the line length L, follows a normal distribution. This is denoted as R | L ~ N(μ(L), σ²), where N is the normal (Gaussian) distribution and σ is the standard deviation.

However, this has a small problem: the normal distribution includes the possibility of sampling a negative value. This outcome is not physically possible in our scenario, as a radius cannot be negative.

To circumvent this issue, we can use the half-normal distribution. This distribution, much like the normal distribution, is defined by a scale parameter σ, but crucially, it is constrained to non-negative values. The radius given the line length follows a half-normal distribution: R | L ~ HN(σ), where HN denotes the half-normal distribution. This way, σ is determined by the desired mean as σ = √(2L) / √(2/π), ensuring that all sampled radii are non-negative and that the mean of the distribution is √(2L)

from PIL import Image, ImageDraw
import numpy as np
import torch
# Define your line length
L = 3000

# Calculate the desired mean for the half-normal distribution
mu = np.sqrt(L * 2)

# Calculate the scale parameter that gives the desired mean
scale = mu / np.sqrt(2 / np.pi)

# Create a half-normal distribution with the calculated scale parameter
dist = torch.distributions.HalfNormal(scale / 3)

# Sample and draw multiple circles
for _ in range(10):
# Create a new image with white background
img_size = (2000, 2000)
img = Image.new('RGB', img_size, (255, 255, 255))
draw = ImageDraw.Draw(img)

# Define the center of the circles
start_x = img_size[0] // 2
start_y = img_size[1] // 2
# Sample a radius from the distribution
r = int(dist.sample())

print(f"Sampled radius: {r}")

# Define the bounding box for the circle
bbox = [start_x - r, start_y - r, start_x + r, start_y + r]

# Draw the circle onto the image
draw.ellipse(bbox, outline ='black',fill=(0, 0, 0))

# Display the image
img.show()

Step 3

Step 3 in our process is a combination of elements from Steps 1 and 2. In Step 1, we tackled the task of sampling and drawing rectangles in set positions. In Step 2, we learned how to use the normal distribution to draw lines on a portion of your canvas. Additionally, we acquired knowledge on how to sample and draw circles.

As we transition to Step 3, we are going to repurpose the techniques from the previous steps. Our aim is to distribute squares and circles harmoniously around the lines that we sampled earlier. The normal distribution, will once again come in handy for this task.

We will re-use the parameters used to create clusters of lines. However, to enhance the visual appeal and avoid overlaps, we introduce some noise to the mean (mu) and standard deviation values.

In this step, instead of positioning lines, our task is to place sampled rectangles and circles. I encourage you to play around with these techniques and try if you can add circles and rectangles to your cluster of lines.



Source link

Leave a Comment