Tag: video synthesis

I kind of hit a wall of the Organelle. I feel like in order to advance my skills I have a bit of a hurdle between where my programming skills are at, and where they would need to be to do something more advanced that the recent experiments I have completed. Accordingly for this month I decided to shift my focus to the EYESY. Last month I made significant progress in understanding Python programming for the EYESY, and that allowed me to come up with five ideas for algorithms in short order. The music for all five mini-experiments comes from PureData algorithms I will be using for my next major album. All five of these algorithms are somewhat derived from my work on my last album.

The two realizations that allowed me to make significant progress on EYESY programming is that Python is super picky about spaces, tabs, and indentations, and that while the EYESY usually gives little to no feedback when a program has an error in it, you can use an online Python compiler to help figure out where your error is (I had mentioned the latter in last month’s experiment). Individuals who have a decent amount of programming experience may scoff at the simplicity of the programs that follow, but for me it is a decent starting place, and it is also satisfying to me to see how such simple algorithms can generate such gratifying results.

Random Lines is a patch I wrote that draws 96 lines. In order to do this in an automated fashion, I have to use a loop, in this case I use for i in range(96):. The five lines that follow are all executed in this loop. Before the loop commences, we choose the color using knob 4 and the background color using knob 5. I use knob 3 to set the line width, but I scale it by multiplying the knob’s value, which will be between 0 and 1, by 10, and adding 1, as line widths cannot have a value of 0. I also have to cast the value as an integer. I set an x offset value using knob 1. Multiplying by 640 and then subtracting 320 will yield a result between -320 and 320. Likewise, a y offset value is set using knob 2, and the scaling results in a value between -180 and 180.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	color = etc.color_picker(etc.knob4)
	linewidth = int (1+ (etc.knob3)*10)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
   	for i in range(96):
		x1 = int (640 + (xoffset+(etc.audio_in[i])/50))
		y1 = int (360 + (yoffset+(etc.audio_in[(i+1)])/90))
		x2 = int (640 + (xoffset+(etc.audio_in[(i+2)])/50))
		y2 = int (360 + (yoffset+(etc.audio_in[(i+3)])/90))
		pygame.draw.line(screen, color, (x1,y1), (x2, y2), linewidth)

Within the loop, I set two coordinates. The EYESY keeps track of the last hundred samples using etc.audio_in[]. Since these values use sixteen bit sound, and sound has peaks (represented by positive numbers) and valleys (represented by negative numbers), these values range between -32,768 and 32,787. I scale these values by dividing by 50 for x coordinates. This will scale the values to somewhere between -655 and 655. For y coordinates I divide by 90, which yields values between -364 and 364.

In both cases, I add these values to the corresponding offset value, and add the value that would normally, without the offsets, place the point in the middle of the screen, namely 640 (X) and 360 (Y). A negative value for the xoffset or the scaled etc.audio_in value would move that point to the left, while a positive value would move it to the right. Likewise, a negative value for the yoffset or the scale etc.audio_in value would move the point up, while a positive value would move it down.

Since subsequent index numbers are used for each coordinate (that is i, i+1, i+2, and i+3), this results in a bunch of interconnected lines. For instance when i=0, the end point of the drawn line (X2, Y2) would become the starting point when i=2. Thus, the lines are not fully random, as they are all interconnected, yielding a tangled mass of lines.

Random Concentric Circles uses a similar methodology. Again, knob five is use to control the background color, while knobs 1 and 2 are again scaled to provide an X and Y offset. The line width is shifted to knob 4. For this algorithm the loop happens 94 times. The X and Y value for the center of the circles is determined the same way as was done in Random Lines. However, we now need a radius and we need a color for each circle.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	linewidth = int (1+(etc.knob4)*9)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
	for i in range(94):
		x = int (640 + xoffset+(etc.audio_in[i])/50)
		y = int (360 + yoffset+(etc.audio_in[(i+1)])/90)
		radius = int (11+(abs (etc.knob3 * (etc.audio_in[(i+2)])/90)))
		r = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+5)]/33000)))
		thecolor=pygame.Color(r,g,b)
		pygame.draw.circle(screen, thecolor, (x,y), radius, linewidth)

We have knob 3 available to help control the radius of the circle. Here I multiply knob 3 by a scaled version of etc.audio_in[(i+2)]. I scale it by dividing by 90 so that the largest possible circle will mostly fit on the screen if it is centered in the screen. Notice that when we multiply knob 3 by etc.audio_in, there’s a 50% chance that the result will be a negative number. Negative values for radii don’t make any sense, so I take the absolute value of this outcome using abs. I also add this value to 11, as a radius of 0 makes no sense, and a radius of less than 10, as having a line width that is larger than the radius will cause an error.

For this algorithm I take a set forward by giving each circle its own color. In order to do this I have to set the value for red, green, and blue separately, and then combine them together using pygame.Color(). For each of the three values (red, green, and blue) I divide a value of etc. audio_in by 33000, which will yield a value between 0 and 1 (more or less), and then multiply this by 100. I could have done the same thing by simply dividing etc.audio_in by 330, however, at the time this process made the most sense to me. Again, this process could result in a negative number and / or a fractional value, so I cast the result as an integer after getting its absolute value.

Colored Rectangles has a different structure than the previous two examples. Rather than have all the objects cluster around a center point I wanted to create an algorithm that spaces all of the objects out evenly throughout the screen in a grid like pattern. I do this using an eight by eight grid of 64 rectangles. I accomplish the spacing using modulus mathematics as well as integer division. The X value is obtained by multiplying 160 times i%8. In a similar vein, the Y values is set to 90 times i//8. Integer division in Python is accomplished through the use of two slashes. Using this operator will return the integer value of a division problem, omitting the fractional portion. Both the X and the Y values have an additional offset value. The X is offset by (i//8)*(80*etc.knob1), so this offset increases as knob 1 is turned up, with a maximum offset of 80 pixels per row. The value i//8 essentially multiplies that offset by the row number. That is the rows shift further towards the right.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
 	etc.color_picker_bg(etc.knob5)
   	for i in range(64):
		x=(i%8)*160+(i//8)*(80*etc.knob1)
		y=(i//8)*90+(i%8)*(45*etc.knob2)
		thewidth=int (abs (160 * (etc.knob3) * (etc.audio_in[(i)]/33000)))
		theheight=int (abs (90 * (etc.knob4) * (etc.audio_in[(i+1)]/33000)))
		therectangle=pygame.Rect(x,y,thewidth,theheight)
		r = int (abs (100 * (etc.audio_in[(i+2)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		thecolor=pygame.Color(r,g,b)
		pygame.draw.rect(screen, thecolor, therectangle, 0)

Likewise, the Y offset is determined by (i%8)*(45*etc.knob2). As the value of knob 2 increases, the offset moves towards a maximum value of 45. However, as the columns shift to the right, those offsets compound due to the fact that they are multiplied by (i%8).

A rectangle can be defined in pygame by passing an X value, a Y value, width, and height to pygame.Rect. Thus, the next step is to set the width and height of the rectangle. In both cases, I set the maximum value to 160 (for width) and 90 (for height). However, I scaled them both by multiplying by a knob value (knob 3 for width and knob 4 for height). These values are also scaled by an audio value divided by 33,000. Since negative values are possible from audio values, and negative widths and heights don’t make much sense, I took the absolute value of each. If I were to rewrite this algorithm (perhaps I will), I would set a minimum value for width and height such that widths and heights of 0 were not possible.

I set the color of each rectangle using the same method as I did in Random Concentric Circles. In order to draw the rectangle you pass the screen, the color, the rectangle (as defined by pygame.Rect), as well as the line width to pygame.draw.rect. Using a line width of 0 means that the rectangle will be filled in with color.

Random Rectangles is a combination of Colored Rectangles and Random Lines. Rather than use pygame’s Rect object to draw rectangles on the screen, I use individual lines to draw the rectangles (technically speaking they are quadrilaterals). Knob 4 is used here to set the foreground color, knob 5 is used here to set the background color, knob 3 is used to set the linewidth.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	color = etc.color_picker(etc.knob4)
 	etc.color_picker_bg(etc.knob5)
	linewidth = int (1+ (etc.knob3)*10)
   	for i in range(64):
		x=(i%8)*160+(i//8)*(80*etc.knob1)+(40*(etc.audio_in[(i)]/33000))
		y=(i//8)*90+(i%8)*(45*etc.knob2)+(20*(etc.audio_in[(i+1)]/33000))
		x1=x+160+(40*(etc.audio_in[(i+2)]/33000))
		y1=y+(20*(etc.audio_in[(i+3)]/33000))
		x2=x1+(40*(etc.audio_in[(i+4)]/33000))
		y2=y1+90+(20*(etc.audio_in[(i+5)]/33000))
		x3=x+(40*(etc.audio_in[(i+6)]/33000))
		y3=y+90+(20*(etc.audio_in[(i+7)]/33000))
		pygame.draw.line(screen, color, (x,y), (x1, y1), linewidth)
		pygame.draw.line(screen, color, (x1,y1), (x2, y2), linewidth)
		pygame.draw.line(screen, color, (x2,y2), (x3, y3), linewidth)
		pygame.draw.line(screen, color, (x3,y3), (x, y), linewidth)

Within the loop, I use a similar method of setting the initial X and Y coordinates. That being said, I separate out the use of knobs and the use of audio input. In the case of the X coordinated, I use (i//8)*(80*etc.knob1) to control the amount of x offset for each row, with a maximum offset of 80. The audio input then offsets this value further using (40*etc.audio_in[(i+2)]/33000). This moves the x value by a value of plus or minus 40 (remember that audio values can be negative. Likewise, knob 2 offsets the Y value for every row by a maximum of 45, and the audio input further offsets this value by plus or minus 20.

Since it takes four points to define a quadrilateral, we need three more points, which we will call (x1, y1), (x2, y2), and (x3, y3). These are all interrelated. The value of X is used to define X1 and X3, while X2 is based off of X1. Likewise, the value of Y helps define Y1 and Y3, with Y2 being based off of Y1. In the case X1 and X2 (which is based on X1) we add 160 to X, giving a default width, but these values are again scaled by etc.audio_in. Similarly, we add 90 to Y1 and Y3 to determine a default height of the quadrilaterals, but again, all points are further offset by etc.audio_in, resulting in quadrilaterals, rather than rectangles with parallel sides. If I were to revise this algorithm I would likely make each quadrilateral a different color.

Frankly, I was not as pleased with the results of Colored Rectangles and Random Rectangles, so I decided to go back create an algorithm that was an amalgam of Random Lines and Random Concentric Circles, namely Random Radii. This program creates 95 lines, all of which have the same starting point, but different end points. Knob 5 sets the background color, while knob 4 sets the line width.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	linewidth = int (1+ (etc.knob4)*10)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
	X=int (640+xoffset)
	Y=int (360+yoffset)
   	for i in range(95):
		r = int (abs (100 * (etc.audio_in[(i+2)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		thecolor=pygame.Color(r,g,b)
		x2 = int (640 + (xoffset+etc.knob3*(etc.audio_in[(i)])/50))
		y2 = int (360 + (yoffset+etc.knob3*(etc.audio_in[(i+1)])/90))
		pygame.draw.line(screen, thecolor, (X,Y), (x2, y2), linewidth)

Knob 1 & 2 are used for X and Y offsets (respectively) of the center point. Using (640*etc.knob1)-320 means that the X value will move plus or minus 320. Similarly, (360*etc.knob2)-180 permits the Y value to move up or down by 180. As is the case with Random Concentric Circles, the color of each line is defined by etc.audio_in. Knob 3 is used to scale the end point of each line. In the case of both the X and Y values, we start from the center of the screen (640, 360) add the offset defined by knobs 1 or 2, and then use knob 3 to scale a value derived from etc.audio_in. Since audio values can be positive or negative, these radii shoot outward from the offset center point in all directions.

As suggested earlier, I am very gratified with these results. Despite the simplicity of the Python code, the results are mostly dynamic and compelling (although I am somewhat less than thrilled with Colored Rectangles and Random Rectangles). The user community for the EYESY is much smaller than that of the Organelle. The EYESY user’s forum has only 13% the activity of that of the Organelle. I seem to have inherited being the administrator of the ETC / EYESY Video Synthesizer from Critter&Guitari group on Facebook. Likewise, I am the author of the seven most recent patches for the EYESY on patchstorage. Thus, this month’s experiment sees me shooting to the top to become the EYESY’s most active developer. The start of the semester has been very busy for me. I am somewhat doubting that I will be coming up with an Organelle patch next month, but I equally suspect that I will continue to develop more algorithms of the EYESY.