Digital Innovation Grant: July 29th Update

It was a fairly short jump from my previous experiment to this one. For the music video for 21AC3627AB4, I focused on using constant, smooth changes of selected parameters to manipulate video footage using the Sleepy Circuits Hypno. For this experiment, I wanted to focus on abrupt parameter changes in order to create a sense of visual rhythm. In order to achieve this, there were very few changes I had to make to the Pure Data algorithm I used to control the Hypno.

The overall algorithm for creating the video for 5B4C31 is nearly identical to the algorithm for creating the video for 21AC3627AB4. The right hand branch off of the counter contains a mod (%) four object followed by a sel 0 object. This causes the subroutines below to execute once every four sixteenth notes, or to put it another way, once every beat.

As was the case with the previous experiment, each of the makeautomation subroutines is largely the same, so we’ll look at the first one. This version of makeautomation is far simpler than the one used in the previous experiment. This subroutine is designed to send controller values for the first eight parameters used in the experiment. Each column is largely identical to the rest of the columns. The spigot statement controls whether or not a bang is sent to the random 128 objects below. When the value stored in parameters is zero the bang does not occur, when it is one the bang happens. The random 128 object outputs a value between 0 and 127 inclusive. This takes advantage of the full spectrum available within MIDI specifications. This value is then sent to the Hypno using a ctlout object.

Screenshot

The video input used for this experiment was the video for the next two tracks on Point Nemo, namely 21AC3627AB4 and 87ED21857E5. The video for 21AC3627AB4 is more chaotic in nature than that of 5B4C31. With the abrupt changes, I had though that the latter video would be more chaotic. However, it is likely the tempo change (the former is at 180 beats per minute, while the latter is at 60 beats per minute), that is one of the main contributors to the somewhat calmer flow of 5B4C31.

The sudden changes and the slower tempo require far less MIDI data to be sent to the Hypno, which resulted in far fewer glitches in terms of settings changes that were not specified by the algorithm. This was definitely a big problem with the previous experiment. In fact, there was only one time where one of the oscillator changed shapes, which I manually put back on the video input shape.

I purposefully created the videos for Point Nemo in reverse order, as I wanted the videos to become more simple as the album progresses, having the source video, late afternoon sunlight glistening on waves, gradually reveal itself over a one hour experience. Given that I know what the source video is, I can occasionally see glimpses of it in the first two movements. That being said, there is a considerable step down in complexity between the second and third movement.

Another thing I discovered in this experiment was that many of the visual beats came out quite dark. This can be due to any number of factors, including bad settings for color, for gain, for frequency (zoom) and the like. In order to deal with this, I recorded twice as much video as I needed to, and then simply edited out the dark sections. This required a couple of hours of work, which while somewhat defeats the goal of creating a music video in real time, a couple of hours to create a music video is a fairly easy work load. For future endeavors, I could limit the range of values to avoid such poor settings, or I could come up with algorithms to reject certain results.

In fact, this experiment successful enough that I have devised a rough plan for creating music videos for my forthcoming album, Monstrum Pacificum Bellicosum, which is due out at the end of 2026. Accordingly, I intend to start working on the videos relatively soon. The first part will likely be to find footage to manipulate. The current plan is to find footage from public domain and silent films.

I am fairly pleased with the numbers of streams of the Point Nemo videos on YouTube (5B4C31: 15, 21AC3627AB4: 90, 87ED21857E5: 213, and 364F234F6231: 118), particularly considering that the first video has only been live for less than 24 hours. While those statistics may not seem earth shattering, with the exception of the stats for 5B4C31, these surpass the streams those pieces have had through bandcamp and distrokid, so the videos are definitely helping with the visibility / distribution of the music.

I am considering a public showing the Point Nemo at Stonehill College, and I definitely plan on doing a live concert using the Hypno. However, my elder cat has a lot of health care issues, and so I’m trying to figure out whether it would be better to schedule such events in the Fall or Spring semesters. I am also considering printing out selected individual frames from the Point Nemo videos, and treating them as 2D art pieces. Since I do plan on using the Hypno to create videos for Monstrum Pacificum Bellicosum, I plan on doing studio reports for my work on them, and treating them as part of my Digital Innovation Grant. That being said, it may be several months until I get to the point where I am ready to report on my work for Monstrum Pacificum Bellicosum.

Digital Innovation Grant: July 25th Update

The good news is that the Hypno features robust MIDI implementation. The bad news is that it may not be well implemented. The MIDI specification allows for 127 different continuous controllers, which could do things like changing the volume or panning of a sound. The Hypno makes use of 50 of these, which is far more than most MIDI devices that I am used to. In addition, there are 10 MIDI notes that are used to act as triggers for the Hypno.

I came up with an algorithm in Pure Data that makes use of 35 of the controller values, and one of the triggers. This is where the bad news comes in. When I run the algorithm there are unpredictable results. Namely two of the triggers I do not use are the triggers that change the shape of oscillators A and B. When I run the algorithm those oscillator shapes often change without being directed to do so.

I spent a lot of time troubleshooting the algorithm, including recording the MIDI data in Logic Pro (see below). The MIDI data is coming out of Pure Data exactly as expected. When I cut the algorithm back to sending just one set of values (for instance, the note trigger or one of the controller values), the Hypno responds as predicted, however, sometimes just adding in a second controller, the Hypno begins to change oscillator shapes without being prompted to. It may also be changing other values, that I am simply not able to track. My working theory here is that the Hypno cannot handle a rapid MIDI input of several controller values.

In fact, it was this problem, which I wasn’t able to solve in March that caused me to back burner my work on this project until the summer. Unfortunately the extra time and brain space afforded me by the summer did not yield different results. So, I decided to move forward with the algorithm as it is. My intent with this project was to have both oscillators using the video input shape from stored videos on a USB thumb drive (in this case, video from the previous two experiments, that is video from the final two tracks of Point Nemo). So, in practice, while I was recording the video output from the Hypno, whenever either oscillator shape changed, I changed it back to video input as quickly as I could. Accordingly, this resulted in a bit more chaos to an already chaotic video.

Alright, so let’s look at the algorithm in Pure Data. We see at the bottom left the part of the algorithm that sends the MIDI note out (it ends with a noteout object). In the upper left with see the initialize subroutine, which includes translating the tempo (180 bpm) into an amount of milliseconds. It also includes the initialization of an array called parameters. Finally, it also sets the MIDI channel to channel 16 (the Hypno is locked into channel 16, and it cannot be set to other MIDI channels). In the center of the screen, we see the primary algorithm. It includes a metronome, and a counter right below that. Then beneath the number object below the counter we see three different branches, one ending in a send object, the second ending in two subroutines labeled changevideo, and the final ending in six different subroutines labeled makeautomation.

We have seen various versions of all of these subroutines before. The first, which terminates in [send thenote] simply sends MIDI note number 123 to the Hypno once every four measures (16 sixteenth notes multiplied by four measures equals 64). This is the note number that toggles the feedback mode of the Hypno. The changevideo subroutines occur under a % 128 object. This causes these subroutines to execute once every eight measures (two times 64). The subroutine changevideo2 is similar as changevideo1, so let’s look at changevideo1.

The bulk of this subroutine is 30 instances of a random 2 object followed by a tabwrite parameter object. What this means is that for each of 30 parameters, which will map onto controller values sent to the Hypno, there is a fifty percent chance for that parameter to be set to either change or not change once every eight measures. The changevideo2 subroutine simlply contains the remaining five instances of the parameters.

On the left we see two random 2 objects followed by a * 100 object. In each case, this will result either in the number 0 or the number 100 being output. These are then sent to ctlout 37 or ctlout 46. These are the controller values that set the file index number for oscillators A and B respectively. To put it in other terms, this will cause the video input to toggle between the two video files stored on the USB thumb drive.

The six makeautomation subroutines are mostly similar to each other, so we’ll look at makeautomation1. This subroutine features eight instances of the same thing. It starts with some version of a mod statement (% 224) or a similar expr object. The number 224 corresponds to 14 measures of music. If we halve that to seven measures the number of sixteenth notes is 112. So why am I dealing with seven minute units here? MIDI controller values range between zero and 127. The number 112 is the largest multiple of 16 that is still less than 127. Each successive expr object increments the incoming value by four before modding the value. This is done so that parameters are not set to the same value.

After the mod or expr object we see a moses object. When the value coming into the left inlet is below the stated value, in his case 112, that value is passed to the left outlet. If it is equal to or above that value, it passes out of the right outlet. This allows us to insert an expr object that subtracts the value from 224. Thus, this will create a motion from 112 down to 0. Thus, what this subroutine does is create incremental parameter changes that result in triangle wave forms. This can be confirmed by looking at the recorded MIDI data in Logic Pro.

After this the value then passes through a spigot object. Here the spigot is being controlled by a value from the parameters array. It that value is 0, the value going into the left inlet does not pass through. When that value is 1, the value at the left inlet will be passed to the outlet. Accordingly, we can see how the subroutine changevideo turns these spigots on and off. The values are then sent to the ctrl objects below.

The subroutine makeautomation1 handles the controllers for the color and the master gain. It also controls the values for frequency, rotation and polarization for both oscillators. The subroutine makeautomation2 handles controllers for oscillator A, including drift, color offset, saturation, fractal axis, and fractal amount. The subroutine makeautomation3 handles the same parameters as the previous subroutine, but for oscillator B. The subroutine makeautomation4 handles values related to feedback including: feedback hue shift, feedback zoom, feedback rotation, feedback X axis, and feedback Y axis. It also handles the controllers for feedback to gain for both oscillators. The subroutine makeautomation5 handles parameters for video manipulation of oscillator A, including X crop, lumakey, Y crop, lumakey max, and aspect A. The final makeautomation subroutine handles the same parameters as the previous subroutine, but for oscillator B.

As an experiment, this video was a success in the sense that it answered the question, can the Hypno handle a lot of MIDI input of controller values. That answer is clearly no. However, while the Hypno did not respond as directed by the algorithm in Pure Data, I cannot say that the results are not satisfying, though they are a considerable leap more chaotic that the results of the previous two experiments. I will be doing one more experiment with driving the Hypno using MIDI data, and will likely continue to use MIDI data when performing with the Hypno. However, when making music videos I will likely use Eurorack control voltages to drive the Hypno, as the results are far more predictable and controllable.

Part of this experiment was to see the outer limits of using MIDI with the Hypno. As already noted, this yielded fairly chaotic results where the source material is almost completely unrecognizable. In practice it seems that I will get more aesthetically pleasing results if I rein in the controller changes, yielding a more subtle response.

Creating Polymetric Beats Using Pure Data

I gave a presentation at Synthfest on November 9th, 2024 in Burlington, Massachusetts. This workshop is related to my ongoing research in using Pure Data as a tool for computer-assisted composition, sound processing, and sound synthesis. Specifically, the presentation is on how to use Pure Data to create polymetric beats. A hackable template is available below for download. You can hear such beats in my most recent album, Rotate (Bandcamp, Spotify, Apple, Amazon).

Pure Data is often used as a tool for sound synthesis or signal processing. A quick history lesson reminds us however that it is also a robust tool for algorithmic, or computer-assisted composition. Pure Data is an open source visual programming environment for sound and multimedia. It is strongly based upon the programming environment Max. When Max first added the ability to process and generate audio and video, it was referred to as Max/MSP/Jitter to highlight these new abilities. However, going back to the very beginnings of Max, originally developed at IRCAM in 1985 by Miller Puckette (who also developed Pure Data), it was centered on interactive, algorithmic and computer-assisted composition.

Defining the Problem

In order to create polymetric beats in Pure Data, it is useful to think of two steps in the process. The first is how do we represent musical patterns in a way that is easy to codify for computers. The second is how do we read that pattern in such a way that we can fire off a MIDI note at the correct time to realize that pattern.

Attached to this blog post is a program shell that we will use to understand a basic process where patterns are defined and initialized, a time structure, a structure for evaluating whether a note should happen, an algorithm for firing off a note, and a structure for changing musical patterns. The way this program shell is designed, it currently only generates kick drum parts in common time. However, once we learn how this algorithm works, we can hack it to create far more complicated beats.

Defining a Pattern

Generating rhythmic content using an algorithm is a very useful exercise, as we do not have to worry about pitch material. On the most basic level, we could think of a measure of as being a list of sixteenth notes (or whatever the fastest pulse of the desired rhythm is), and we can express the rhythm by using zeros where notes do not happen, and ones where notes are played. Thus, if we want to define a four on the floor kick drum part in common time, we could express it as . . .

1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0

When we define tables in Pure Data, the first number indicates where in the table we’re putting the values. Typically, we would want to start at the beginning of the table, which would be position 0. Thus, from now on in, I will start every rhythm description with a 0, resulting in . . .

0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0

This is pretty basic. If we wanted to go a bit more advanced, we could imagine two different options, a normal kick drum, which we’ll represent with the number 1, and an accented kick drum, which we’ll represent with the number 2. If we want to accent beat one of the measure, we now get the table description . . .

0 2 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0

Notice, it would be pretty easy at this point to use the number zero for when notes do not occur, but use the MIDI velocity number(1-127) to indicate when a note should occur. Doing this, you can get very finely tuned dynamic drum parts. However, such subtlety is not for me, I’m more of a boom / bap / boom-boom / bap guy myself, so I will be sticking with accented and unaccented notes. Let’s look at this table definition as it occurs in the program shell. To get there, double click on the object labeled pd initialize, which is below loadbang . . .

For those who are relatively new to Pure Data, loadbang designates algorithms that execute when you open the program. Likewise, any object in Pure Data that starts with the letters pd are subroutines. Double clicking them allows you to view and edit the algorithm contained inside the subroutine. Subroutines are a great way to declutter your screen, and to create algorithms that you may want to copy and paste into other programs you create. Note that the subroutine pd initialize also sets the tempo, translates that into milliseconds, and then into sixteenth notes (still expressed in milliseconds). Finally, we also see that pd initialize contains a definition for a table called phrase, but we’ll get to that later.

Changing Patterns

Before we get to how to use this kick drum pattern, I want to introduce one more level of complexity. In the long term it would be useful to define several different kick drum patterns so we can change patterns every eight measures or so to have a dynamic drum part. As we start to add more layers to the drum part (snare, high hat, toms, etc.) it would also be nice if some of those layers occasionally did not play at all, so the combination of layers also change as we move from phrase to phrase.

For the purposes of this algorithm, I’m going to allow for each layer of the drum part to select between three patterns (1, 2, & 3). Furthermore, I’ll also include a fourth possibility of 0, which would correspond to no pattern playing. In order to see how this is implemented in our algorithm, let’s look at the counter mechanism beneath the main metronome object. Beneath this metronome we see a trigger object, and beneath that, we see an object that states % 128. This is modular mathematics, which limits the value to a number between 0 and 127 (inclusive). We then feed the outcome of this to an object that states sel 0. When this object receives a 0 in its leftmost inlet, it will send a bang to its leftmost outlet. We then send this bang to trigger the subroutine pd patternchoice. Musically, what this means, is that the subroutine pd patternchoice executes at the beginning of every eight measure phrase. Since we are thinking in terms of sixteenth notes (8 x 16 = 128), a 0 would indicate the beginning of a phrase.

If we look inside pd patternchoice, we see two structures that are currently not connected to the inlet. In time we will copy and paste these structures, changing some of the numeric values, and connect them to the inlet. We will make these changes as we add more layers and patterns to our drum beat. However, for the time being since we only have one kick drum pattern, neither structure is connected to the inlet.

We will treat the algorithm on the left as being connected to the kick drum, while the algorithm on the right will eventually be used for the snare drum. Notice there is a difference between the two structures. The one on the left is simpler, and if we follow what it does, we figure out that that algorithm will write a 1, 2, or 3, but not a 0 to the table pattern in position 0. The structure on the right however, will write a 0 to the table patttern at position 1 half of the time. To put this into musical terms, the difference between the two means that kick drum will always be playing a pattern, while the snare drum will only play a pattern 50% of the time.

Firing Off a Note

Now lets see how pattern and kick1 are used to either fire off notes or not. We see at the bottom of the screen a makenote object connected to a noteout object that performs the final task of sending a midi note out. However pd makekick is the subroutine that determines whether or not a kick drum should occur at a given point. Above pd makekick we see the object % 16 which comes from the counter. This object mods the current counter to a number between 0 and 15 inclusive. These values correspond to the number of sixteenth notes in a measure of common time. Thus, when we return to the number 0 after the number 15 occurs it corresponds to returning to the beginning of the measure.

When we look at the structure pd makekick a lot of the heavy lifting of the subroutine is handled by the object spigot. Spigot receives numeric values through its leftmost inlet. However, it only passes those values through to its outlet if the numeric value being fed to its right most inlet is not zero. Thus, we can think of spigot as a valve that shuts off the flow of numbers when the rightmost inlet is zero. We can use this to selectively shut down the rest of the algorithm, which will effectively stop it from making notes.

The first step is to determine whether a pattern should be playing or not. Likewise, at the same time we can determine which patterns should be playing if one occurs. To do this, we’ll read the value of pattern at array position 0 (which we’ll use to store the current kick drum pattern). We can then route the output of that to a number of different outcomes using a sel statement. Each outcome of the sel statement sends either a 0 or a 1 to the rightmost inlet of three spigots. These spigots correspond to the three possible kick drum patterns. Thus, when the pattern is 0, a 0 is sent to all three spigots, effectively shutting off the rest of the subroutine. When the pattern is 1, we send a 1 to the spigot for pattern one (turning it on), and a zero to the other two, making sure any previously used pattern is turned off, and so fourth.

Once we pass through one of the spigots, we encounter the part of the subroutine that determines whether a note should be fired at any given time. Underneath each spigot is a tabread object that reads the current position of a given pattern (kick1, kick2, or kick3, respectively). This will return the result of a 0, 1, or 2, corresponding to no note, normal note, and accented note. Since we don’t have to do anything when no note is played, we can simply ignore that result. All we have to do is correctly route the results for 1 and 2. Since all three patterns will be outputing 0, 1, and 2, we will treat those three results the same, we can route the output of each tabread statement to the same number box, and then route the results using sel 1 2. Both results will sending the number 36 to s note (36 is the MIDI note number corresponding to a kick drum), and will send a velocity of either 90 or 120 to s velocity.

Understanding Execution Order

In order to understand the mechanics of this, we have to understand a little about execution order for Pure Data. When an outlet branches off in several directions, Pure Data first executes the connection that was created first, regardless of where it is placed on the screen (in Max, it executes from right to left). When an algorithm branches off in this way, it travels all the way down until it hits the end, and then Pure Data goes back and executes the other branch of the algorithm. When an object has multiple outlets, it typically executes the rightmost outlet first (following it all the way down the algorithm) before it most to the next outlet.

Alternately, when an object has several inlets, the object typically does not spring into action until the leftmost inlet is triggered. We can think of new values that are connected to inlets that are note the leftmost inlet as queueing until the leftmost inlet is triggered. This order of execution in Pure Data allows non-crossed connections to fire in the correct order. To illustrate, imagine an object with two outlets and an object below with two inlets (similar to the makenote / noteout algorithm below). Furthermore, let’s imagine that the right outlet is connected to the right inlet of the object below, and the left outlet is connected to the left inlet of the object below. First, the value from the right outlet will queue at the right inlet of the object below, then the left outlet will send its value to the left inlet, triggering the object below.

However, whenever we start to make complicated algorithms, it can be confusing to see in what order the algorithm will flow. When we want to force a specific order to achieve a specific outcome, or simply to make order clear, we can use a trigger object. A trigger takes a single inlet, and uses it to trigger several outlets (including the option of passing the inlet to one or more outlets) in a specific order. That order is (you guessed it, right to left). We will use this to control the order of sending the velocity and sending the note number.

The makenote object has three inlets that correspond to midi note number, velocity, and duration expressed in milliseconds. Since midi note number is the leftmost inlet, we have to send that value last. This means we have to send the velocity before we send the note number. Here we do not have to worry about duration, since duration is often irrelevant when triggering drum samples. Thus, we can set a duration of a sixteenth note during the initialization process.

Hacking the Shell

With the knowledge we have gained thus far, we are now equipped to start hacking the program shell. A good place to start would be to double click [pd initialize], copy both the object [table kick1] and the corresponding message that populates that table. You can then paste these objects, move them so they don’t overlap, change both to say kick2 instead of kick1, and change the numbers in the in the kick2 message to be a different pattern. Again, your pattern should only use the numbers 0, 1, and 2. Likewise, the table should be 17 numbers long with the first number being 0, in order to denote that we’ll load the array starting at the beginning. You can then connect the kick2 message to the [inlet] object.

Now do the same process again creating a table and message for kick3, making sure to connect the kick3 message to inlet. Now, we can finally make use of the subroutine [pd patternchoice] in the main algorithm. Double click on [pd patternchoice], and connect the outlet of the bang to both the [random 3] object and the message that contains the number 0. Now, when we run the the algorithm, we should hear the kick drum pattern randomly change once every eight measures.

Now we can get into the good stuff. Let’s add a snare drum part. First, we should decide what meter we want to use for the snare drum. For the purposes of this demonstration, let’s put the snare drum in three. We’ll start by copying the object [pd makekick] and pasting it. More the copied object under the [% 12] object, and connect the outlet of [% 12] to the inlet of the copied object. Let’s also rename the copied object to [pd makesnare].

We have to change some details of [pd makesnare] to get it to make a snare part. Double click the object and change the message box near the top from 0 to 1. Change the array names in the three [tabread] objects to be snare1, snare2, and snare3. Finally, right above the object [s note], change the number in the message box from 36 to 38 (the MIDI note number for snare drum).

Now, double click the object [initialize], and copy the three kick drum tables, as well as the three messages that define those tables. Paste those items, and move them so they don’t overlap with other items. change the array names to snare1, snare2, and snare3 in both the table objects and the messages that define the arrays. Now, let’s change the patterns. Since these are patterns that are in three, they will feature 12 sixteenth notes. Thus, each message will have to include 13 numbers, the first being the number 0 to indicate that the pattern is loaded at the beginning of the array. Again, use 0 to indicate no note, 1 to indicate a note, and 2 to represent an accented note. Make sure you connect each of the three messages to the inlet.

Now we need to allow the snare patterns to change, so let’s double click the subroutine [pd patternchoice]. In this subroutine we need to connect the outlet of the bang to both the [random 2] object and the message that contains the number 1. The [random 2] here yields a 50% chance that the snare appear in a given phrase. The [random 3] below that will select one of the three snare patterns when it is determined that the snare should appear in a phrase.

If we wanted to make a percussion part that used two different MIDI notes, we should look at the subroutine, [pd makehh]. Two notes are required for a high hat part to allow for closed and open high hats. If we look inside pd makehh, we see that the patterns allow for five possibilities, 0, 1, 2, 3, & 4. When we look at the sel statement near the bottom of the screen, the number 0 does not appear in it. Again, this is because 0 means do not play a note, so we can simply ignore that result. The way this is setup, 1 and 2 are closed high hats with the latter being accented, while 3 and 4 are open high hats with 4 being accented.

If we wanted to fully implement this subroutine, we’d have to attach it to one of the meter generators, % 16, % 12, % 14, % 10, or % 11. Then we would have to create three patterns (hh1, hh2, hh3) in [pd initialize], remembering to make the table, and use a message to populate it. You would populate the patterns using only the numbers 0, 1, 2, 3, and 4. Then you’d have to copy the block of code inside of pd patternchoice that we used to choose the pattern for the snare drum, paste it, change the index in the message object to 2, and connect the random object and the index message to the inlet.

We could also adapt [pd makehh] for creating a subroutine that will generate a toms or conga part consisting of a high tom / conga and a low tom / conga. You would have to remember to use a new / different index number to store the pattern, and change the MIDI note numbers to those corresponding to either the a high tom or conga and a low tom or conga. Ultimately we can continue in this vein, creating increasingly complicated drum beats, using [pd makekick] as a means creating layers of single note percussion parts, and [pd makehh] as a model for creating percussion layers using two different notes. As the number of meters used increases, you’ll find the resulting beats get increasingly funky, and take longer times to evolve in compelling ways, enabling you to inject rhythmic interest into your music.

Experiment 12: MBTA

As I mentioned in my previous experiment, it has been a busy and difficult semester for me for family reasons. Accordingly, I am two and a half months behind schedule delivering my 12th and final experiment for this grant cycle. Additionally, I feel like my work for the past few months on this process has been a bit underwhelming, but unfortunately this work what my current bandwidth allows for. I hope to make up for it in the next year or so.

Anyway, the experiment for this month is similar to the one done for experiment 11. However, in this experiment I am generate vector images that reference maps of Boston’s subway system (the MBTA). Due to the complexity of the MBTA system I’ve created four different algorithms, reducing the visual data at any time to one quadrant of the map, thus, the individual programs are called: MBTA NW, MBTA NE, MBTA SE, & MBTA SW.

Since all four algorithms are basically the same, I’ll use MBTA – NE as an example. For each example Knob 5 was used for the background color. There were far more attributes I wanted to control than knobs I had at my disposal, so I decided to link them together. Thus, for MBTA – NE knob 1 controls red line attributes, knob 2 controls the blue and orange line attributes, knob 3 controls the green line attributes, and knob 4 controls the silver line attributes. Each of the four programs assigns the knobs to different combinations of colored lines based upon the complexity of the MBTA map in that quadrant.

The attributes that knobs 1-4 control include: line width, scale (amount of wiggle), color, and number of superimposed lines. The line width ranges from one to ten pixels, and is inversely proportional to the number of superimposed lines which ranges from on to eight. Thus, the more lines there are, the thinner they are. The scale, or amount of wiggle is proportional to the line width, that is the thicker the lines, the more they can wiggle. Finally, color is defined using RGB numbers. In each case, only one value (the red, the green, or the blue) changes with the knob values. The amount of change is a twenty point range centered around the optimal value. We can see this implemented below in the initialization portion of the program.

	RElinewidth = int (1+(etc.knob1)*10)
	BOlinewidth = int (1+(etc.knob2)*10)
	GRlinewidth = int (1+(etc.knob3)*10)
	SIlinewidth = int (1+(etc.knob4)*10)
	etc.color_picker_bg(etc.knob5)
	REscale=(55-(50*(etc.knob1)))
	BOscale=(55-(50*(etc.knob2)))
	GRscale=(55-(50*(etc.knob3)))
	SIscale=(55-(50*(etc.knob4)))
	thered=int (89+(10*(etc.knob1)))
	redcolor=pygame.Color(thered,0,0)
	theorange=int (40+(20*(etc.knob2)))
	orangecolor=pygame.Color(99,theorange,0)
	theblue=int (80+(20*(etc.knob2)))
	bluecolor=pygame.Color(0,0,theblue)
	thegreen=int (79+(20*(etc.knob3)))
	greencolor=pygame.Color(0,thegreen,0)
	thesilver=int (46+(20*(etc.knob4)))
	silvercolor=pygame.Color(50,53,thesilver)
	j=int (9-(1+(7*etc.knob1)))

The value j stands for the number of superimposed lines. This then transitions into the first of four loops, one for each of the groups of lines. Below we see the code for red line portion of program. The other three loops are fairly much the same, but are much longer due to the complexity of the MBTA map. An X and a Y coordinate are set inside this loop for every point that will be used. REscale is multiplied by a value from etc.audio_in which is divided by 33000 in order to change that audio level into a decimal ranging from 0 to 1 (more or less). This scales the value of REscale down to a smaller value, which is added to the numeric value. It is worth noting that because audio values can be negative, the numeric value is at the center of potential outcomes. Scaling the index number of etc.audio_in by (i*11), (i*11)+1, (i*11)+2, & (i*11)+3 lends a suitable variety of wiggles for each instance of a line.

	j=int (9-(1+(7*etc.knob1)))
	for i in range(j):
		AX=int (320+(REscale*(etc.audio_in[(i*11)]/33000)))
		AY=int (160+(REscale*(etc.audio_in[(i*11)+1]/33000)))
		BX=int (860+(REscale*(etc.audio_in[(i*11)+2]/33000)))
		BY=int (720+(REscale*(etc.audio_in[(i*11)+3]/33000)))
		pygame.draw.line(screen, redcolor, (AX,AY), (BX, BY), RElinewidth)

I arbitrarily limited each program to 26 points (one for each letter of the alphabet). This really causes the vector graphic to be an abstraction of the MBTA map. The silver line in particular gets quite complicated, so I’m never really able to fully represent it. That being said, I think that anyone familiar with Boston’s subway system would recognize it if the similarity was pointed out to them. I also imagine any daily commuter on the MBTA would probably recognize the patterns in fairly short order. However, in watching my own video, which uses music generated by a PureData algorithm that will be used to write a track for my next album, I noticed that the green line in the MBTA – NE and MBTA – SW needs some correction.

The EYESY has been fully incorporated into my live performance routine as Darth Presley. You can see below a performance at the FriYay series at the New Bedford Art Museum. You’ll note that the projection is the Random Lines algorithm that I wrote. Likewise graduating senior Edison Roberts used the EYESY for his capstone performance as the band Geepers! You’ll see a photo of him below with a projection using the Random Concentric Circles algorithm that I wrote. I definitely have more ideas of how to use the EYESY in live performance. In fact, others have started to use ChatGPT to create EYESY algorithms.

Ultimately my work on this grant project has been fruitful. To date the algorithms I’ve written for the Organelle and EYESY have been circulated pretty well on Patchstorage.com (clearly the Organelle is the more popular format of the two) . . .

2opFM (Organelle) 2 likes, 586 views, 107 downloads
Additive Odd / Even (Organelle) 6 likes, 969 views, 184 downloads
Bass Harmonica (Organelle) 7 likes, 825 views, 174 downloads
Basic Circle (EYESY) 307 views, 7 downloads
Wavetable Sampler (Organelle) 2 likes, 796 views, 123 downloads
Basic Circles (EYESY) 1 like, 279 views, 16 downloads
Random Lines (EYESY) 198 views, 18 downloads
Random Concentric Circles (EYESY) 132 views, 18 downloads
Colored Rectangles (EYESY) 1 like, 149 views, 31 downloads
Random Rectangles (EYESY) 168 views, 26 downloads
Random Radii (EYESY) 1 like, 169 views, 16 downloads
Constellations (EYESY) 1 like, 264 views, 14 downloads
MBTA (EYESY) 21 views
Total (Organelle) 4 patches, 17 likes, 3,176 views, 588 downloads
Total (EYESY) 23 patches, 4 likes, 1,687 views, 146 downloads
Total: 27 patches, 21 likes, 4,863 views, 734 downloads

Experiment 11: Constellations

February was a very busy month for me for family reasons, and it’ll likely be that way for a few months. Accordingly, I’m a bit late on my February experiment, and will likely be equally late with my final experiment as well. I have also stuck with programming for the EYESY, as I have kind of been on a roll in terms of coming up with ideas for it.

This month I created twelve programs for the EYESY, each of which displays a different constellation from the zodiac. I’ve named the series Constellations and have uploaded them to patchstorage. Each one works in exactly the same manner, so we’ll only look at the simplest one, Aries. The more complicated programs simply have more points and lines in them with different coordinates and configurations, but are otherwise are identical.

Honestly, one of the most surprising challenges of this experiment way trying to figure out if there’s any consensus for a given constellation. Many of the constellations are fairly standardized, however others are fairly contested in terms of which stars are a part of the constellation. When there were variants to choose from I looked for consensus, but at times also took aesthetics into account. In particular I valued a balance between something that would look enticing and a reasonable number of points.

I printed images of each of the constellations, and traced them onto graph paper using a light box. I then wrote out the coordinates for each point, and then scaled them to fit in a 1280×720 resolution screen, offsetting the coordinates such that the image would be centered. These coordinates then formed the basis of the program.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	linewidth = int (1+(etc.knob4)*10)
 	etc.color_picker_bg(etc.knob5)
	offset=(280*etc.knob1)-140
	scale=5+(140*(etc.knob3))
	r = int (abs (100 * (etc.audio_in[0]/33000)))
	g = int (abs (100 * (etc.audio_in[1]/33000)))
	b = int (abs (100 * (etc.audio_in[2]/33000)))
	if r>50:
		rscale=-5
	else:
		rscale=5
	if g>50:
		gscale=-5
	else:
		gscale=5
	if b>50:
		bscale=-5
	else:
		bscale=5
	j=int (1+(8*etc.knob2))
	for i in range(j):
		AX=int (offset+45+(scale*(etc.audio_in[(i*8)]/33000)))
		AY=int (offset+45+(scale*(etc.audio_in[(i*8)+1]/33000)))
		BX=int (offset+885+(scale*(etc.audio_in[(i*8)+2]/33000)))
		BY=int (offset+325+(scale*(etc.audio_in[(i*8)+3]/33000)))
		CX=int (offset+1165+(scale*(etc.audio_in[(i*8)+4]/33000)))
		CY=int (offset+535+(scale*(etc.audio_in[(i*8)+5]/33000)))
		DX=int (offset+1235+(scale*(etc.audio_in[(i*8)+6]/33000)))
		DY=int (offset+675+(scale*(etc.audio_in[(i*8)+7]/33000)))
		r = r+rscale
		g = g+gscale
		b = b+bscale
		thecolor=pygame.Color(r,g,b)
		pygame.draw.line(screen, thecolor, (AX,AY), (BX, BY), linewidth)
		pygame.draw.line(screen, thecolor, (BX,BY), (CX, CY), linewidth)
		pygame.draw.line(screen, thecolor, (CX,CY), (DX, DY), linewidth)

In these programs knob 1 is used to offset the image. Since only one offset is used, rotating the knob moves the image on a diagonal moving from upper left to lower right. The second knob is used to control the number of superimposed versions of the given constellation. The scale of how much the image can vary is controlled by knob 3. Knob 4 controls the line width, and the final knob controls the background color.

The new element in terms of programing is a for statement. Namely, I use for i in range (j) to create several superimposed versions of the same constellation. As previously stated, the amount of these is controlled by knob 2, using the code j=int (1+(8*etc.knob2)). This allows for anywhere from 1 to 8 superimposed images.

Inside this loop, each point is offset and scaled in relationship to audio data. We can see for any given point the value is added to the offset. Then the scale value is multiplied by data from etc.audio_in. Using different values within this array allows for each point in the constellation to react differently. Using the variable i within the array also allows for differences between the points in each of the superimposed versions. The variable scale is always set to be at least 5, allowing for some amount of wiggle given all circumstances.

Originally I had used data from etc.audio_in inside the loop to set the color of the lines. This resulted in drastically different colors for each of the superimposed constellations in a given frame. I decided to tone this down a bit, by using etc.audio_in data before the loop started allowing each version of the constellation within a given frame to be largely the same color. That being said, to create some visual interest, I use rscale, gscale, and bscale to move the color in a direction for each superimposed version. Since the maximum amount of superimposed images is 8, I used the value 5 to increment the red, green, and blue values of the color. When the original red, green, or blue value was less than 50 I used 5, which moves the value up in value. When the original red, green, or blue value was more than 50 I used -5, which moves the value down in value. The program chooses between 5 and -5 using if :else statements.

The music used in the example videos are algorithms that will be used to generate accompaniment for a third of my next major studio album. These algorithms grew directly out of my work on these experiments. I did add one little bit of code the these puredata algorithms however. Since I have 6 musical examples, but 12 EYESY patches, I added a bit of code that randomly chooses between 1 of 2 EYESY patches and sends out a program (patch) change to the EYESY on MIDI channel 16 at the beginning of each phrase.

While I may not use these algorithms for the videos for the next studio album, I will likely use them in live performances. I plan on doing a related set of EYESY programs for my final experiment next month.

Experiment 11A: Aries & Taurus:

Experiment 11B: Gemini & Cancer:

Experiment 11C: Leo & Virgo:

Experiment 11D: Libra & Scorpio:

Experiment 11E: Sagittarius & Capricorn:

Experiment 11F: Aquarius & Pisces:

Experiment 10: Five EYESY Algorithms

I kind of hit a wall of the Organelle. I feel like in order to advance my skills I have a bit of a hurdle between where my programming skills are at, and where they would need to be to do something more advanced that the recent experiments I have completed. Accordingly for this month I decided to shift my focus to the EYESY. Last month I made significant progress in understanding Python programming for the EYESY, and that allowed me to come up with five ideas for algorithms in short order. The music for all five mini-experiments comes from PureData algorithms I will be using for my next major album. All five of these algorithms are somewhat derived from my work on my last album.

The two realizations that allowed me to make significant progress on EYESY programming is that Python is super picky about spaces, tabs, and indentations, and that while the EYESY usually gives little to no feedback when a program has an error in it, you can use an online Python compiler to help figure out where your error is (I had mentioned the latter in last month’s experiment). Individuals who have a decent amount of programming experience may scoff at the simplicity of the programs that follow, but for me it is a decent starting place, and it is also satisfying to me to see how such simple algorithms can generate such gratifying results.

Random Lines is a patch I wrote that draws 96 lines. In order to do this in an automated fashion, I have to use a loop, in this case I use for i in range(96):. The five lines that follow are all executed in this loop. Before the loop commences, we choose the color using knob 4 and the background color using knob 5. I use knob 3 to set the line width, but I scale it by multiplying the knob’s value, which will be between 0 and 1, by 10, and adding 1, as line widths cannot have a value of 0. I also have to cast the value as an integer. I set an x offset value using knob 1. Multiplying by 640 and then subtracting 320 will yield a result between -320 and 320. Likewise, a y offset value is set using knob 2, and the scaling results in a value between -180 and 180.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	color = etc.color_picker(etc.knob4)
	linewidth = int (1+ (etc.knob3)*10)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
   	for i in range(96):
		x1 = int (640 + (xoffset+(etc.audio_in[i])/50))
		y1 = int (360 + (yoffset+(etc.audio_in[(i+1)])/90))
		x2 = int (640 + (xoffset+(etc.audio_in[(i+2)])/50))
		y2 = int (360 + (yoffset+(etc.audio_in[(i+3)])/90))
		pygame.draw.line(screen, color, (x1,y1), (x2, y2), linewidth)

Within the loop, I set two coordinates. The EYESY keeps track of the last hundred samples using etc.audio_in[]. Since these values use sixteen bit sound, and sound has peaks (represented by positive numbers) and valleys (represented by negative numbers), these values range between -32,768 and 32,787. I scale these values by dividing by 50 for x coordinates. This will scale the values to somewhere between -655 and 655. For y coordinates I divide by 90, which yields values between -364 and 364.

In both cases, I add these values to the corresponding offset value, and add the value that would normally, without the offsets, place the point in the middle of the screen, namely 640 (X) and 360 (Y). A negative value for the xoffset or the scaled etc.audio_in value would move that point to the left, while a positive value would move it to the right. Likewise, a negative value for the yoffset or the scale etc.audio_in value would move the point up, while a positive value would move it down.

Since subsequent index numbers are used for each coordinate (that is i, i+1, i+2, and i+3), this results in a bunch of interconnected lines. For instance when i=0, the end point of the drawn line (X2, Y2) would become the starting point when i=2. Thus, the lines are not fully random, as they are all interconnected, yielding a tangled mass of lines.

Random Concentric Circles uses a similar methodology. Again, knob five is use to control the background color, while knobs 1 and 2 are again scaled to provide an X and Y offset. The line width is shifted to knob 4. For this algorithm the loop happens 94 times. The X and Y value for the center of the circles is determined the same way as was done in Random Lines. However, we now need a radius and we need a color for each circle.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	linewidth = int (1+(etc.knob4)*9)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
	for i in range(94):
		x = int (640 + xoffset+(etc.audio_in[i])/50)
		y = int (360 + yoffset+(etc.audio_in[(i+1)])/90)
		radius = int (11+(abs (etc.knob3 * (etc.audio_in[(i+2)])/90)))
		r = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+5)]/33000)))
		thecolor=pygame.Color(r,g,b)
		pygame.draw.circle(screen, thecolor, (x,y), radius, linewidth)

We have knob 3 available to help control the radius of the circle. Here I multiply knob 3 by a scaled version of etc.audio_in[(i+2)]. I scale it by dividing by 90 so that the largest possible circle will mostly fit on the screen if it is centered in the screen. Notice that when we multiply knob 3 by etc.audio_in, there’s a 50% chance that the result will be a negative number. Negative values for radii don’t make any sense, so I take the absolute value of this outcome using abs. I also add this value to 11, as a radius of 0 makes no sense, and a radius of less than 10, as having a line width that is larger than the radius will cause an error.

For this algorithm I take a set forward by giving each circle its own color. In order to do this I have to set the value for red, green, and blue separately, and then combine them together using pygame.Color(). For each of the three values (red, green, and blue) I divide a value of etc. audio_in by 33000, which will yield a value between 0 and 1 (more or less), and then multiply this by 100. I could have done the same thing by simply dividing etc.audio_in by 330, however, at the time this process made the most sense to me. Again, this process could result in a negative number and / or a fractional value, so I cast the result as an integer after getting its absolute value.

Colored Rectangles has a different structure than the previous two examples. Rather than have all the objects cluster around a center point I wanted to create an algorithm that spaces all of the objects out evenly throughout the screen in a grid like pattern. I do this using an eight by eight grid of 64 rectangles. I accomplish the spacing using modulus mathematics as well as integer division. The X value is obtained by multiplying 160 times i%8. In a similar vein, the Y values is set to 90 times i//8. Integer division in Python is accomplished through the use of two slashes. Using this operator will return the integer value of a division problem, omitting the fractional portion. Both the X and the Y values have an additional offset value. The X is offset by (i//8)*(80*etc.knob1), so this offset increases as knob 1 is turned up, with a maximum offset of 80 pixels per row. The value i//8 essentially multiplies that offset by the row number. That is the rows shift further towards the right.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
 	etc.color_picker_bg(etc.knob5)
   	for i in range(64):
		x=(i%8)*160+(i//8)*(80*etc.knob1)
		y=(i//8)*90+(i%8)*(45*etc.knob2)
		thewidth=int (abs (160 * (etc.knob3) * (etc.audio_in[(i)]/33000)))
		theheight=int (abs (90 * (etc.knob4) * (etc.audio_in[(i+1)]/33000)))
		therectangle=pygame.Rect(x,y,thewidth,theheight)
		r = int (abs (100 * (etc.audio_in[(i+2)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		thecolor=pygame.Color(r,g,b)
		pygame.draw.rect(screen, thecolor, therectangle, 0)

Likewise, the Y offset is determined by (i%8)*(45*etc.knob2). As the value of knob 2 increases, the offset moves towards a maximum value of 45. However, as the columns shift to the right, those offsets compound due to the fact that they are multiplied by (i%8).

A rectangle can be defined in pygame by passing an X value, a Y value, width, and height to pygame.Rect. Thus, the next step is to set the width and height of the rectangle. In both cases, I set the maximum value to 160 (for width) and 90 (for height). However, I scaled them both by multiplying by a knob value (knob 3 for width and knob 4 for height). These values are also scaled by an audio value divided by 33,000. Since negative values are possible from audio values, and negative widths and heights don’t make much sense, I took the absolute value of each. If I were to rewrite this algorithm (perhaps I will), I would set a minimum value for width and height such that widths and heights of 0 were not possible.

I set the color of each rectangle using the same method as I did in Random Concentric Circles. In order to draw the rectangle you pass the screen, the color, the rectangle (as defined by pygame.Rect), as well as the line width to pygame.draw.rect. Using a line width of 0 means that the rectangle will be filled in with color.

Random Rectangles is a combination of Colored Rectangles and Random Lines. Rather than use pygame’s Rect object to draw rectangles on the screen, I use individual lines to draw the rectangles (technically speaking they are quadrilaterals). Knob 4 is used here to set the foreground color, knob 5 is used here to set the background color, knob 3 is used to set the linewidth.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	color = etc.color_picker(etc.knob4)
 	etc.color_picker_bg(etc.knob5)
	linewidth = int (1+ (etc.knob3)*10)
   	for i in range(64):
		x=(i%8)*160+(i//8)*(80*etc.knob1)+(40*(etc.audio_in[(i)]/33000))
		y=(i//8)*90+(i%8)*(45*etc.knob2)+(20*(etc.audio_in[(i+1)]/33000))
		x1=x+160+(40*(etc.audio_in[(i+2)]/33000))
		y1=y+(20*(etc.audio_in[(i+3)]/33000))
		x2=x1+(40*(etc.audio_in[(i+4)]/33000))
		y2=y1+90+(20*(etc.audio_in[(i+5)]/33000))
		x3=x+(40*(etc.audio_in[(i+6)]/33000))
		y3=y+90+(20*(etc.audio_in[(i+7)]/33000))
		pygame.draw.line(screen, color, (x,y), (x1, y1), linewidth)
		pygame.draw.line(screen, color, (x1,y1), (x2, y2), linewidth)
		pygame.draw.line(screen, color, (x2,y2), (x3, y3), linewidth)
		pygame.draw.line(screen, color, (x3,y3), (x, y), linewidth)

Within the loop, I use a similar method of setting the initial X and Y coordinates. That being said, I separate out the use of knobs and the use of audio input. In the case of the X coordinated, I use (i//8)*(80*etc.knob1) to control the amount of x offset for each row, with a maximum offset of 80. The audio input then offsets this value further using (40*etc.audio_in[(i+2)]/33000). This moves the x value by a value of plus or minus 40 (remember that audio values can be negative. Likewise, knob 2 offsets the Y value for every row by a maximum of 45, and the audio input further offsets this value by plus or minus 20.

Since it takes four points to define a quadrilateral, we need three more points, which we will call (x1, y1), (x2, y2), and (x3, y3). These are all interrelated. The value of X is used to define X1 and X3, while X2 is based off of X1. Likewise, the value of Y helps define Y1 and Y3, with Y2 being based off of Y1. In the case X1 and X2 (which is based on X1) we add 160 to X, giving a default width, but these values are again scaled by etc.audio_in. Similarly, we add 90 to Y1 and Y3 to determine a default height of the quadrilaterals, but again, all points are further offset by etc.audio_in, resulting in quadrilaterals, rather than rectangles with parallel sides. If I were to revise this algorithm I would likely make each quadrilateral a different color.

Frankly, I was not as pleased with the results of Colored Rectangles and Random Rectangles, so I decided to go back create an algorithm that was an amalgam of Random Lines and Random Concentric Circles, namely Random Radii. This program creates 95 lines, all of which have the same starting point, but different end points. Knob 5 sets the background color, while knob 4 sets the line width.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	linewidth = int (1+ (etc.knob4)*10)
 	etc.color_picker_bg(etc.knob5)
	xoffset=(640*etc.knob1)-320
	yoffset=(360*etc.knob2)-180
	X=int (640+xoffset)
	Y=int (360+yoffset)
   	for i in range(95):
		r = int (abs (100 * (etc.audio_in[(i+2)]/33000)))
		g = int (abs (100 * (etc.audio_in[(i+3)]/33000)))
		b = int (abs (100 * (etc.audio_in[(i+4)]/33000)))
		thecolor=pygame.Color(r,g,b)
		x2 = int (640 + (xoffset+etc.knob3*(etc.audio_in[(i)])/50))
		y2 = int (360 + (yoffset+etc.knob3*(etc.audio_in[(i+1)])/90))
		pygame.draw.line(screen, thecolor, (X,Y), (x2, y2), linewidth)

Knob 1 & 2 are used for X and Y offsets (respectively) of the center point. Using (640*etc.knob1)-320 means that the X value will move plus or minus 320. Similarly, (360*etc.knob2)-180 permits the Y value to move up or down by 180. As is the case with Random Concentric Circles, the color of each line is defined by etc.audio_in. Knob 3 is used to scale the end point of each line. In the case of both the X and Y values, we start from the center of the screen (640, 360) add the offset defined by knobs 1 or 2, and then use knob 3 to scale a value derived from etc.audio_in. Since audio values can be positive or negative, these radii shoot outward from the offset center point in all directions.

As suggested earlier, I am very gratified with these results. Despite the simplicity of the Python code, the results are mostly dynamic and compelling (although I am somewhat less than thrilled with Colored Rectangles and Random Rectangles). The user community for the EYESY is much smaller than that of the Organelle. The EYESY user’s forum has only 13% the activity of that of the Organelle. I seem to have inherited being the administrator of the ETC / EYESY Video Synthesizer from Critter&Guitari group on Facebook. Likewise, I am the author of the seven most recent patches for the EYESY on patchstorage. Thus, this month’s experiment sees me shooting to the top to become the EYESY’s most active developer. The start of the semester has been very busy for me. I am somewhat doubting that I will be coming up with an Organelle patch next month, but I equally suspect that I will continue to develop more algorithms of the EYESY.

Experiment 9: Wavetable Sampler

This month’s experiment is a considerable step forward on three fronts: Organelle programming, EYESY programming, and computer assisted composition. In terms of Organelle programming, rather than taking a pre-existing algorithm and altering it (or hacking it) to do something different, I decided to create a patch from scratch. I first created it in PureData, and then reprogrammed it to work specifically with the Organelle. Creating it in PureData first meant that I used horizontal sliders to represent the knobs, and that I sent the output to the DAC (digital to analog converter). When coding for the Organelle, you use a throw~ object to output the audio.

The patch I wrote, Wavetable Sampler, reimagines a digital sampler, and in doing so, basically reinvents wavetable synthesis. The conventional approach to sampling is to travel through the sound in a linear fashion from beginning to end. The speed at which we travel through the sound determines both its length and its pitch, that is faster translates to shorter and higher pitched, while slower means longer and lower pitched.

I wanted to try using an LFO (low frequency oscillator) to control where we are in a given sample. Using this technique the sound would go back and forth between two points in the sample continuously. In my programming I quickly learned that two parameters are strongly linked, namely the frequency of this oscillator and the amplitude of the oscillator, which becomes the distance travelled in the sample. If you want the sample to be played at a normal speed, that is we recognize the original sample, those two values need to be proportional. To describe this simply, a low frequency would require the sample to travel farther while a higher frequency would need a small amount of space. Thus, we see the object expr (44100 / ($f1)), with the number 44,100 being the sample rate of the Organelle. Dividing the sample rate by the frequency of the oscillator yields the number of samples that make up a cycle of sound at that frequency.

Obviously, a user might want to specifically move at a different rate than what would be normal. However, making that a separate control prevents the user from having to mentally calculate what would be an appropriate sample increment to have the sample play back at normal speed. I also specified that a user will want to control where we are in a much longer sample. For instance, the sample I am using with this instrument is quite long. It is an electronic cello improvisation I did recently that lasts over four minutes.

The sound I got out of this instrument was largely what I expected. However, there was one aspect that stood out more than I thought it would. I am using a sine wave oscillator in the patch. This means that the sound travels quickly during the rise and fall portion of the waveform, but as it approaches the peak and trough of the waveform it slows down quite dramatically. At low frequencies this results in extreme pitch changes. I could easily have solved this issue by switching to a triangle waveform, as speed would be constant using such a waveform. However, I decided that the oddness of these pitch changes were the feature of the patch, and not the bug.

While I intended the instrument to be used at low frequencies, I found that it was actually far more practical and conventionally useful at audio frequencies. Human hearing starts around 20Hz, which means if you were able to clap your hands 20 times in a second you would begin to hear it as a very low pitch rather than a series of individual claps. One peculiarity of sound synthesis is that if you repeat a series of samples, no mater how random they may be, at a frequency that lies within human hearing, you will hear it as a pitch that has that frequency. The timbre between two different sets of samples at the same frequency may vary greatly, but we will hear them as being, more or less, the same pitch.

Thus, as we move the frequency of the oscillator up into the audio range, it turns into somewhat of a wavetable synthesizer. While wavetable synthesis was created in 1958, it didn’t really exist in its full form until 1979. At this point in the history of synthesis it was an alternative to FM synthesis, which could offer robust sound possibilities but was very difficult to program, and digital sampling, which could recreate any sound that could be recorded but was extremely expensive due to the cost of memory. In this sense wavetable synthesis is a data saving tool. If you imagine a ten second recording of a piano key being struck and held, the timbre of that sound changes dramatically over those ten seconds, but ten seconds of sampling time in 1980 was very expensive. Imagine if instead we can digitize individual waveforms at five different locations in the ten second sample, we can then gradually interpolate between those five waveforms to create a believable (in 1980) approximation of the original sample. That being said, wavetable synthesis also created a rich, interesting approach to synthesizing new sounds such that the technique is still somewhat commonly used today.

When we move the oscillator for Wavetable Sampler into the audio range, we are essentially creating a wavetable. The parameter that effects how far the oscillator travels through the sample creates a very interesting phenomenon at audio rates. When that value is very low, the sample values vary very little. This results in waveforms that approach a sine wave in their simplicity and spectrum. As this value increases more values are included, which may differ greatly from each other. This translates into adding harmonics into the spectrum. Which harmonics are added are dependent up the wavetable, or snippet of a sample, in question. However, as we turn up the value, it tends to add harmonics from lower ones to higher ones. At extreme values, in this case ten times a normal sample increment, the pitch of the fundamental frequency starts to be over taken by the predominant frequencies in that wavetable’s spectrum. One final element of interest with the construction of the instrument in relation to wavetable synthesis is related to the use of a sine wave for the oscillator. Since the rate of change speeds up during the rise and fall portion of the waveform and slows down near the peak and the valley of the wave, that means there are portions of the waveform that rich in change while other portions where the rate of change is slow.

Since the value that the oscillator travels seems to be analogous to increasing the harmonic spectrum, I decided to put that on knob four, as that is the knob I have been controlling via breath pressure with the WARBL. On the Organelle I set knob one to set the index of where we are in the four minute plus sample. The frequency of the oscillator is set by the key that is played, but I use the second knob as a multiplier of this value. This knob is scaled from .001, which will yield a true low frequency oscillator, to 1, which will be at the pitch played (.5 will be down an octave, .25 will be down two octaves, etc.). As stated earlier, the fourth knob is used to modify the amplitude of the oscillator, affecting the range of samples that will be cycled through. This left the third knob unused, so I decided to use that as a decay setting.

The PureData patch that was used to generate the accompaniment for this experiment was based upon the patch created for last month’s experiment. As a reminder, this algorithm randomly chooses between one of four musical meters, 4/4, 3/4, 7/8, and 5/8, at every new phrase. I altered this algorithm to fit a plan I have for six of the tracks on my next studio album, which will likely take three or four years to complete. Rather than randomly selecting them, I define an array of numbers that represent the meters that should be used in the order that they appear. At every phrase change I then move to the next value in the array, allowing the meters to change in a predetermined fashion.

I put the piece of magic that allows this to happen in pd phrasechange. The inlet to this subroutine goes to a sel statement that contains the numbers of new phrase numbers expressed in sixteenth notes. When those values are reached a counter is incremented, a value from the table meter is read, which is sent to the variable currentmeter and the phrase is reset. This subroutine has four outlets. The first starts a blinking light that indicates that the piece is 1/3 finished, the second outlet starts a blinking light that starts when the piece is 2/3 of the way finished. The third outlet starts a blinking light that indicates the piece is on its final phrase. The fourth outlet stops the algorithm, bringing a piece to a close. Those blinking lights are on the right hand side of the screen, located near the buttons that indicate the current meter and the current beat. A performer can then, with some practice watch the computer screen to see what the current meter is, what the current beat is, and to have an idea of where you are in form of the piece.

This month I created my first program for the EYESY, Basic Circles. The EYESY manual includes a very simple example of a program. However, it is too simple it just displays a circle. The circle doesn’t move, none of the knobs change anything, and the circle isn’t even centered. With very little work I was able to center the circle, and change it so that the size of the circle was controlled by the volume of the audio. Likewise, I was able to get knob four to control the color of the circle, and the fifth knob to control the background color.

However, I wanted to create a program that used all five knobs on the EYESY. I quickly came up with the idea of using knob two to control the horizontal positioning, and the third knob to control the vertical positioning. I still had one knob left, and only a simple circle in the middle of the screen to show for it. I decided to add a second circle, that was half the size of the first one. I used knob five to set the color for this second circle, although oddly it does not result in the same color as the background. Yet, this still was not quite visually satisfying, so I set knob one to set an offset from the larger circle. Accordingly, when knob one is in the center, the small circle is centered within the larger one. As you turn the knob to the left the small circle moves to the upper left quadrant of the screen. As you turn the knob to the right the smaller circle moves towards the lower right quadrant. This is simple, but offers just enough visual interest to be tolerable.

import os
import pygame
import time
import random
import math

def setup(screen, etc):
    pass

def draw(screen, etc):
	size = (int (abs (etc.audio_in[0])/100))
	size2 = (int (size/2))
	position = (640, 360)
	color = etc.color_picker(etc.knob4)
	color2 = etc.color_picker(etc.knob5)
	X=(int (320+(640*etc.knob2)))
	X2=(int (X+160-(310*etc.knob1)))
	Y=(int (180+(360*etc.knob3)))
	Y2=(int (Y+90-(180*etc.knob1)))
	etc.color_picker_bg(etc.knob5)  
	pygame.draw.circle(screen, color, (X,Y), size, 0)
	pygame.draw.circle(screen, color2, (X2,Y2), size2, 0)

While the program, listed above, is very simple, it was my first time programming in Python. Furthermore, targeting the EYESY is not the simplest thing to do. You have plug a wireless USB adapter into the EYESY, connect to the EYESY via a web browser, upload your program as a zip file, unzip the file, and then delete the zip file. You then have to restart the video on the EYESY to see if the patch works. If there is an error in your code, the program won’t load, which means you cannot trouble shoot it, you just have to look through your code line by line and figure it out. Although, I learned to use an online Python compiler to check for basic errors. If you have a minor error in your code the EYESY will sometimes load the program and display a simple error message onscreen, which will allow you to at least figure where the error is.

I’m very pleased with the backing track, and given that it is my first program for the EYESY, with the visuals. I’m not super pleased with the audio from the Organelle. Some of this is due to my playing. For this experiment I used a very limited set of pitches in my improvisation, which made the fingering easier than it has been in other experiments. Also, I printed out a fingering chart and kept it in view as I played. Part of it is due to my lack of rhythmic accuracy. I am still getting used to watching the screen in PureData to see what meter I am in and what the current beat is. I’m sure I’ll get the hang of it with some practice.

One fair question to ask is do I continue to use the WARBL to control the Organelle if I consistently find it so challenging? The simple answer is that consider a wind controller to be the true test of the expressiveness of a digital musical instrument. I should be able to make minute changes to the sound by slight changes in breath pressure. After working with the Organelle for nine months, I can say that it fails this test. The knobs on the Organelle seem to quantize at a low resolution. As you turn a knob you are changing the resistance in a circuit. The resulting current is then quantized, that is its absolute value is rounded to the nearest digital value. I have a feeling that the knobs on the Organelle quantize to seven bits in order to directly correspond to MIDI, which is largely a seven bit protocol. Seven bits of data only allow for 128 possible values. Thus, we hear discrete changes rather than continual ones as we rotate a knob. For some reason I find this short coming is amplified rather than softened when you control the Organelle with a wind controller. At some point I should do a comparison where I control a PureData patch on my laptop using the WARBL without using the Organelle.

I recorded this experiment using multichannel recording, and I discovered something else that is disappointing about the Organelle. I found that there was enough background noise coming from the Organelle that I had to use a noise gate to clean up the audio a bit. In fact, I had to set the threshold at around -35 dB to get rid of the noise. This is actually pretty loud. The Volca Keys also requires a noise gate, but a lower threshold of -50 or -45 dB usually does the trick with it.

Perhaps this noise is due to a slight ground loop, a small short in the cable, RF interference, or some other problem that does not originate in the Organelle, but it doesn’t bode well. Next month I may try another FM or additive instrument. I do certainly have a good head start on the EYESY patch for next month.

Experiment 8: Bass Harmonica

For this month’s experiment I created a sample player that triggers bass harmonica samples. I based the patch off of Lo-Fi Sitar by a user called woiperdinger. This patch was, in turn, based off of Lo-Fi Piano by Henr!k. According to this user, this was based upon a patch called Piano48 by Critter and Guitari.

The number 48 in the title refers to the number of keys / samples in the patch. Accordingly, my patch has the potential for 48 different notes, although only 24 notes are actually implemented. This is due to the fact that the bass harmonica I used to create the sample set only features 24 pitches. I recorded the samples using a pencil condenser microphone. I pitch corrected each note (my bass harmonica is fairly out of tune at this point), and I EQed each note to balance the volumes a bit. Initially I had problems as I had recorded the samples at a higher sample rate (48kHz) than the Organelle operates at (44.1kHz). This resulted in the pitch being higher than planned, but it was easily fixed by adjusting the sample rate on each sample. I had planned on recording the remaining notes using my Hohner Chromonica 64, but I ran out of steam. Perhaps I will expand the sample set in a future release.

The way this patch works is that every single note has its own sample. Furthermore, each note is treated as its own voice, such that my 24 note bass harmonica patch allows all 24 notes to be played simultaneously. The advantage of having each note have its own sample is that each note will sound different, allowing the instrument to sound more naturalistic. For instance the low D# in my sample set is really buzzy sounding, because that note buzzes on my instrument. Occasionally hitting a note with a different tone color makes it sound more realistic. Furthermore, none of the samples have to be rescaled to create other pitches. Rescaling a sample either stretches the time out (for a lower pitch) or squashes the time (for a higher pitch), again creating a lack of realism.

The image above looks like 48 individual subroutines, but it is actually 48 instantiations of a single abstraction. The abstraction is called sampler-voice, and each instance of this abstraction is passed the file name of the sample along with the key number that triggers the sample. The key numbers are MIDI key numbers, where 60 is middle C, and each number up or down is one chromatic note away (so 61 would be C#).

Inside sampler-voice we see basically two algorithms, one that executes when the abstraction loads (using loadbang), and one that runs while the algorithm is operating. If we look at the loadbang portion of the abstraction, we see that it uses the object soundfiler to load the given sample into an array. This sample is then displayed in the canvas to the left. Soundfiler sends the number of samples to its left outlet. This number is then divided by 44.1 to yield the time in milliseconds.

As previously stated, the balance of the algorithm operates while the program is running. The top part of the algorithm, r notes, r notes_seq, r zero_notes, listens for notes. The object stripnote is being used to isolated the MIDI note number of the given event. This is then passed through a mod (%) 48 object as the instrument itself only has 48 notes. I could have changed this value to 24 if I wanted every sample to recur once every two octaves. The object sel $2 is then use to filter out all notes except the one that should trigger the given sample ($2 means the second value passed to the abstraction). The portion of the algorithm that reads the sample from the array is tabread4~ $1-array.

Knob 1 of the Organelle is used to control the pitch of the instrument (in case you want to transpose the sample). The second knob is used to control both the output level of the reverb as well as the mix between the dry (unprocessed) sound and the wet (reverberated) sound. In Lo-Fi Sitar, these two parameters each had their own knob. I combined them to allow for one additional control. Knob 3 is a decay control that can be used if you don’t want the whole sample to play. The final knob is used to control volume, which is useful when using a wind controller, such as the Warbl, as that can be used to allow the breath control to control the volume of the sample.

The PureData patch that controls the accompaniment is basically the finished version of the patch I’ve been expanding through this grant project. In addition to the previously used meters 4/4, 3/4, and 7/8, I’ve added 5/8. I’d share information about the algorithm itself, but it is just more of the same. Likewise, I haven’t done anything new with the EYESY. I’m hoping next month I’ll have the time to tweak an existing program for EYESY, but I just didn’t have the time to do that this month.

I should have probably kept the algorithm at a slower tempo, as I think the music works better at a slower tempo. The bass harmonica samples sound fairly natural, except for when the Organelle seems to accidentally trigger multiple samples at once. I have a theory that the Warbl uses a MIDI On message with a velocity of 0 to stop notes, which is an acceptable use of a MIDI On message, but that PureData uses it to trigger a note. If this is the case, I should be able to fix it in the next release of the patch.

It certainly sounds like I need to practice my EVI fingering more, but I found the limited pitch range (two octaves) of the sample player made the fingering easier to keep track of. Since you cannot use your embouchure with an EVI, you use range fingerings in order to change range. With the Warbl, your right hand (lower hand) is doing traditional valve fingerings, while your left hand (upper hand) is doing fingerings based upon traditional valve fingerings to control what range you are playing on. Keeping track of how the left hand affects the notes being triggered by the right hand has been my stumbling block in terms of learning EVI fingering. However, a two octave range means you really only need to keep track of four ranges. I found the breath control of the bass harmonica samples to be adequate. I think I’d really have to spend some time calibrating the Warbl and the Organelle to come up with settings that will optimize the use of breath control. Next month I hope to create a more fanciful sample based instrument, and maybe move forward on programming for the EYESY.

Sabbatical: Week 11 Update

This week I managed to mix and incorporate the Jetliner String Quartet recording for 737. I also was able to record eight phrases on my electric cello. All in all, I recorded two phrases for each of the following: 737, DC-8, 707, & 747. While I haven’t played cello regularly in nearly 40 years, I find that I am slightly better at it than playing the trombone. I’m still not particularly good at playing the instrument, but if you slap a bunch of effects on it, it does sound nice and spacey.

I’ve also been re-editing the string orchestra samples. One of the first things I did for the Rotate project was to add samples of Musiversal’s Budapest String Orchestra that I had recorded for my previous album. These samples were added right after the backing tracks were recorded. Accordingly, I added a lot of them, and now that the recordings are getting kind of thick, I want to thin out the string orchestra samples so they do not compete as much with the string quartet recordings. I managed to thin out TriStar and 737 in this manner. All in all, it was a decent amount of work accomplished for a week in which I was driving to tech rehearsals in Andover, MA for more than half of the week. It puts me a bit ahead of the game in terms of what I hope to accomplish next week.

Since I’ve been posting teasers related to the next album project over the last couple of updates, I’ll share a bit more. I’m pleased to announce that I have working algorithms for two of the six movements that I plan on recording the backing tracks for this coming summer. At the rate I’m crafting these algorithms, I could be ready to record those backing tracks sometime in early 2024. Regardless, I will start sharing examples from these algorithms in early in the new year.

As I posted a link to the new mix of DC-8 featuring the Jetliner String Quartet record, I’ll repost the string score for the movement. This is the only movement that uses quarter note arpeggiations. I’m also fond of the D# diminished chord over an E pedal at rehearsal C. I think it’s a particularly tasty harmony.

Sabbatical: Week 10 Update

I had hoped to post this on Friday, Saturday, or Sunday, but it has been a busy time. The good news is that I got more work done than I had expected to. I mixed and incorporated the string quartet recordings for eight of the nine movements. Some of the movements had multiple usable takes, so in some instances I chose to double (or even in 1 instance, triple) track the string quartet recordings to thicken things up a bit. Ultimately, I was able to add string quartet recordings to TriStar, A300, DC-8, 727, 707, DC-10, DC-9, and 747.

This will be a busy week for me as I am in tech week for a production of A Wrinkle in Time up in Andover. That being said, I expect to be able to complete the final string quartet mix, and to be able to get started recording some electric cello, which will put me a bit ahead of schedule. Since I have little to share this week, I’ll share a bit more about my next album project that I unveiled last week.

My plan is to have the album consist of 18 tracks, which should make a good middle ground between ME7ROPOL17AN 7RANSPOR7A71ON AU74R17Y‘s lots of short tracks approach, and Rotate‘s a few long tracks approach. As is the case with Rotate, the drum machine and synth parts will be generated by algorithms written in PureData. However, there will be three different broad models for these algorithms, so that this forthcoming album will feature more variety. The plan is to record the backing tracks to 6 of the movements during the summer of 2024, another 6 (using different algorithms) during the following summer, and a final 6 movements (using a third set of algorithms) during the summer of 2026.

Since I’ve released the recording of TriStar featuring the string quartet recording, I’ll re-share the score for the quartet for those who want to follow along . . .