Evolving Music in Python (Tie-In to the Presentation at Ljubljana Python Meetup)

First a bit of a background. I am a cognitive science student, so that means that I study the mind from the perspective of multiple disciplines. Some people do it by studying how we experience things through interviews or descriptive experience sampling (phenomenology) or through the fMRI, EEG, TMS or any other way to measure or influence neural signals (cognitive neuroscience), or doing the work of defining the concepts (philosophy).

The perspective that for me is has the highest interest*skill is computational modeling. I am a lot better at it than any kind of qualitative studies, and it is a lot more interesting than measuring neural data.

So, when I was introduced to the computational modeling in one of my classes, I decided that I wanted to do a more practical project about it. So, when I took the evolutionary algorithms next semester, I sort of knew, that this may be a good opportunity to see, how much a like it.

But the problem was, that I had no idea, what sort of problems could evolutionary algorithms solve at the time. I know that I am interested in the decision making and individual differences, but I could not find any projects using evolutionary algorithms in personality. Any the only ones that I found, that were connected to decision making, were from evolutionary game theory.

So what I did, was ask my professor for ideas. She told me a couple of ideas, but all except one, I would be able to write in python in up to 30 lines of code. The one, that I had no idea, how to attempt, was creating music with evolutionary algorithms.

Which, considering that I am not even listening to music most of the time, may not have been the best idea. Domain knowledge and all of that. I did learn how to play the guitar for 2-3 years, when I was in primary school, but even back there, my teacher had problems with my utter lack of knowledge about what songs exist and which I like. It did not help, that I only took these lessons, because I heard, that it is impossible to become a teacher without playing an instrument. I never took a music theory anywhere.

So, in evolutionary algorithms, there are a couple of thing that are important: the representation, the parameters in the algorithms and the fitness function.

As far as the presentation goes... the standard one is to put the gray representation. It is supposed to be better than binary because of the hamming distance (how many numbers change, when going to the next integer) and more efficient in the use of memory.

The code to change from binary to gray code in below:

def gray_from_binary(number_binary):
    number_binary_list = [int(a) for a in str(number_binary)]
    number_binary_zip = zip(number_binary_list, [0] + number_binary_list[:-1])
    number_gray = [1 if number1 + number2 == 1 else 0 for number1, number2 in number_binary_zip]
    return int("".join([str(n) for n in number_gray]))

But since I had no problems with the memory, I decided to use integers. But then the bigger problem in representation still exist. How to encode music in numbers?

Thankfully, I remembered the lilypond project. Mostly because I use it to store the music that I know how to play in plain text files.

So I took the subset of it. I encoded the piano keys in integers. From c as 1 up to h.

self.notes_to_numbers = {"c": 1, "des": 2, "d": 3, "ees":4, "e": 5, "f": 6, "ges":7, "g": 8, "aes":9, "a": 10, "bes":11, "b": 12, "pause": 13}

The same is for duration. in lilypond the base of 2 is used. So 1 is a whole note, 2 is a half note, 4 is a quarter note,... So, when I needed to make changes, I simply took the exponents and then changed back to the notes. I only encoded the 'normal' notes.

The next is algorithms. There are a couple of things to decide here, but none of them were that interesting. I usually allowed the best x songs in the next stage, and I did the mutation of the duration and the pitch separately. I also made 'children' songs by randomly choosing a place and putting the remaining songs in each children.

The interesting part comes from the fitness function. What criteria was used to choose songs. I did a some tries of my own and I copied two from the scientific articles. That was mostly because, I don't know music theory. I mean, I had to check the what tonality is the first time it was mentioned. But when I started reading about it, I realized that none of it is written from the perspective of understanding music, that could be expressed mathematically.

What I used was checking for repetition of the same pitch or too big of a difference in it, difference in duration, for long notes, too many pauses, and whenever the notes could be given into a single cords.

The ideas that I stole for the papers were the following:

• The first was the minimal mathematical criteria, that would need to be true, to create listenable piano music by Rodney Waschka
• The second idea was about how pleasant the notes sound together, which was based on being consonance (harmonic or dis-harmonic) by Bilotta, Pantano and Talarico

Based on my ideas are functions cost_first_try and cost_c_tr, while the function costs based on articles are cost_article2_try1, cost_article2_try2 and cost_article1_try.

def cost_first_try(self, song, final_song=None):
    cost = 0
    #add the cost of two notes repeating the pitch
    cost = cost + sum([1 for note1, note2 in zip(song[:-1], song[1:]) if note1[0] == note2[0]])
    #add the cost of notes having too big of a pitch
    cost = cost + sum([abs(note1[0] - note2[0])**2 for note1, note2 in zip(song[:-1], song[1:]) if not abs(note1[0] - note2[0]) == 1])
    #add cost of notes having too big of a difference in duration
    cost = cost + sum([abs(note1[1] - note2[1]) for note1, note2 in zip(song[:-1], song[1:]) if abs(note1[1] - note2[1]) > 1])
    #add cost of long notes
    cost = cost + sum([1/note1[1] if not note1[1] == 0 else 2 for note1 in song])
    #add cost for cords
    for key in self.keys:
        if set([note[0] for note in song]) in key:
            cost = cost - 20
            break
    return cost

In the first fitness function, I simply included all the ideas that I could think off. I think in this example, my comments tell the story of what I was using pretty well.

def cost_c_try(self, song, final_song=None):
    cost = 0
    #add the cost of two notes repeating the pitch
    cost = cost + 10*sum([1 for note1, note2 in zip(song[:-1], song[1:]) if note1[0] == note2[0]])
    #add the cost of two notes repeating the duration
    cost = cost + 10*sum([1 for note1, note2 in zip(song[:-1], song[1:]) if note1[1] == note2[1]])
    #add the cost of notes having too big of a pitch
    cost = cost + sum([abs(note1[0] - note2[0]) for note1, note2 in zip(song[:-1], song[1:]) if not abs(note1[0] - note2[0]) == 1])
    #add cost of notes having too big of a difference in duration
    cost = cost + sum([abs(note1[1] - note2[1]) if abs(note1[1] - note2[1]) > 1 else 1 for note1, note2 in zip(song[:-1], song[1:])])
    #add cost of long notes
    cost = cost + 20*sum([1/note1[1] if not note1[1] == 0 else 2 for note1 in song])
    #add cost for pauses
    cost = cost + 10*sum([1 for note1 in song if note1[0] == 13])
    #add cost for keys
    cost = cost + sum([25 for note in song if note[0] not in self.keys[0]])
    #add cost for cords
    duration = 0
    current_notes = []
    for pitch, dur in song:
        if dur == 0 and duration != 0:
            cost = cost + 50
            break
        elif dur != 0:
            duration = duration + 1/dur**2
        if dur == 0 and duration == 0:
            duration == 1
        current_notes.append(pitch)
        if duration > 1:
            cost = cost + 50
            break
        if duration == 1:
            for cords in self.cords_c:
                if set(current_notes) in set(cords):
                    cost = cost - 10
                    break
        duration = 0
        current_notes = []
    return cost

My second try was pretty similar to the first, except, that I added my idea about the cords. My idea was, that if I test all the notes in the certain duration, and if in this duration, the notes can all be put in one of the cords, then this would be good, and it would be the sign of a good music. The idea for this come from the guitar playing with the cords. I don't think it added much to it, but you have an examples to listen to.

My third idea was from the first article, but since this one has horrible convergence, I am not going to go into it.

The forth and fifth ideas were taken from the second article. There the idea was that some notes sound good in combination and some sound bad. So what I did was test two neighboring notes for this.

def cost_article2_try1(self, song, final_song=None):
    #Based on Article: Evolutionary Music and Fitness Functions by E. Bilotta, P. Pantano, and V. Talarico
    cost = sum([1/((self.notes_values[note1[0]] + self.notes_values[note2[0]])/(self.notes_values[note1[0]]*self.notes_values[note2[0]])) for note1, note2 in zip(song[:-1], song[1:]) if note1[0] != 13 and note2[0] != 13])
    #add cost of long notes
    cost = cost + sum([1/note1[1] if not note1[1] == 0 else 2 for note1 in song])
    #add cost for pauses
    cost = cost + sum([1 for note1 in song if note1[0] == 13])
    return cost

The forth one was based on the mathematical formula, that I found in the article. I then added the cost for the pause, because if I would not, then half of the notes would be pauses. This may be because, I skipped testing notes, if one of them was a pause. After that, this was no longer a problem.

def cost_article2_try2(self, song, final_song=None):
    cost_function={1:{1:100,2:100,3:100,4:100,5:0,6:0,7:100,8:0,9:100,10:0,11:100,12:50,13:10},
                   2:{1:100,2:100,3:100,4:100,5:100,6:100,7:0,8:100,9:0,10:100,11:0,12:100,13:10},
                   3:{1:100,2:100,3:100,4:100,5:100,6:100,7:100,8:0,9:100,10:100,11:100,12:0,13:10},
                   4:{1:100,2:100,3:100,4:100,5:100,6:100,7:100,8:100,9:0,10:100,11:100,12:100,13:10},
                   5:{1:0,2:100,3:100,4:100,5:100,6:100,7:100,8:100,9:0,10:100,11:100,12:100,13:10},
                   6:{1:0,2:100,3:100,4:100,5:100,6:100,7:100,8:100,9:100,10:0,11:100,12:100,13:10},
                   7:{1:100,2:0,3:100,4:100,5:100,6:100,7:100,8:100,9:100,10:100,11:0,12:100,13:10},
                   8:{1:0,2:100,3:0,4:100,5:100,6:100,7:100,8:100,9:100,10:100,11:100,12:0,13:10},
                   9:{1:100,2:0,3:100,4:0,5:0,6:100,7:100,8:100,9:100,10:100,11:100,12:100,13:10},
                   10:{1:0,2:100,3:100,4:100,5:0,6:0,7:100,8:100,9:100,10:100,11:100,12:100,13:10},
                   11:{1:100,2:0,3:100,4:100,5:100,6:100,7:0,8:100,9:100,10:100,11:100,12:100,13:10},
                   12:{1:50,2:100,3:0,4:100,5:0,6:100,7:100,8:0,9:100,10:100,11:100,12:100,13:10},
                   13:{1:10,2:10,3:10,4:10,5:10,6:10,7:10,8:10,9:10,10:10,11:10,12:10,13:10},
                   }

    cost = sum([cost_function[note1[0]][note2[0]] for note1, note2 in zip(song[:-1], song[1:])])
    #add cost of long notes
    cost = cost + 20*sum([1/note1[1] if not note1[1] == 0 else 2 for note1 in song])
    return cost

In the last one, I uses the table, that they were using, instead of the mathematical function. But otherwise it was based on the similar principle then the upper one. Since the formatting of this function is not that great, you can also see it here.

Then I simply did a quick gui, because I knew that most people would not know what to do with the script. But in this way, I can actually package it. Not that I would know how to do this for anything else but Arch Linux.

In the last 30s, I wanted to present a sample of music created with one of the fitness functions. Since I did not ended up using computer and I later realized that my speakers are not actually loud. The examples can be found next to the explanation of each fitness function.

For anybody interested in more, the code for this project can be found on my github: https://github.com/sarajaksa/creating-songs-research-project

P.S. (added 2018-10-26): A friend of mine taped me, and sent me the video. You can find it here