Saturday, 12 September 2015

Entropy and Aesthetics: Some musical improvisation experiments

I've been doing some weird musical experiments recently. What started out as a very simple Javascript program to illustrate the calculation of entropy on-the-fly turned into doing a similar thing for musical notes recorded as MIDI signals. With text, the problem of calculating entropy is quite simple: you can look at individual letters and calculate their entropy (basically the sum of the logs of their probabilities) - it's a measure of the average 'surprisingness' of the text. You can also look at the 'digrams', or successions of symbols, and continue with trigrams and so on.

I initially started by looking at the notes sent down the MIDI cable to do the same thing as I had with text. But then there are so many aspects to music: the harmony, rhythm, intervals (which are perhaps more interesting and important than the MIDI value of the notes), dynamics, etc. Somehow all of this stuff works together to produce the aesthetic effect. But each of these elements has its own entropy value: there's rhythmic entropy, harmonic (or bass-line) entropy, and so on. So I elaborated my program to take these measurements as well. Then it struck me that some way was needed of relating each of these elements to one another. I feel somehow there ought to be a way of calculating what Shannon calls "mutual information"... although doing this with music is not at all like doing entropy calculations with text. Short of doing this, the most obvious thing to do is to indicate the differences between the different levels of entropy at different points in time. So that is what I've done in the first instance. It's very crude.

When I improvise, I watch the values changing and the relationships between the different elements of entropy shift. It's quite fascinating. What I am looking for, basically, is a correlate for how I feel as I play. But I've reached a stage now where I think this structure, with its oscillating connections, needs to grow over time: because that's what happens to me as I play. What is this growth?

Drawing on my previous post about 'learning gain', I think this growth is the emergence of order. But as Von Foerster said with regard to this, there is a process of growth that occurs within constraints, and there is a growth which occurs by expanding or changing the constraints. How would my model of improvisation identify its constraints?

In music analysis, one of the principal tasks is 'segmentation' of the music into parts that can be analysed: the architectural units of a symphony, for example. It is a bit of a mystery as to how musicians identify these segments. It may be that the identification of boundaries where one set of constraints gives way to another is what is going on here. Is it possible to write a computer program that does that? Currently, I haven't got anything that measures constraint at all: I am measuring entropy which is (kind of) the opposite - so that is my next task.

Perhaps its worth saying why I think this is important. The measurement of different levels of entropy is a useful technique applicable to all kinds of experiences. Imagine being able to measure the moves that a player makes in a computer game, for example; or the drawing process of an artist; or the behaviour of individuals when interacting with an artwork, or the use of a new website, etc. In each case there is emergent order within constraints that change. Music is a powerful domain for beginning with this kind of research because it has such a powerful emotional component.

It makes me think that the most powerful research domains are those where analytical endeavour is most entangled with human emotional responses.


dkernohan said...

That's *fascinating* Mark!

Apologies if I've missed a point in the explanation, but is "note entropy" measuring against a particular scale (i.e. how "weird" the note is in relation to the identified harmonic centre of the improvisation)?

So in G major, D would be pretty boring, A slightly less boring, F natural moderately interesting and Ab very interesting indeed...

Not sure. Would love to play and watch those numbers though....

Mark Johnson said...

Hi David,

I'm using a 'sliding window' of note values (this uses a window size of 20 events), so everything is relative to what's gone on before within the window. The interesting thing is that if I play in the key of C major, quickly the entropy settles on a set of pitches (do you remember pitch class set theory? - I was very unconvinced by it at university, but now it becomes useful!). Now if I play an F#, this will be a surprising event, and immediately be recognised by the entropy value rising. Over time, if I include the F# in my C major stuff (like a new modal adjustment), the entropy will settle back... until something else unexpected happens. Actually the interesting thing about playing an F# is that the constraint changes. I'm not measuring constraint yet, but that's what I need to do.

I could create symbols for scales and keys, and the shifts between the different symbols will also have an entropy. Haven't done this yet.... The interesting thing is that there are so many different measures to take, but it's possible to simply add more and more in and see what happens. I've been very influenced in this by the statistical ecology of Robert Ulanowicz. He said, in a mailing list post a couple of days ago, that:

"The ability to make [the] separation [between constraint and indeterminacy] has wide-ranging consequences. For example, it provides a pathway by which process philosophy can be brought to bear on quantitative physical systems! It is no longer necessary to rely solely on positivist "objects moving according to law". That's no small advance!"

I think it's not just quantitative physical systems; psychologists, educationalists and even music analysts adopted this physicalist "objects moving to a law" too. There's a real chance of being able to challenge them. And timely, if this "learning gain" thing is upon us....

(Of course I could be wasting my time and this is a fantasy. I should probably be applying for jobs before we are made homeless!!)