Information theoretical approaches to Ecology in Music

Information theory provides an index of "surprise" in a system. It concerns the difference between what is anticipated and what is unexpected. So in a string of events which might read A, A, A, A, A there is a strong anticipation that the next event will be A. The Shannon formula reveals this because the probability of A in this string is 1, and log 1 is 0, thus making the degree of surprise 0. But if the string read A, A, A, A, Z then we get a much higher number: the probability of A is 4/5, and the probability of Z is 1/5, and their logs (base 2) are  -0.32192809488 and -2.32192809489. Multiply these by the probabilities we get:

(4/5 * -0.32192809488) + (1/5 * -2.32192809489) =

-0.25754247591 + -0.464385618978 = -0.721928094888

The problem with this approach is that it sees a sequence of characters as a single description of a phenomena that can be treated independently from any other phenomena. But nothing only has a single description. There are always multiple descriptions of this. This means that there are multiple dimensions of "surprise" which must be considered together when doing any kind of analysis - and each dimension of surprise constrains the surprising nature of other dimensions.

A musical equivalent to the A, A, A, A, A might be seen to be

But is this right? By simply calculating the entropy of the note C, this would give an entropy of 0. And so would this...

What if the Cs continued for hours (rather like the B-flats in Stockhausen's "Stimmung") - is that the same? No. 

A better way to think about this is to think about the interacting entropies of multiple descriptions of the notes. How many descriptions are there of the note C? Well, there are descriptions about the timbre, the rhythm, the volume, and so on. And these will vary over time, both from note to note, and from time t1 to time t2.. 

I've written a little routine in Python to pull apart the different dimensions in a MIDI file and analyse it in time segments for the interactions between the entropies of the different kinds of description (I'll put the code on GitHub once I've ironed-out the bugs!). 

Analysing the midi data produces entropies over time sections, which look a bit like this (using 2-second chunks):

These values for entropy for each of the dimensions can be plotted against one another (one of the beauties of entropy is that it normalises the "surprise" in anything - so sound can be compared to vision, for example). Then we can do more with the resulting comparative plots. For example, we can spot where the entropies move together - i.e. where it seems that one entropy is tied to another. Such behaviour might suggest that a new variable could be identified which combines the coupled values, and that the occurrence of that new variable can then be searched for and its entropy calculated. This overcomes the fundamental problem with Shannon in that it seems tied to a predefined set of  variables. 

Comparing the interaction of entropies in music can be a process of autonomous pattern recognition - rather like deep learning algorithms. But rather than explore patterns in a particular feature, it explores patterns in surprisal between different features: the principal value of Shannon's equations is that they are relational. 

The point of pursuing this in music is that there is something in music which is profoundly like social life: its continuous emergence, the ebb and flow of emotional tension, the emergence of new structure, the articulation of a profound coherence, and so on. David Bohm's comment that music allows us to apprehend an "implicate order" is striking. I realised only recently Bohm's thought and the cosmological thought of the composer Michael Tippett might be connected (Tippett only became aware of Bohm very late in his life, but expressed some interest in it). That's my own process of seeking cosmological order at work!