I'm writing a paper on music at the moment in which I'm using the (fantastic) Music21 analytical tools from MIT (see http://web.mit.edu/music21/) together with R text-mining. Having a tool like Music21 is a bit of a revelation: now we have musical scores encoded in XML (through Musicxml) and we have libraries for processing that XML (for example, to perform harmonic reductions, etc) it becomes possible to dig into aspects of musical experience methodically in ways which have previously relied on subjective judgement. The challenge it presents us is to understand the nature of what we might be able to do, and the meaning of any results we might gain from it.
The first challenge is the challenge of the 'corpus'. It does appear that corpus techniques work. Where the AI specialists have tried for years to establish sophisticated grammars for text translation, Google's brute-force corpus-based techniques continue to transform the world of linguistic translation. Related techniques like discrete wavelet transforms applied to continuous data like audio waves and video have meant that the openness of the web has been accompanied by new means of protecting copyright (not such a great outcome, but a technical achievement nonetheless!). Such techniques open themselves up to increasingly rich and sophisticated corpus-oriented algorithms. Might we be able to identify the patterns of communication in the optic nerve? or brain waves in deep sleep? is mind reading possible? Where I would be firmly sat at the skeptics end of the spectrum with regard to these topics a few years ago, I can now see that this is, after all, simply continuous data. And if our analytical techniques are delivering results with the continuous data of video streams, why not with brain waves? It would be reasonable to expect significant advances in the next decade.
The central question in this, however, is "what does it mean?" When we are looking at music, there is a corpus of documents (scores). These represent the collective identity of a musical culture. Music21 comes with a variety of corpuses, so for example, we can look over a range of different Bach chorales. What's the point in knowing that 50% of those chorales are in the major key? or that 2% contain a melodic leap of a minor 6th downwards? (I made the figures up of course!). One way of thinking about this is to say that the corpus represents a kind of 'cultural expectation'. It is an indication of established norms.
But music isn't about conforming to norms. If it was it would never have developed. Music is about expression, and that requires a somewhat elastic relationship with norms. Swinging far from the norm can either be a recipe for obscurity or revolution. But there are different levels of 'matching' that we can examine. Schoenberg's example is instructive. In his serial music, he still wrote notes - which are, on their own, the same notes that Bach wrote - just not in the 'right order' as Eric Morcambe would say! The emerging experience of listening to Schoenberg is one of feeling oneself pulled away from norms, from the sounds that would match the corpus of established music. But we are not left high-and-dry. We are pulled into a different world. The different world has a deep level 'aesthetic' matching to the corpus of established music. It appears to us in a new way. That is the nature of revolutions that catch-on (and Schoenberg's did - at least for 50 years or so).
In musical experience, there isn't just a normative corpus to which experiences can be matched. There is also the emerging corpus of the unfolding work itself. At the opening of a new piece (or an improvisation) there is nothing in the corpus of the work to compare with the broader corpus of music. So matching will take place with what is known outside the piece. But as the piece evolves, it establishes its own patterns, and the corpus of the piece may well become the equal of the broader corpus.
What matters in this process of matching the piece with the broader corpus is the examination and generation of possibilities. These are the redundancies that associate with a particular moment in music: the many different ways in which the same thing might be said. There are moments where there are thousands of ways in which the same thing might be said; there are other moments where there are only a few. We move from one to the other continuously. Music, ultimately, moves towards the moment where only one possibility is there. Then it stops.
My algorithms run slow at the moment. But the basics are there. I need a much faster machine. But there will be much much faster machines. What will they be able to do? This isn't just about music. It is about aesthetic experience. It is about emotion. And I cannot think of anything more important in the world (particularly the world of education) than emotion. If we could really analyse aesthetics, if we could probe emotion... what might the world be like?
The first challenge is the challenge of the 'corpus'. It does appear that corpus techniques work. Where the AI specialists have tried for years to establish sophisticated grammars for text translation, Google's brute-force corpus-based techniques continue to transform the world of linguistic translation. Related techniques like discrete wavelet transforms applied to continuous data like audio waves and video have meant that the openness of the web has been accompanied by new means of protecting copyright (not such a great outcome, but a technical achievement nonetheless!). Such techniques open themselves up to increasingly rich and sophisticated corpus-oriented algorithms. Might we be able to identify the patterns of communication in the optic nerve? or brain waves in deep sleep? is mind reading possible? Where I would be firmly sat at the skeptics end of the spectrum with regard to these topics a few years ago, I can now see that this is, after all, simply continuous data. And if our analytical techniques are delivering results with the continuous data of video streams, why not with brain waves? It would be reasonable to expect significant advances in the next decade.
The central question in this, however, is "what does it mean?" When we are looking at music, there is a corpus of documents (scores). These represent the collective identity of a musical culture. Music21 comes with a variety of corpuses, so for example, we can look over a range of different Bach chorales. What's the point in knowing that 50% of those chorales are in the major key? or that 2% contain a melodic leap of a minor 6th downwards? (I made the figures up of course!). One way of thinking about this is to say that the corpus represents a kind of 'cultural expectation'. It is an indication of established norms.
But music isn't about conforming to norms. If it was it would never have developed. Music is about expression, and that requires a somewhat elastic relationship with norms. Swinging far from the norm can either be a recipe for obscurity or revolution. But there are different levels of 'matching' that we can examine. Schoenberg's example is instructive. In his serial music, he still wrote notes - which are, on their own, the same notes that Bach wrote - just not in the 'right order' as Eric Morcambe would say! The emerging experience of listening to Schoenberg is one of feeling oneself pulled away from norms, from the sounds that would match the corpus of established music. But we are not left high-and-dry. We are pulled into a different world. The different world has a deep level 'aesthetic' matching to the corpus of established music. It appears to us in a new way. That is the nature of revolutions that catch-on (and Schoenberg's did - at least for 50 years or so).
In musical experience, there isn't just a normative corpus to which experiences can be matched. There is also the emerging corpus of the unfolding work itself. At the opening of a new piece (or an improvisation) there is nothing in the corpus of the work to compare with the broader corpus of music. So matching will take place with what is known outside the piece. But as the piece evolves, it establishes its own patterns, and the corpus of the piece may well become the equal of the broader corpus.
What matters in this process of matching the piece with the broader corpus is the examination and generation of possibilities. These are the redundancies that associate with a particular moment in music: the many different ways in which the same thing might be said. There are moments where there are thousands of ways in which the same thing might be said; there are other moments where there are only a few. We move from one to the other continuously. Music, ultimately, moves towards the moment where only one possibility is there. Then it stops.
My algorithms run slow at the moment. But the basics are there. I need a much faster machine. But there will be much much faster machines. What will they be able to do? This isn't just about music. It is about aesthetic experience. It is about emotion. And I cannot think of anything more important in the world (particularly the world of education) than emotion. If we could really analyse aesthetics, if we could probe emotion... what might the world be like?
I like 'probe emotion'. When I look at my personal (text based) corpus through the lens of R I'm not sure what it means, but it's telling a story of who I was, maybe it is telling me I am no longer the same person, in fact studying the corpus itself changes me. With analytical words of absolution does it matter 'what does this mean?'
ReplyDelete