Mutual Redundancies and Entropy in Online Discussion and Music

There is a problem when analysing information transfer insofar as a particular 'item' of information that is to be transferred must be identified first. This might then be used as an index of the success of failure of a particular communication network. However, when we communicate, it may not be the transfer of particular items of information which is the best measure, but rather the coordination around shared redundancy. Where shared redundancy can be measured, then this might provide a better way in which expectations are managed.

Measuring redundancy is easy enough. We can look at the maximal lossless ecoding of a message and compare it to its uncompressed version to see those things which are most redundant. In a Huffman code, the lowest value items (which occupy the least number of bits) will be the most redundant. Where the redundancy relation between two speakers online is a set pattern, then we might say there is no information transfer. However, there are moments when the redundancies change. What's important in this process is the process of chunking sections to find those section where the redundancies change. These are the most significant moments.

When redundancies change, it is a sign that something new has been discovered. There is a sign that there is a restructuring of expectations. I think this is because there is a relationship between expectations and  redundancy. Basically, I think redundancy serves to prolong (or retain) expectations. Expectation is not only a synchronic feature (a structured set of probabilities); it is a living diachronic  process. The thing that keeps the diachronic aspect of expectation alive is redundancy.

As with any new idea like this, I think "what does this mean in music?" Basically, I am talking about the relationship between a melody (which can be thought of as a set of expectations) and its accompaniment. The accompaniment serves to prolong the melody. Most accompaniments generate redundancy (think of the Alberti bass). The redundancies punctuate the expectations of the melody to keep it alive (see below)

What about in a dialogue? How might we spot the melody? How might we spot the accompaniment? Redundancies are easier to spot than expectations. In fact, if we can see the redundancies, then what isn't redundant can be seen to be prolonged. At a very simple level, text analysis tools like http://www.wikisummarizer.com/ illustrate the point nicely. By using a simple entropy calculation on Wikipedia text, wikisummarizer is able to pick out passages that are of key importance. Everything else punctuates it...

Now the key question is whether such techniques can tell us something about online discussion list communication (or even, large-scale discussion within MOOC forums). The key issue is where expectations change. This might occur where a particular set of expectations entertained by one person and prolonged by one set of redundancies, becomes transformed as a set of similar expectations prolonged by the redundancies of another person.

How could we test for this? Well, identifying the redundancies of an individual's utterances is fairly easy. Guessing the set of expectations might be possible by looking for common patterns. But what is most important is examining the relationship between the set of expectations and the set of redundancies. Where these is  a common pattern, where the pattern changes for each person, these are (I think) the moments we should look for in the data.

In order to think about this clearly we need some sort of concrete example to play with. Music fits the bill most effectively. It appears that different aspects of redundancy (motif, rhythm, harmony, tonality, dynamics, articulation, etc) create different prolongational and expectation structures. It is through the interplay of these that expectations shift. As they shift, so we can talk about 'information transfer'. In an online discussion, the lack of redundancy means that expectations shift  less easily. New redundancies have to be generated by individuals consciously. If no new redundancies are created, there is effectively no communication. There is more redundancy in face-to-face communication, much of it unconscious.

But it will take another post to look at this in more detail....

Notes on Abstraction, Information theory and Stories

I want to explore three statements about information theory I made in a post last week (see http://dailyimprovisation.blogspot.co.uk/2013/04/the-pattern-which-connects.html). They are:

1. "abstraction is the removal of redundancy from the flow of experience" 
2. "learning an abstraction is a process of recreating the redundancy that was removed in the abstracting process" 
3. "teaching is the process of creating the conditions for the production of redundancies related to a particular abstraction"

The statements are guesses. I find them tantalising because they present the prospect of a more analytical approach to learning. Because online learning presents many measurable factors, it is particularly amenable to this kind of analytical approach (although I will stop short of saying 'quantitative' - that would be going too far!).

There is a complex relationship between abstraction and cognition. Our daily existence would be impossible were if not for the fact that we create abstractions and largely coordinate ourselves through the world with them. Sometimes we believe  our abstractions more than the evidence of our own eyes, ears or hearts - and then we find ourselves in a kind of 'false consciousness'. The abstracting process, whatever it may consist of, is fundamental. In the cybernetics of the Viable System Model, abstraction in this sense is simply an attentuation process as part of an effort to maintain requisite variety.

I don't think its going too far to suggest that learning is essentially a process of engagement with abstractions. Most academic knowledge (the stuff that we go to school to learn) exists in essentially abstract forms that we call "subjects". In the process of learning, we often have to abstract from the experience of engaging with subjects - for Bateson this would be 'learning to learn'. And Bateson is probably right that the process of abstracting from the flow of experience leads to "levels" of learning. (Although I think I might prefer 'stratification')

But the question is "what's driving the show?" Could it be Ashby's (and Beer's) concept of "Variety"? In this sense, Beer's ideas of amplification of variety might amount to 'reproducing redundancy from abstractions'; attenuation is the process of abstraction. But I think this is too flat. The stratification of abstractions and the way that variety appears to get 'chunked' at the higher levels of learning do not, to my mind, get satifactorily explained by this process. Neither does the production of new abstractions to account for new experiences. Finally, the production of novelty seems poorly accounted for.

For these reason, I think looking at redundancy itself makes more sense. In Shannon's theory, redundancy seems to have at least two meanings: on the one hand there is the idea of 'excess bits' to carry messages; on the other hand, there is the simpler notion of repetition of messages. At some level, these may be the same - although they do not immediately appear to be related. But the point is that redundancy seems to do something.

If I continuously repeat a message to somebody, they are likely to get bored. What is boredom? My first inclination is to say the boredom is a response to low variety, and to say that my "variety" must be low if I am simply repeating the same message over and over again. But now I'm not so sure. Indeed, all human beings probably have very high variety (a large number of possible states) irrespective of whether they say the same thing over and over again or not. The boredom is more likely to be poor variety management - between myself and my environment. I am in oscillation - I don't know what to do because the complexity is overwhelming. I see many students like this!

But the redundancy produced in oscillation is important. The unmanaged variety in a sender that might produce the redundancy also might produce unmanaged variety in receivers. In this way it may be possible to say that the redundancy has a causal bearing in a social dynamic which increased the probability that some novelty emerges which serves to "mop up" the unmanaged variety - the mopping up is done by new concepts which change expectations within the social dynamic. Redundancy causes expectations to be attenuated in such a way that variety is unmanaged.

When expectations are attenuated because of high redundancy then it is unlikely that there is much information transfer between people. It is interesting that the idea of 'attenuated expectations' is highly compatible with the notion of absences in a metagame tree. The high redundancy produced results from the few strategies that are available for inspection - this becomes a self-perpetuatng mechanism. But eventually, there must be some 'determination' of absence. And it is at this moment that expectations are adjusted. At this moment, there is high information transfer.

Sometimes I find that I become too abstact. At that moment, it is useful to 'tell stories' (I often do this through music). What happens here is the deliberate production of redundancy. My blogging activity often has this form: most of what I write is redundant!! But through the telling of stories, through the production of redundancy there is a process by which the "deep absence" can be identified. The story itself is in itself a flow of redundancy and abstraction. Like the play within a play, this appears to be a necessary process to keep the wheels moving.

The brief retreat into a fantasy world is the move that I need to find the thing that is missing from the 'real' world. Is our response to unmanaged variety to "generate redundancy?"

Trades Unions and Education: Some fundamental questions

There is little doubt that the government's "reforms" of higher education have caused a terrible mess. The characteristics of the mess are confusion, competing agendas, loss of clarity of purpose and pockets of managerial opportunism which exploit all the distractions. It is a terrible situation for everyone. For staff, the threat to jobs which they had imagined would see them to a reasonably well-funded retirement mean that expectations dramatically have to readjust to the circumstances. Vice Chancellors find themselves in a catch-22 situation: I imagine that few of them in their ambitions for running a university would ever have forseen themselves having to disembowel their institutions in this manner. Whatever staff may think, this is an unenviable task. Faced with this, VCs have tended to react to the unpleasant job by surrounding themselves with (often sufficiently unpleasant) people to carry it through. There is little that is particular in this situation that distinguishes any one institution from any other: the pattern tends to be the same (as indeed it is in the corporate sector when faced with difficult choices).

The targets of cuts are academics. That's because they are very expensive. Their salaries used to be competitive because there was a market (supported by the state) for lecturers in Higher Education. But the state support has evaporated, the market disappeared, and individuals left on relatively high salaries (for the education sector as a whole) as a time of downward pressure on salaries generally and increasingly high unemployment. Moreover, salaries which used to be funded through general taxation, are now funded directly by the students themselves. This all leaves academics vulnerable, particularly as there is no shortage of young enthusiastic post-docs who are keen to get their first teaching experience for a fraction of the salary.

Faced with this, the Trades Unions are in a difficult situation. In traditional industries, Trades Unions represented the workers' interests by forming collectives whereby management couldn't bully individuals or make changes to working conditions by sleight of hand manoeuvres on the shop floor. The nature of the industrial situation within which collectives formed was that each individual was in themselves dispensible and that any individual could be replaced easily. The collective prevented the easy replacement of the individual by threatening a strike.

'Replacement' of individuals in such an individualised job as education is not the same as replacement in coal mining. In Universities particularly, each individual establishes their own market value through individual reputation, publication, project funding, etc. This means that the collective is never equal. Whilst the avowed intent of the collective is to protect the members' interests as a group, some members will inevitably see their interests served best by the disappearance of other members!

But in education, there is a further complication. Whilst the Union is meant to serve the members' interests, arguments for doing this will be based on 'protecting the future of education'. However, it would not follow that the future of education is best served by all the members keeping their jobs. By any logic, with tightened budgets, all members cannot possibly expect to keep their jobs (or at least their current salary levels). This sets up a contradiction: serving the members interests will put the future of education in a more perilous position, and indeed the future generation of students who will be left with the bill for protecting the members interests.

In terms of "protecting the future of education", management may well claim the higher ground, arguing that it is not for members of staff to profiteer from students, who will be beholden to debt for at least the next 20 years. They will argue that education has to be affordable, that technologies must be allowed to play their role in making learning and teaching efficient. All of these things stand to threaten the 'members interests'.

So, are Trades Unions in education about "Protecting their members' interests" or are they about "Protecting the  future of education"? Here we can ask "What if Trades Unions were specifically dedicated to protecting education? What would they do?"

The Union's strength is that it represents the voice of the staff. In an economic crisis, the dynamics of organisations are remarkably similar. Typically, management goes deaf. The classic recent case is the HBOS failure (see http://dailyimprovisation.blogspot.co.uk/2013/04/hbos-bad-decisions-and-fear.html). The fear of making the wrong decision creates the conditions for making the wrong decision. In situations like HBOS, management becomes increasingly allergic to criticism; it prefers to hears positive feedback from those close to it (in whose interests it is to provide positive feedback!!); it will actively avoid contact with those on the ground who know the  organisation better, and who might criticise current policy. These are the classic pathologies of communication in organisations in crisis.

Making the voice of the staff heard in these circumstances requires great skill. Shouting and berating management usually only stimulates the antibodies. What is needed is good teaching. The collective knowledge that the unions represent is vital to good decision-making. But it needs to be presented in a way which is unthreatening to those who will otherwise ignore it and make worse decisions. At the same time, a pedagogical approach can help in the other direction - for members of Unions to appreciate the gravity of the situation faced and the importance of careful positioning. Whatever suspicions exist concerning the probity of managerial conduct, it is unlikely ill-doing will be exposed through confrontation. And if there is wrong-doing, the logic upon which it is based can be properly inspected and shown to be counter-productive. It is in the interests of everyone that this is done carefully.

Most importantly, a pedagogical approach is precisely about "protecting the future of education". For it is with effective teaching and learning in the management of Universities, as with their practice and mission, that there is the guarantee of good decisions. And it is good decisions that will ultimately serve to protect the education system for future generations.

Why am I so slow? (or "redundancy to the rescue?")

I've been writing a particular paper for a very very long time. Since the inception of the idea of the paper (which is written with a friend), it has taken nearly 2 years! My friend is very quick, and I have been incredibly slow. I simply haven't been able to do it any quicker - and that makes me think about my experience, and why I have felt unable to move any quicker. [I have to say that other friends of mine with very prestigious careers will say "2 years is nothing!"]

In many creative endeavours, it is important that time passes. 'Wasting' time serves a function. For what appears to be a waste seems to prepare the ground for the moment when something can finally be 'put to bed' and something new is produced.

Given my thinking about redundancy in my last few posts, I'm interested in the business of 'wasting' time. I think that time wasted is time spent on "manufacturing redundancy". If we consider that redundant data, or 'absence', is causal, then the production of redundant data will eventually have an effect.

Of course, there's a lot of a different kind of redundancy around at the moment. People with no jobs. The experience of loss, and more importantly the fear of loss is everywhere. "How will I pay the mortgage?", "Will I have to move?", "what about my pension?" and so on. What is the fear really? Would we be so scared if we knew our lives would change, that others lives would change around us, that we need to keep together? Would we be so scared if a new war destroyed our cities? I think it would be a different kind of fear. We would probably be terrified about the threat to life of those we love. But the loss of homes, loss of jobs is not so dissimilar in a war to the kind of economic readjustment we are living through at the moment. But what is missing now is a sense of solidarity - war affects everyone. What is dehumanising now is the sense that individuals are 'picked off' and cast out of the community. The fear is of rejection and alienation - and it appears that it might strike anywhere. Perhaps this is closer to the kind of existential fear that Camus wrote about in "La Peste"...

But how have we made ourselves vulnerable to being 'picked off'? Answer: we half did the job before the crisis hit! We alienated ourselves, we lived inauthentically, we behaved in a dehumanising way, and we were often only too happy to see other people 'picked off'. But as we treated each other, we treated ourselves. We became alienated, and as we did, we became more vulnerable.

But this is really about waste. When the redundancy letter actually arrives, it means changes. It obviously means waking up and not going to work. It means lots of daytime telly. It means time passes without any sense of achievement. It means worrying about time passing without any sense of achievement (or income!). But perhaps most importantly, it means having fewer people to talk to, and having nothing to talk about.

But is the passing of time without any sense of achievement really a problem? I doubt it. If it is characterised as a collapse in the social network, what actually happens is an attenuation in the bandwidth of one's communication channels. That means redundancy of communication goes up (we say the same thing to fewer people). But it is all part of the transition. It is part of the process whereby creativity gradually takes root again in a new way. It is not the employed who change societies in an economic slump; it is the unemployed.

But, as with writing my paper, the death-trap with wasting time is worrying about wasting time. There is a need for a change of focus on individual achievement (which is part of inauthentic living and alienation) towards a greater sense of being within the flow of history. That's where the unemployed are are at the moment. That's where the 'time wasters' could be if they stopped worrying about wasting time. In the long run, it may be a better place to be.

Grounds for Critique in the "Hole in the Wall Experiment"?

Donald Clark recently wrote a critical post about Sugata Mitra's Hole in the Wall experiments (see http://donaldclarkplanb.blogspot.co.uk/2013/03/sugata-mitra-slum-chic-7-reasons-for.html) This prompted a vigorous discussion, which Sugata Mitra participated in. There was a considerable amount of academic ping-pong, whose objective appeared to be to defend the rightness of one position against another. I found all of this rather depressing - but perhaps not for the reasons that Mitra did.

The problem with the debate was that only ground upon which the argument was based appears to be individual ego differences ("I'm right"). Never is there any attempt at critical inquiry in looking at what happens (or appears to happen) and then to ask "why?" Instead, evidence is produced (empty holes in the wall) which appear to conflict earlier evidence and the claims of Mitra. Yet, since Mitra's underlying theoretical concern is "self-organising behaviour in learning", the empty holes in the wall hardly refute the theory of self-organising behaviour. Indeed, they rather add a bit more spice to the mixture. What might this mean? How might it change what we originally thought? How might our theory need to change? It's unfortunate that many people find thinking hard and unpleasant, and where the thinking gives up, ego kicks in.

The empty holes in the wall are telling us something. Personally I wasn't particularly satisfied with Mitra's original conception of self-organisation (I discussed this with him). In particular, viewing the technology as having causal powers per-se in producing the behaviour of the children seemed teleologically-focused. I doubt technologies have purposes. But I think people have agendas, and their agency with technologies reflects this. (Indeed, I have a friend who vehemently denies the reality of technology, preferring to simply call  it an "artefact" - he may be right)

The agency behind the hole in the wall was Sugata Mitra. He established the conditions for a self-organising activity. There is something complex in the way that we conduct activities with people: without the continual care and coordination, things tend to fall apart. Mitra's computers were his means of coordinating the activity. My view is that his agency was mediated through them. The children's engagement was a function of the technology (which framed the activity) and the agency which coordinated it. Take the coordination away, and things will fall apart (in cybernetic terms, they go into oscillation). In effect, Mitra taught the students. But he taught in a very imaginative way. An experiment to be applauded, in my view.

But there's much more to get out of this - particularly about the mechanisms of activity and self-organisation. One of the mistakes in thinking about the hole in the wall is that it is an addition to the environment - some additional complexity which generates self-organisation. This isn't correct, in my view. Indeed, it is the opposite. The computer is an attenuator: it attenuates the sheer diversity of the childrens' daily concerns and focuses them on a particular task (playing with the computer).

Most learning activities are attenuations of one sort or another - indeed most games (think of the attenuation of table tennis!). But what does the attenuation do? Why is it that the attenuation of a learning activity sponsors creativity and self-organisation?

I can only speculate on a possible answer to this. It has to do with redundant action - those actions which are simple, repetitive and recursive. A high-variety channel can carry many different signals - the probability that one signal is the same as another is low. A low-variety channel carries only a few signals. The probability that signals recur is higher. Recurring signals are effectively redundant signals. In the computer, the idea of 'clicking' is a good example. But redundancy serves a purpose. I think it catalyses new forms of communication.  This may be the mechanism that Mitra had uncovered. A low variety activity stimulated rich social interaction and new forms of communication. If the redundancy of the communication is catalytic on the production of new communications, then this process can be accounted for.

But the attenuator (the computer) had to be managed. Teachers manage the attenuators of their activities (they remind children of the rules). Mitra had to manage the attenuation of his computer. When this was no longer managed, what happens to all complex systems in their natural environment occurred:

Entropy.

The Pattern that Connects

Gregory Bateson's famous but somewhat mystical phrase raises as many questions as it attempts to address. In essence it is an appeal for holism - particularly the kind of holism that cybernetics specialises in: the reduction of the world to the interactions of recursive processes. In this sense, 'pattern' is an allusion to 'abstraction' - the description of a mechanism. Bateson himself created many abstractions: 'double description', 'double bind', 'levels of learning', 'schizmogenesis', etc. In his language, these things are seen to be patterns by virtue of the fact that they repeat themselves at different levels of recursion, and much of his work was focused on unpicking the presence of these patterns in biological, sociological, anthropological and mathematical/logical domains.

Holism, as understood by cybernetics, is precisely this identification of recursive repetition, or 'patterning'. The cybernetic abstraction is the 'explanatory principle' (another Bateson term) which is applicable at many levels: a "defensible metaphor", as Pask would say.

But I'm wondering if this is a particular understanding of holism. The question is "does this understanding of holism leave anything out?" I'm worried that it might.

My worry is grounded in a suspicion that the holistic explanatory aspiration of cybernetics cannot account for the personal desire to hold to a holistic explanatory principle. Erich Fromm wrote about this is "Haben und Sein". He points out that there is much in theological discourse that guards against the kind of Faustian ambition of being able to explain everything - to "have" an explanation. It's probably better to "be" an explanation (although that's hard to explain!) - it's the kind of thing that Jesus or Buddha attempted to get across.

It all comes back to our relationship with abstraction. I've struggled with this, and if I attempt to 'explain' abstraction, then I'm thinking of saying:

"abstraction is the removal of redundancy from the flow of experience"

But abstraction creates its own flow of experience. So no abstraction can be complete. We're into a territory that requires a kind of Cantor-like diagonal argument...

Abstractions have to be learnt by others. Indeed, if they are not taught and learnt, there is no point to them. So I'm also thinking of saying:

"learning an abstraction is a process of recreating the redundancy that was removed in the abstracting process" 

and what about teaching?

"teaching is the process of creating the conditions for the production of redundancies related to a particular abstraction"

but this is all getting a bit abstract!

It's very much like the relationship between music as played and music as notated. A performance is the process of creating redundancy from the abstraction of the score. Notation - which is the hard job of all composers - is the process of distilling experience into notated redundancies. [this is helping me think about my own difficulties in composition]

Bateson does talk about the relationship between classification and process (in "Mind and Nature" - where he reproduces this diagram from his anthropological work)

But the essence of the problem with abstaction (essence is another abstraction!) is that we lose sight of our own personhood, and the particular importance of "love" in being a person. As Faust realised, it is impossible to get beyond love, and whilst it is tempting to abstract it away (to remove its redundancy), to do this is to remove our humanity. If I say that "love is only redundancy" it reminds me of the damage that abstractions can do (although "loving" is partly "abstracting"...)

We cannot know the pattern which connects. But we can know the source of our desires to abstract knowledge. Knowing this, we might know ourselves and each other better. Then we might be in a position not to abstract further, but to do the opposite. To love the world more and to create redundancy.

Pattern

When we see a pattern, what we really see is redundancy.

A

is not a pattern. but

AAAAAAAAAAAAAAA

is.

all those As are not that dissimilar from the Fs here...

The one A (or the one F) can summarise the pattern. But if we say

A x 15

that doesn't have the same patterning as 

AAAAAAAAAAAAAAA

but...

A x 15, A x 14, A x 13, A x 12...

does!

(but at a different level)

Is the difference between abstraction and experience the difference between levels of redundancy?

When we abstract, we remove redundancy. That's what happens in Schenker's analyses (this one of Bach's first prelude):

The key redundant component removed in all of this is:

With Schenker, it simply becomes a C-major chord. But if it was really just a C-major chord, it would be a very boring piece!

Redundancy shapes its form. Creation is a process of redundancy generation: ABABABABCABABBABAABABC... (and so on!)

How does this work? The question is about the relationship between redundancy and novelty: when things are redundant, they create the conditions for something new. 

Today, Spain's unemployment statistics are 27%. That's a lot of redundancy. Maybe that is also the condition for something new. (let us hope it is beautiful rather than ugly!)

I suspect that redundancy creates novelty through a kind of catalytic process (Terry Deacon has got Autocatalysis just about right, I think). With a lot of catalysts in the air, an unexpected reaction becomes more probable, suddenly sparking into life from nowhere. Something new causes a reorganisation of expectations: this is when we know something is meaningful. The new thing causes everything else to be re-cast. Then new redundancies can grow, and the process begins again. It's tension and release.

Schenker was right about layers though. This (I think) is the emergent irreducible stratification of experience. In this way, we remember motifs, tonalities, progressions, etc - they become irreducible components: almost like the "code" or the components of a grammar. The emergence of the code, and the expression of statements within it produces new absences.

There is a magical moment, however, when an old code is completely transformed into something different...