Thursday 26 February 2015

Statistical Rank Correlation and the Measurement of Political Ecologies

I'm working on an EU bid at the moment which is concerned with creating a platform for researching the social and political ecologies in institutions, companies and societies. Slightly ironically my department has been experiencing first-hand some of the pathologies which result when the ecologies of an institution get unbalanced or it becomes monocultural. However, this seems to be the order of the day in many Universities these days. Our job is to make sure that we put ‘these days’ behind us and hand on a more healthy, diverse and better-governed education system to our children. Anyway, rhetoric is fine, but what about the practicalities?

First of all, I think that in understanding political ecologies we have to understand the dynamics between people who are fundamentally different in all sorts of ways, and yet who find ways of working together and making decisions. In more technical language, this is the difference between the mutual information between people (the “working together”) and the flexibility, or dissipations whereby energy and information is lost to the environment - apparently wastefully. Whilst most universities seem to want to turn themselves into efficient bureaucracies, encouraged by government bureaucracies like the QAA, the best things in education (like the best things in life) are exuberant, playful and superficially wasteful. The key message of ecology is that waste isn't wasted: dissipations drive ecological dynamics because they are where growth comes from. Moreover, the most important analytical element is not the individual, but the relationship.

In a political ecology, dissipations are more prevalent in a diverse environment. This is partly because it is more wasteful to devote time and energy in maintaining one’s identity in the face of those who would challenge your identity. It highlights the fact that the relationships in a diverse environment are the driving force, not the attributes of individuals. I have been interested in finding analytical methods for identifying this kind of situation.

My first starting point was to look at ‘big data’. Big Data takes rather shallow snapshots of the behaviour of individuals (those aspects of behaviour which are exposed in brief engagements online) and aggregates them. Although what happens in this process is a homogenising of the individual, if we take enough homogenised individuals it appears that we can begin to identify the ‘important things’ or the ‘things that matter to (some) people’. When we see the results of techniques like Topic Modelling, we are amazed because they accord with the things that we individually believe really are important! Big data is a bit like the mirror on Snow White’s Stepmother’s wall: it will tell us what we want to hear. Despite being able to identify trends, Big Data can’t really expose the deep interpersonal dynamics because it abstracts away so much of the individual.

Can we get further if we assume that people know what’s important to them? Why don’t we just ask them, rather than trying to algorithmically calculate it (clever though it is!)? However, when we think about what’s important, what’s striking is that it is not a single thing, but a structure which connects the most important thing to the least important things via things which are semi-important. What’s important is an ordered list. Now, when we look at the relationships between people, and we look at the different things that matter to them, what we are really doing is comparing two ordered lists.
This is relatively easily done using statistical measures for rank correlation like Kendall’s Tau and Spearman's Rho. With each of these techniques (and they produce different results in different situations), we can calculate an index of the ranking of ‘important things’ between different pairs of individuals. What the index tells us is effectively how much people have in common. However, it also indicates something else: what the biggest point of dispute is.

For big disputes between individuals (i.e. priority rankings which are fundamentally different – particularly at the extremes), there will be considerable defence by each individual of their position. This effectively is a dissipative activity. What will the result be of these dissipative dynamics? Individuals will seek new opportunities to either reinforce their position within the institution, or look to move into a different social context where they might be more effectively integrated.

I'm thinking about this as I'm looking at what’s happening to my department. The mutual information between us and our host university is decreasing rapidly. The priority lists of “things which matter to us” are increasingly divergent. The dissipative dynamics cause growth beyond the bounds of the native habitat. Sometimes that can be a good thing - but it's natural and unnatural at the same time.

No comments: