Learning Analytics and Social Relations: A reorientation

Learning Analytics has become very popular in recent years. With increasing use of online courses and resources in education, it was perhaps obvious to consider how clicks on resources and words in submissions might be counted and aggregated so as to reveal some mysterious intentional forces lurking behind learner behaviour. It might even help prevent learners dropping out of courses, or help teachers design better courses.

Clearly, with online content, clicks and words do become countable. And with this countability comes a range of statistical resources which can detect average trends, probabilities, and so on (although an average learner is not easy to come across in reality!). However, in reality, such analysis often amounts to little more than "students who never log in fail", just as students who never turn up for class tend not to do very well either. Most teachers are likely to spot the trend quicker than an algorithm.

Part of the ambition of Learning Analytics is to identify causal relations for patterns of engagement. For example, by categorising particular pedagogical practices, those practices which lose most of the students can be identified and eliminated. Once again, however, this is both neither rocket science nor is it terribly reliable. Lectures can be brilliant or terrible, learning activities hit and miss. We might say, with a good teacher, the learners learn most from the hits, and the teacher learns most from the misses.

Part of the problem here is that it is unclear exactly what Learning Analytics is analysing. More fundamentally, Learning Analytics cannot analyse learning because nobody can see learning: we do not possess the capacity to look into each others heads. We can only speculate on learning processes. To do any more is to fall victim to an ideology.

If Learning Analytics does not analyse learning, what does it analyse? I believe the answer to this question can help us be more focused on the analytical tools at our disposal.

Learning Analytics analyses the constraints within which humans organise their learning. 

What does that mean? It means that clicks on a web page are indicators of constraint. Now, of course, constraint has to be defined. In Information Theory, constraint, or redundancy, is the background of information. Information, for its part, is the measurement of the surprisingness of a sequence of events. Those events may be words in a sentence, or letters in a word, or notes in piece of music or patterns on a carpet. Words in a sentence have the degree of surprisingness they have because of the structures which the words have to fit into: the grammar. Grammar is a significant constraint of language. Clicks on a web page have a pattern of surprisingness too: that pattern will partly be determined by the design of the web page. This in itself may be thought of as a kind of grammar. However, the design isn't separable from the content of the page, and other constraints regarding content also affect the pattens of clicks.

Constraints are difficult to separate. The design of a tool is hard to separate from the social context within which the tool is to be used (although crafty tech companies use clever design to trick people into ignoring other constraints - for example in pernicious legal agreements which we ought to spend more time reading before we click 'agree'). In an online course, the dominant constraints are social, not design-oriented. So counting clicks or counting words is an indication of social constraint.

In education, social constraints may be managed. Indeed, this is fundamentally what teachers do when they organise activities, or find different ways of managing conversations. It is unfortunate that with much current Learning Analytics, the search for causal relations has led to over-focus on where there is data, and general ignorance of where there is little data. The problem is that where there is little data, there is the most constraint! Whilst teachers will always work hard with the taciturn learner in the face-to-face setting to work out where the blockages are, online it's much easier to ignore them and focus on the 'success' of their pedagogy by looking at where the data is. This is probably the single biggest mistake of online learning.

If analytics was turned to analysing constraint, and questions asked about the constraints which are encountered, then I'm sure we would have far better online education than we do. There are ways of coordinating conversations online where constraints can be identified and moved if necessary. In order to do this, we need a much more precise and technical focus on what analytics does and does not mean. George Siemens recently asserted that the biggest problem with the current state of MOOCs is the inability to understand emotion:


I think the question about emotion wouldn't appear as surprising if our analytics was constraint-oriented, rather than cause-oriented. Emotions are always the biggest constraints of all.

Critical Realism and Relations

I stumbled across the fact that in the sociology literature, there are two distinct definitions of the term "we-relation". On the one hand there is the "we-relation" which I am familiar with - that of Alfred Schutz - who talked about it being the intersubjective relationship apparent in face-to-face conversation. Central to Schutz's idea is the sharing of a vivid simultaneity. Time is central to his conception. Indeed, one of the things I find fascinating about Schutz's intersubjective philosophy is that he sees time more generally through the eyes of a person. Time is constructed in the sense that it is created in the very process of movement and interaction with others who share the same sense of time passing. Since time is such a thorny sticking point in mechanistic philosophies, I find this subsumption of time within personal being very powerful - and convincing.

The other use of "we-relation" is by Margaret Archer and Pierpaulo Donati (see http://www.relationalstudies.net/uploads/2/3/1/5/2315313/the_relational_subject_(1).pdf ). This builds partly on Archer's theory of reflexivity and her broader approach to social morphogenesis. Donati argues that we

"derive from a relational context, [are] immersed in a relational context and bring about a relational context"

Donati and Archer's "we-relation" presents a delineated model where the boundaries between personal reflexivity and social relation are  clearly drawn. This kind of delineation is familiar in Archer's work on morphogenesis where the distinction between social structure and agency is drawn on the basis that there exist  concrete social structures of institutions, practices and so on distinct from the reflexivity and action of individuals who act to reproduce structure conditions, and whose actions are constrained by them. Archer has argued strongly against Giddens in whose structuration theory social structures are not considered real, but manifestations of a kind of collective intentionality. Searle's social ontology would meet a similar criticism. Archer calls these approaches 'elisionist'.

Archer bases her assertions on the discreteness of structure and agency, and the discreteness of social relations and personal reflexivity on the deeper  philosophy of Critical Realism which upholds the existence of two ontological domains: the intransitive domain which exists outside human agency, and the transitive domain which exists through human agency. Fundamental to this is the identification of causal mechanisms.

This is where the problem is and where I think Schutz's we-relation is most interesting. The problem is the relationship between time and mechanism. Quite simply, how can you have a mechanism without time? If our conception of time is intersubjectively created then the grounds for asserting the strict separability of structure and agency, or personal reflexivity and social relations starts to crumble.

We are left with an ontology with personal-being-with-others at its heart. 

Online Intersubjectivity and Constraint

Face-to-face conversations are different things from online conversations. Today I had an interesting conversation about conversation. I could look into the eyes of the person I was talking to and try to gauge what was happening in their 'inner world'. Each utterance I made carried with it a set of expectations as to how the person I was talking to would respond. If a response surprised me too much, I discovered I found it difficult to formulate another utterance - there were a couple of awkward silences! These moments were the most interesting for me. I found that I had to reflexively adjust my set of expectations in quite a fundamental way. I found I needed space to do this, and sometimes the intensity of a face-to-face exchange is not the most conducive to this kind of reflexivity.

Online, now, writing my blog, I think I can do it better. Moreover, I feel the need to do it. There's something about blogging which helps to resolve intellectual tension: it may be a kind of masturbation (which worries me sometimes).

Part of the discussion today concerned the we-relations that Schutz talks about. Fundamentally, the question is, Does being part of an online community constitute a 'we relation'? In Schutz, the pure we-relation is exclusively focused on the face-to-face interaction. Central to the pure we-relation is the shared flow of time: through this flow of time, “the reciprocal sharing of the other’s flux of experiences in inner time, by living through a vivid present together” occurs.

Modern technology provides new ways of accessing a 'vivid simultaneity': video is the most obvious example. New real-time technologies are continually emerging on the back of websocket technology. There is also a kind of vivid simultaneity in Facebook and Twitter - except that it extended across a span of time. Where's the common factor between these different kinds of activity? And what are the phenomenological differences between them?

Facebook and Twitter are not quite the same as Schutz's rather remote "world of contemporaries" (the term he uses to describe human relations with people who are real and alive but not present in front of us). In exploring the common ground of these phenomena, I am driven to the conclusion that it is the constraints of communication which matter. And yet much of the conversation today concerned the nature of constraint and absence, and whether constraint was merely a kind of causal power.

I once believed that constraint was a causal power, and argued about this with cyberneticians who disagreed. I now see constraints and causes as fundamentally different orientations in looking at the world - and the constraint orientation is helpful because it bypasses the problems of causal thinking including the inevitable tendency towards reductionism.

Face to face discussions are profoundly constrained by the myriad of sensory perceptions which are exchanged as we talk and look at each other. Choices of utterances, as with choices about anything, emerge through what can't be thought: forgetting and ignorance are often the drivers of decision. So to talk of a pure we-relation is to talk of a highly constrained situation where the constraints are simultaneous - one might call them synchronic. It is not necessarily the case that there are fewer constraints in an online conversation. But it is true that those constraints are less synchronic and more diachronic. A Twitter feed is the gradual exposure of diachronic constraint.

So can a we-relation exist online? Reframing the question, I would say the issue is whether a we-relation which rests of synchronic constraints is the same as a relation which rests on diachronic constraints. This is complicated because a synchronic we-relation is an inescapable constraint for both parties. A diachronic we-relation (if such a thing is possible) is dependent on different parties reading similar constraints in the flow of communications between them. There are fewer guarantees of effective 'tuning in'. However, if different kinds of media are used, including video, then there is more chance that exchanges can be more meaningful and insightful into the other's inner world.

Part of the condition for this relates to the constraints which contextualise the conversation in the first place. Peoples' bodies are constraints; their life histories are constraints; their emotions, socio-economic conditions, political views, and so on are all constraining. An online support group for people with particularly similar histories, personal tendencies, enthusiasms and so on contextualises interactions which are diachronically constraining, but which - because of shared common constraints beyond the immediate interaction - (maybe) can provide the conditions for mutual tuning-in. 

Critical Realism and Cybernetics: A struggle for a perspective on education

I've been trying to finish a book chapter on cybernetics and education. I've got stuck. The reasons why I've got stuck have to do with changes in my orientation not just towards cybernetics, but also to the philosophical perspective which I found valuable in conjunction with cybernetics, Critical Realism.

Both critical realism and cybernetics are, on the surface at least, concerned with mechanism. Critical realism asserts that what exists in the world are causal mechanisms which are, through scientific inquiry, discoverable. There are intransitive mechanisms which exist independently of human agency (mechanisms of physics, for example), and there are transitive mechanisms which exist through human agency. Transitive and intransitive mechanisms are interconnected at different levels where the discovery of mechanisms connects empirically derived knowledge of the transitive and intransitive domains with deep mechanisms of human flourishing: Critical realism connects science to politics. Cybernetics, by comparison, is a scientific practice of building models to explore mechanisms. It is often argued in cybernetics and systems thinking that the relation between a model and reality is one of isomorphism. Cybernetic science (it is argued) can proceed by exploring the isomorphisms, improving the models and making social interventions consistent with the models. Critical realism often criticises cybernetics for considering only 'closed systems' (with feedback), for idealism and scepticism about reality, for ignoring what critical realism sees as the polyvalent structure of mechanisms (so there are mechanisms of biology and physics (intransitive) which interact with transitive mechanisms of norms, politics, positions, rights, obligations, power and so on), and for an elision between the key domains (as critical realism sees it) of human agency and social structure. Despite all this, there are many good scholars who have found the connection between critical realism and cybernetics useful, including John Mingers, Soren Brier, Loet Leydesdorff, John Shotter and Terry Deacon (to name a few). Personally I found the conjunction of cybernetics with the Critical Realist methodology of Realistic Evaluation (Pawson and Tilley) particularly useful.

For my part, I have been more sympathetic to the critical realist position than the cybernetic position - despite the fact that I identify with the cybernetic practice of building models. I tended to see cybernetics as a variety of functionalism which whilst being useful for clarifying concepts, also held within it a pathological blindness to its political implication (something which is indicated in critiques by Horkheimer, Fromm, Ilyenkov and numerous others). At the same time I was also disturbed by the dogmatism of many critical realists, and their lack of criticality concerning their own theories.

Over the last year or so - a period which also saw various personal crises - I've arrived at a different position which tends to be more favourable to the cyberneticians - or at least, to key early figures like Ross Ashby, Warren McCulloch, Heinz von Foerster and Stafford Beer. Two things changed for me.

Firstly, I began to explore Hume's epistemology more carefully - spurred on by Quentin Meillassoux's wonderful "After Finitude" and some careful critique in Christian Smith's "What is a Person?". Hume is very important to critical realism because the CR endeavour is founded on a critique of his epistemology. Now I think Bhaskar's critique is ill-founded, and Hume's scepticism stands up more powerfully (now that we are clearing the layers of varnish off it) than it has done for 200 years. So it is not a given that mechanisms have to be real because scientific laws found in the laboratory are seen to work in open systems (by sending a rocket to the moon, for example). It may be that Bhaskar's right - but we are perhaps safer with Bohr: "physics is the study of what we can say about nature". The assertion of the reality of causes is unsafe for reasons that Hume basically got right.

At the same time, the cybernetic ideas about modelling and isomorphism are equally vulnerable (partly for reasons which critical realists like Tony Lawson identify). Modelling causal mechanisms is a dangerous business because it inevitably creates a fictional ideal world which is a poor guide to the actual environment within which we live. There are many pathologies in the use of models and statistical analysis which has led us into repeated economic and other social crises.

However, some cyberneticians were more clear-sighted in what they were doing. Ross Ashby, in particular, saw the relationship between his theorising, model building and his practical experiments as a process of identifying constraints, not causes. If our scientific knowledge is always going to be speculative (because there can be no objectivity for the cyberneticians), the scientific process is one of generating logical possibilities from models, and then exploring which of them can not be found in nature. Knowledge emerges through the identification of constraint. On reflecting on my own practice, I think this is a better description of what I do. I also think it helps shed light on a range of phenomena in education about which we have been silent - most importantly, human relations and intersubjectivity.

This is where I now am - but it has been a struggle, and my chapter is only slowly emerging. Hume remains a key figure. He was right that scientific knowledge lay in discourse. He was wrong in thinking that it was 'causes' which were agreed (he can be forgiven because it was the scholastic idea of causation which he was attacking). To say that it is constraints which are agreed opens many new doors - particularly onto the enormously complex, multi-variate (although variables have causal connotations!) and obscure phenomena we see in education.

Learning, Design and Pedagogy: Slickness and Constraints

After years of talking about ‘Learning Design’ (both the ill-fated IMS variety and the ‘softer’ version typified by Laurillard’s equally ill-starred Pedagogic Planner), it’s curious to think how we thought learning and pedagogy were the important and difficult topics, and not for one moment did we think that ‘design’ was more problematic a concept than either of them. The dream of ‘design formulae’ as a kind of recipe for effective learning online hasn’t gone away – “might analytics help?” we now ask… (probably not)

There are perhaps some obvious points to make about the ‘slickness’ of interfaces. The experience of tapping and swiping through apps on an iPad creates the kind of ‘designed experience’ that was drilled into Apple developers by Steve Jobs. This is one kind of ‘design’ – a variety which produced interfaces which present few barriers, which appear ‘natural’ or ‘intuitive’. This natural intuitiveness has become the goal of (some) instructional designers who believe that “the slicker the better” – wheeling out various cognitive speculations to support their claim. The interesting question concerns what can be defensibly said about slickness.

If an artefact is produced to a kind of design which intends that “user” behaviour falls within particular parameters to which the design relates, and that without such a design, user behaviour would be more chaotic, then it is reasonable to say that such a design has had a constraining influence on behaviour. The Highway code is a designed set of constraints to encourage (if not force) drivers into strict parameters of behaviour. When graphic designers talk of ‘eye-catching’ designs, what they mean is that the otherwise wandering gaze is attracted and held by a particular image: that its options for looking at other things are attenuated. Indeed, the different uses of the word “design” – from the design of Apple Macs, to the UML diagrams of an IT system refer to different kinds of constraint. The IT system design attenuates through enforcing rules of behaviour on systems programmers, and eventually, users. The beautifully designed product attenuates not by enforcing rules, but often through paradoxical displays making the seemingly impossible possible. But one way or another, design is the manipulation of constraint.

Learning design then presents a curious problem because learning is not like crossing bridge in the same way as everyone else. With a range of resources at their disposal, learners meander, take unusual turns and explore. As they do so, they encounter constraints in the world – not just the constraints pertaining to a particular domain of study, or the constraints of a particular learning resource, but (most importantly) constraints of other people. Having said this, learners left to wander freely will often become lost or overwhelmed within a world of constraint which they cannot adapt to. Institutions apply constraints in terms of curricula, assessments and deadlines. Teachers apply constraints when they are necessary: “do it like this”, “concentrate of this subject” and so on; they will also sometimes find it necessary to remove constraints from the learner (“do whatever you think is interesting”). Teachers also have to operate within the constraints of the institution. As they apply other constraints in their teaching, they reveal the particular constraints operating on themselves. The constrained learner sees the constrained teacher negotiating barriers some of which they recognise, and perhaps others which they can’t yet fathom. Learners do not learn content: they learn the constraints which operate on the minds of those who present content – even when those minds are hidden behind the mask of a textbook. Good teachers will make themselves objects for inspection and do what they can to reveal their own constraints. If there is a ‘Zone of proximal development’ it lies in the relationship between the constraints of the learner and those of the teacher.

The teacher designs constraints through pedagogic instruction as a way of opening themselves up to learners. This kind of design is not simply a manifestation in the external world – although it may look like that (say a textbook or a video). It is a strategy for laying a path for an imagined learner to come to understand the inner world of the teacher who carries the knowledge they might wish to gain. Such strategies are experiments: attempts to try things out, see what happens, adjust one’s approach.  In a face-to-face setting, constraints are more available for inspection: the sheer number of different signals and senses flowing through time provides a powerful way in which constraints can be discovered. If the learner is one of a crowd in a lecture room, then the learner’s constraints are less visible to the teacher. Even fewer constraints are available for inspection if the interactions are online, although in this case there is a common constraint is technology.

In considering these intersubjective situations of coordinating constraints, designing is not just done by a teacher: learners design too. Their designs take the form of actions in attempting to articulate their understanding as a way of testing the response of teachers. Although this sounds like Pask’s conversation model, it is different in one important way: the gradual acquisition of concepts and the coordination of utterances is not the root of understanding. That lies through the shared articulation of concepts which are the constraints which provide insight into the constraints of a particular teacher or learner.

Teachers, technologists and learners all engage in design. We all formulate half-baked plans, and set out to explore what happens when we enact them. We learn about constraints as we do this. What is typically called ‘design’ in learning design circles is a kind of constraint bearing upon the design processes of the learner. Success is measured in terms of the predictable behaviour of learners with designed resources where the more effective the constraint applied in learning, the more “successful” is deemed the design. There are few ways (particularly with technology) for learners to meander: they cannot incorporate new kinds of tool and are forced to participate by posting text in forums. If they all post text, then the course is deemed successful. Shouldn’t this kind of attenuation be seen as a failure, not a success?