Socio-technical Evaluation in the ITEC project

The ITEC project has one year to run. In the last 3 years, it has done many good things in stimulating technical engagement in schools across Europe. There have been some very heartening and enthusiastic engagements with the project coming from places ranging from Turkey and Portugal to Norway and Finland with talented teachers finding a voice and a platform for experimenting with technology and doing 'cool things with the kids' - even if it's simply using video (the most powerful and easy-to-use technology) to boost confidence and encourage reflection...

ITEC has a technical vision whereby it aims to bring technologies closer to-hand for teachers across Europe. As part of this vision, it has created a 'Widget store' where teachers can upload and organise small tools (which can be easily found on the web) and bring them into their teaching platforms. As with all technical visions, however, this is more easily said than done!

The Widget Store exists as an entity (see it at http://itec-moodle.eun.org where you can log-in with Facebook or Google) although usage is more sporadic amongst teachers than those who envisaged and designed the technology might have wished. Does this mean that it's not very good?

This is a question I've been asking myself quite a lot (partly because the Widget Store is something my own University has played a big role in). The problem is that when we design technologies in all hopefulness and expectation of how people will react to them, we don't really have "real" people in mind: we have 'abstract' people who tend to be little versions of ourselves: our "abstract people" get excited about our technology because we are excited about our technology. There is a kind of delusion that sets in with any technical design. In a project, this is a collective delusion!

What's interesting is that sometimes, despite delusions, things really do catch on. Whilst the delusion or abstract users will rarely be correct, something does set the public imagination alight. Nobody who originally designed Twitter, or many other  social software services would have imagined what they might become. Even YouTube's early ambition would not have dared to reach the extent of its business model now. But without the initial delusion, we'd get nowhere. Delusions are important.

But perhaps we shouldn't feel so bad if our ambitions are thwarted by 'real people' - or at least, we shouldn't feel so bad if that happens and we LEARN FROM IT. That is the real purpose of evaluation. (It would of course be dumb to not only be delusional in our design and idealism but to not learn from what happens!) Many brave technical experiments have gone this way: Apple have made many technical interventions far more expensive than ITEC's Widget Store before it caught onto something which grabbed the imagination of everyone: the Newton was ahead of its time; OpenDoc was a remarkable technical development. Google too are continually creating and discarding developments. We ought to be doing this in education, and it's work that ought to be funded properly. (Or we could let Google and Apple do it for us - but that takes us in dangerous water, I think)

The value of a technical development is complex. ITEC is a large project with many strands of intervention, from technical to pedagogical. Telling a simple story about the project has been much facilitated by the tools it has produced. The simple fact is that when explaining the project to a group of teachers, rather than ramble on about concepts of pan-european integrated education, ITEC could talk about its tools, and it can demonstrate them. Teachers can use those tools, and through using them, understand the deeper aims of the project. Tools like the Widget Store are particularly valuable in this regard because they are heterogeneous in their embrace of technology. That means that it is not just one technology that might be introduced to teachers (say, making a video), but also Etherpad, or a Shared drawing tool, or Prezi, or Poplet. Any one of these could be introduced individually, but that would concentrate the discussion on one tool. The Widget Store maintains an overview of possible technologies, whilst situating all of those technologies within the broader narrative and vision of the project. Even if people don't go on to use it in their daily practice, its discursive intervention can be significant in inspiring and changing practice in line with the aims of the project.

The way I'm beginning to think about this is through understanding the different discourses that interact when doing e-learning projects. There is always a group of technologists talking about technology. There is always a group of teachers who the technologists want to get to use their technology. There is always some kind of project management. These discourses are quite separate from one another, and the big challenge in ITEC has been to connect them. Sometimes it has been possible: but only under one particular set of circumstances: Where there is coherence in an identified 'need' or absence within one community with an identified absences in the other two communities. So when a pedagogic need coheres with a technical requirement which coheres with a project objective, then things work. If this isn't the case, they don't.

What I have learnt in the technical evaluation of ITEC is that absences really matter. Our challenge in evaluation (and project management) is to find better ways of identifying them. The delusions of software developers are, in a deep way, driven by absences. The better we get at orienting those technical absences to the real needs (also absences) of real people, the more virtuous the circle between design, implementation and management will become.

Excuses, Explanations and Social Ontology in E-learning and Economics

The Cambridge Realist Workshop (see http://www.econ.cam.ac.uk/events/seminars/realist/workshop_programme.htm) has been very important to me over the years. When I realised as a fledgling educational technologist about 10 years ago, that most attempts to evaluate educational technology were ridiculous, and that 'ontology' was the thing that mattered, I signed up to the group - simply because they were the only people who appeared to be thinking similar thoughts (they had a conference on 'technology and ontology' that year - although I didn't go). Most of what they talked about in the workshop was economics and, undaunted by the  fact that I knew nothing of economics, I started reading the papers that (then) were sent to me at Bolton by post (the first one was by Amartya Sen - which was a good start!). Only within the last few years have I mustered the energy to do the 3 hour drive to Cambridge to attend - but I learnt an enormous amount anyway. It was a kind of MOOC I guess - but this one really worked because it was driven by passionate commitment on the part of all who engaged with it to get to grips with fundamentally difficult problems.

I was there on Monday for Tony Lawson's talk which was a brilliant analysis of Thorstein Veblen's coinage of the term 'neo-classical economics' - a term generally used abusively but without definition by economists, but which fundamentally according to Veblen (and to Lawson) referred to that group of economists who were progressive in their thinking, but whose methods contradicted their ontology. But after the talk, the topic of 'explanation' arose. "We all want to explain away the things we don't like," I argued. It's not the what an explanation that is interesting, but how it does its explaining.

Thinking about it, and following another discussion that occurred among friends a bit earlier in the evening, I have been thinking about the relationship between an explanation and an excuse. Much of Lawson's argument against formalism in modern economics is that it brings inappropriate tools (mathematical models) to a reality which it doesn't understand, has no ambition to critique, and yet seeks to explain it through the use of these tools. Its explanatory power is dreadful and we all live with the consequences. There is basically an intellectual laziness to engage with reality born out of reliance on tools and maintaining the professional identity of individual economists. To what extent is this laziness the result of using tools to make excuses for the status quo? To what extent is the excuse-making habitual?

An excuse is a kind of explanation: religion is an excuse for war; austerity is an excuse for maintaining inequality; national security is an excuse for surveillance. The distinction between an excuse and an explanation can be difficult to pinpoint in the statement itself: the distinction lies in the orientation of the observer. I asked my 13 year-old daughter what fear is the other week: "it's an excuse!" she said (she kicked off my thinking on this whole topic!) To call something an excuse is be oriented to do something about it: to reject the explanation and move out of the cycle where the explanation (or excuse) does its explaining. To call something an excuse summons up passion, anger and a determination to challenge: it is to wake up.

When might an explanation not be an excuse? This is difficult because it depends on one's orientation: one person's explanation is another person's excuse. What matters is the underpinning value system. Where does that come from? Most likely from life experiences - childhood attachments, etc. That's where the how of an explanation really matters more than the what. 

This is where I think we have to be careful in recognising the nature of the challenges that identifying an excuse can present. It's not enough to say "you're just making excuses!" - because we will have our own excuses for making the challenge. But the question "what are you wanting to excuse?" is more powerful. Any excuse carries within it imminent traces of the mind that wishes to hold on to it. "Why are you wanting to excuse the status quo? What is in you that wishes to believe this explanation? What is in me that wishes to challenge you?" - this is a more constructive way of moving forwards. It is where we can see the layers of excuse-making in each other. We might hope that by exposing this, there might be a deeper meeting point that is therapeutic rather than confrontational.

No Excuses: Double Binds and Academic Pay

My post about student employability gave me some worries after I'd published it. Maybe, I thought, to suggest that people are 'addicted' to excuses is too strong: after all, we all need excuses as ways of explaining the things that we don't do: I feel a need to excuse my addiction to excuses! There are many things I should have done two or three weeks ago that I haven't done that I need to make excuses for (and why am I blogging rather than getting them done??). Having said this, I recognise the pattern in myself: the excuse deals with the pain of guilt (guilt is not only a "Catholic" thing!)

There is something static and almost grounded about this cycle of feeling inadequacy and making excuses for it. Is there a comforting feel to going round the cycle. Which makes me wonder what happens if I genuinely break out of it.

Of course, what I have to do to break out of the cycle is to actually do the thing that I was making excuses for not doing. How do I put myself on that trajectory? Perhaps one of the main things that happens is that I see a way out of the cycle; I start to feel a sense of recognition about the cycle I am in, and at the same time know what to do about it (is the feeling of recognition also the feeling of knowing what to do?)

Deciding to enter an educational institution with some purpose of achievement is the kind of act that is designed to move people out of the cycles they are stuck in. Education promises (but doesn't always deliver) the required impetus to 'move on'. But if education reinforces feelings of inadequacy in students, it can make things worse. If my intellectual commitment and determination lies principally in education, it is because I believe this ought not to happen. Indeed, I think its dubious morality implicates those involved in a social ill for which nobody taking money from students (which those students will be repaying for the next 20 years) ought to be able to shy away from. My (overlarge) salary is part paid for by the European tax-payer and part by students (approximately 3 students yearly fees are needed to pay the remainder of my annual salary, which those students will be repaying for up to the next 20 years). Vice Chancellors typically require the "souls" of two students per month to pay their salaries. This makes me feel sick.

The rule is 'no excuses'. Vice Chancellors excuse their pay in the context of the 'global market' of higher education: bollocks, isn't it?? Most lecturers will defend theirs in terms of the cost of living. But our intellectual job, the purpose of our lives in education, is to think hard and well, not to become wealthy. Thinking carries its own rewards which do not require the subjugation of the future generation. Great thinking more often than not arises from positions of material hardship.

The problem is that we will continue to make excuses. We will even make excuses for not fighting the gross injustices and imbalances of pay within our own institutions. We will make excuses for not challenging people who are 'in charge' but clearly incompetent. But worst of all, we will make excuses for the status quo which increasingly will enslave students to alienating experiences for which many of them will still be paying for after we are dead.

Music. Information. Learning.

I think that Information is the most important phenomenon that urgently requires a coherent scientific understanding. Currently we only have fragments of 'information theory' and the fragments don't fit together: information as communication between machines (Shannon's theory) which has been so important in technology; information in meaningful human communication processes (i.e. not machines); information in biology - particularly information in DNA (but enzymes? or proteins?); information as experience (the experience of becoming informed); information in learning; information as qualia; information 'flows'; information overload... and so on.

Information is, before we make any kind of distinction about it, fundamentally an aspect of experience. My eyes pass over the spreadsheet in the way that they might pass over a painting or my ear might latch onto a piece of music. At some point, things draw my attention. But at what point? What has happened for one particular number (a patch of black on the white screen) to draw my attention?

This is why I think, as an experience, information is 'musical'. It is hard to study the movements of my eye over a painting, but the sensation of my ear during music can be inspected at some depth. There are moments which 'strike' me - moments at which I have a sensation of revelation or discovery. There are also moments when the tedium of it all causes it to wash over my attention without me taking notice. I think if we can understand this in musical experience, our understanding of information as experience will be increased.

What I am interested in, which underpins information as experience, are human expectations. These shift when we witness an 'information flow', and they change when we listen to music. They also change during learning processes: indeed, our expectations are always shifting. I think that our emotional responses are closely aligned to shifts in expectation. We anticipate with eagerness when we are certain about something that is about to happen: the result of a calculation, or the resolution of a cadence. We feel satisfaction when the resolution occurs or the equation is solved. We become anxious and disorientated when we don't know what to expect.

I think expectations are formed within constraints. An expectation is the 'thinkable' part within a context which is largely unthinkable. What we conventionally think of as 'information' is bounded by what we cannot think. There is much more to any 'information' but it is not thinkable. This is certainly the case with music: the absences are bigger than the presences.

What is 'unthinkable' is effectively redundant. Yet whilst it is redundant, it is also causal as a constraint on what we can think. Redundancies may be analysable. Music is full of redundancies: repetitions of motifs, accompanimental patterns, etc. With information (say on the internet) there is also huge amounts of redundancy. There is a job  to do in analysing this redundancy and understanding how it is causal as a constraint on what is thinkable.

The 'topic' of any piece of information is the thinkable part that is sustained in a sea of redundancy that maintains it. The topic of any piece of information is a kind of expectation. The topic of a piece of music is a deep level structure within which successive events can be situated (like, for example, a deep Schenkerian structure).

In learning, expectations of both teachers and learners change. In their conversations there are huge amounts of redundancy (this is why Pask's conversational model is mistaken, because it only looks at the thinking part of utterances). The redundancies constrain the expectations of each. By having knowledge, the teacher is able to generate more redundancies which can further constrain the learner's expectations. With this constraint, the learner can be led to feel safe in first making utterances that they feel secure in anticipating the outcome of. This is basically 'play'.

Which then raises the question about the role of redundancy in play. Most games operate with simple rules which are applicable in a variety of ways. Is this also redundancy? Cyberneticians talk of 'reducing variety' with things like rules. But I'm beginning to think that this is better understood as 'increasing redundancy' - effectively it amounts to the same thing: variety is reduced if the constraints are amplified. Then the difference between play fighting and real fighting becomes one where in play fighting the redundancies are always maximised, whereas in actual fighting there is absolutely no redundancy in a real punch!!

Employability, Student Confidence and the Double Bind

I was in a meeting about student employability today. My institution is one that gives opportunities to students who would not otherwise have the opportunity of getting a degree, and who have often had difficult experiences with education in the past. I believe this is the most important group of students in education - it's where we can really make a difference. But it's very difficult.

The moment of transition when the course stops, the students gain their qualification and they have to turn and face the 'world outside' to get a job can present particular challenges. Student confidence, the ability to be flexible about expectations of employment, the resilience to 'keep going' if initial applications are unsuccessful, the restlessness that is necessary to keep students anxious to carry on learning new skills, the continual monitoring of what's available, the sense of how to position oneself... these are all things which tend not to get taught on degree programmes which, naturally enough, tend to focus on "how to get a degree". Sometimes, student confidence can be hit if their experiences of trying to get a degree have reinforced expectations of failure repeated from previous experiences of education: the cycle of 'referrals', 'repeats', 'change of module pathway', etc can be demoralising. From that position it is very hard to turn to face the world with the necessary vigour which will fight the competition.

All this has led me to think about 'student confidence'. We had a rather authentic meeting today where both staff and students were encouraged to share their hopes and fears about their employability situation (the elephant in the room with employability in HE today is that staff often have worse prospects of employment than students!). It seemed to be a good thing for students to see the vulnerabilities in their teachers. The message is "it's not just you!". It was almost like "Employability Anonymous".

Which has got me thinking...

Gregory Bateson wrote what I think is his best paper about Alchoholics Anonymous (see "Steps to an Ecology of Mind"). What he identified was a self-defeating cycle whereby the individual was caught in a trap: feeling terrible that they were an alcholic while they were sober that they would drink to alleviate the depression. Is this feeling of helplessness so different from (or not part of) low self-esteem? Then, Bateson describes what he thinks AA does: it helps individuals to see the trap they are in. On becoming aware of the dynamic, they are able to step outside it.

The pattern when talking to troubled employment-seekers about the positive action they might take to find work tends to be "could you could do something to make your work more visible so others can see what you can do?", being met with reasons why the individual can't do that. Or the employment-seeker might grudgingly agree that there is something they can do, agree to do it, and then not do it. Is the intransigent employment-seeker like the alchoholic and the bottle? Is their modus operandi one of:

1. feeling bad about not wanting to look for work
2. addressing the badness of their feelings by reinforcing excuses for not doing anything

This would mean that the "empoyability problem" is a problem of addiction to "excuse-making" for not doing anything about employability.

If this is true, how can we treat it?

Following Bateson's example, helping people see the cycle they are trapped in is very important. The only way they can see this is by seeing it in operation in other people. The AA-style meeting is essential. 

An "excuse" is a particular kind of explanation. It addresses something that hurts by convincing the excuse-maker that the pain is a natural function and that nothing can be done about it. This is the easiest way of addressing the pain. By showing that excuse-maker that the explanation doesn't explain how they feel (and in fact reinforces it) forces the need for new explanations to be sought. This is the first step towards jumping outside the cycle. 

Is confidence where excuses are turned into opportunities? Maybe we should be treating "excuse addiction" (and not just to our students!)

Quick Oculus Rift development with Oculus Bridge

I'm very grateful to @paddytherabbit (David Sherlock) for showing me the easiest way to get started with Oculus Rift development. For David's post, see http://paddytherabbit.com/3d-worlds-web-using-oculus-rift-attempt-1/. From my perspective, this is almost as exciting as the Rift itself. It means that development can be placed in the hands of hacky programmers like me who are too lazy to use Unity (or pay for their license) or the SDK, but can get something rich and immersive up and running in 5 minutes with a bit of HTML.
David's video is here:

The Rift's main benefit is that it is a very simple bit of kit: basically it's a screen with a motion-sensing controller tied-in. All Oculus bridge has to do is to render to the screen in the 'barrel distortion' pattern that is required, and to relay signals from the USB port about the movement of the head.

But the ease with which the web can be integrated into this set-up means that developments in VR will, I think, be even quicker than I imagined. There are already a lot of games built with Unity and other platforms that are appearing. The problem with these tools is that they are geared for doing something far more complex and sophisticated than most people will want when exploring what might be possible.

With Oculus Bridge (and the associated threejs (http://threejs.org/) library which does the WebGL rendering) I can hack together all sorts of things in my virtual world and see what it's like. More to the point, I could get my daughter to have a go! Let's try something with sound in 3D sound positioning... let's try getting some unusual textures... let's try importing a 360 degree photo taken with a Ricoh Theta (see https://theta360.com/en/), or let's see about integration with Leap Motion (https://www.leapmotion.com/) or integrate graphical effects in a virtual world with the range of consumer EEG brain monitors. But it's all quick and easy.

This is also important for WebGL - that is a technology that's been in development for some time, and sometimes the rationale for it hasn't been entirely clear (particularly because the performance isn't quite as good as a native 3D rendering). But as a platform for creating quick virtual worlds, are we really going to worry about the details of rendering performance if all we want to do is to play around with things? Added to that, the performance of Web-based 3d will also only get better: the computers that matter now are our web browsers.

Someone said to me that with the Oculus Rift, it was like technology going back to CD-Rom authoring. It may be, but I'm beginning to sense that the development from CD-Rom to social tools is going to be extremely rapid. Indeed, by the time the Rift is released commercially next year, it might all be in place. And then I think things will look very different.

Dominic Cummings Thesis on Education reported @patrickwintour in The @guardian: A response

-
I have a thesis about education ministers: if we connected the personal biographies of successive Secretaries of State for Education with their political actions in office we would be able to identify strong correlations between family circumstance (household income, emotional stability or upheaval, social capital, schooling, career development) and political orientation. (Actually, the same could be said of all of us who hold an opinion about education.) In defending my thesis, I would argue that it shows the difficulty in arriving at any kind of objectivity about education. It might also highlight the patterns of early attachments which give rise to politicians who are convinced that they are privileged with not only a unique objectivity about education, but also a sense of entitlement to lead the country's teachers and students along the right path towards the golden future - whatever the consequences. Political opinion about education has an ontogeny which we can inspect in hindsight and which we ought to be aware of in our challenge to ministers in office.

Dominic Cummings is a special adviser to the Secretary of State for Education. Shortly before his departure from Mr. Gove's office, he has written "Some thoughts on Education and Political Priorities". I haven't read this report - but I will rely on the account of it written in by Patrick Wintour in The Guardian.

According to Wintour, Mr. Cummings doesn't appear to like education very much. Cummings complains that "there is widespread dishonesty about standards in English schools and low aspirations" - a critique which drives his main point that we are letting the cleverest children down. "The block to higher performance is the management and quality of teaching in the school," he argues - the solution being the reduction of government regulation, reducing the Department for Education to "the employment of accountants and inspectors". He also complains about "useless" courses in "third-rate universities".

Cummings exhibits the rationalistic ingenuity of an engineer as he talks about militaristic reforms to the civil service which would make the firing of poorly performing civil servants much easier. But his engineering ambition is most eloquently expressed as he compares the UK to Apple Corp, arguing that Britain needs to reshape its product line. Bring on the iPhones (and let's not talk too loudly about the conditions under which they are manufactured). It's the kind of language that ought to make him ineligible for running any organisation (but apparently he wants to run a "Free School" - one wonders what sense of the word 'free' he has in mind!)

People like this are a problem - not just because they are powerful and many of us find these views objectionable, but because in objecting, we ourselves are caught in the same "objectivity" trap that debates in education and social policy expose. What ought to be a critical debate is reduced to journalist brickbats. The deep problem is that education ministers of all political colour treat the "education debate" as a vehicle for their own career ambitions and not as a genuine attempt to create the conditions for the cultivation of the future (to be fair to Cummings, he explicitly identifies this tendency in others - but chooses not to see it in himself!). It may be that our political system is to blame for this - after all, educational interventions are a slow-burn: an economic policy decision may see GDP rise or fall within a parliamentary term with electoral consequences. Educational interventions (for example, Comprehensive Schools) may take 20 years, and perhaps in the context of political careers, it is inevitable that there is merely a 'mock seriousness' about policy.

But Cummings laying his cards on the table like this is an opportunity for which we should be grateful. But we have to engage in a different kind of political debate. Instead of objecting to his points, we need to ask the 'powerful questions' both of Cummings and ourselves that will expose the deep differences in how he and we think the world works. We should see Cummings as opening a debate about "social ontology". That sounds grander than it is: it simply means saying "Given that these things you say may be happening in education (e.g. poor schools, etc) what do you think the world is like?"

We all have a metaphysics sitting beneath our opinions of education. This is because the study of mind, of intellectual life, learning, teaching and daily experience (all of which belong to education) are not things which belong to the domain of naturalistic inquiry. Behind any educational opinion, one can find metaphysical ontologies ranging from religious fundamentalism, cognitivism to solipsism. The differences between these are fundamental and, one would hope, their exposure ought to encourage greater humility, practicality and humanity in our approach to the messy business of education. Cummings has a fashionable metaphysics that underpins his educational opinion: 'genetics'. Without articulating exactly how he believes specific genes exercise causal powers in the biological morphogenesis that turns cells into embryos and embryos into education ministers, Cummings says "it's all about the genes".

He should be challenged about this. He should be challenged to articulate his genetic metaphysics. He should be confronted with the logical consequences of the mechanisms he describes (if he can describe it). If he can't describe any mechanisms, he should be challenged to try. He should be challenged to explain the gaps between a logically-emerged social world from his abstract genetic principles and the real world he sees around him - even those aspects of that world which he likes. And perhaps, on seeing the explanatory deficiencies of his ontology which can be exposed for all to see, he might be given cause to think again.

Education is the domain where critical scientific inquiry, political discourse, democratic legitimation and practical operationalisation meet. We need clever people like Cummings (I'll stop short of saying we need Gove!). But we need them to think harder. But they will only do this if there is a space for them to be effectively challenged. The onus is on all of us to think better about education.