Tuesday 29 October 2013

Socio-technical Evaluation in the ITEC project

The ITEC project has one year to run. In the last 3 years, it has done many good things in stimulating technical engagement in schools across Europe. There have been some very heartening and enthusiastic engagements with the project coming from places ranging from Turkey and Portugal to Norway and Finland with talented teachers finding a voice and a platform for experimenting with technology and doing 'cool things with the kids' - even if it's simply using video (the most powerful and easy-to-use technology) to boost confidence and encourage reflection...


ITEC has a technical vision whereby it aims to bring technologies closer to-hand for teachers across Europe. As part of this vision, it has created a 'Widget store' where teachers can upload and organise small tools (which can be easily found on the web) and bring them into their teaching platforms. As with all technical visions, however, this is more easily said than done!

The Widget Store exists as an entity (see it at http://itec-moodle.eun.org where you can log-in with Facebook or Google) although usage is more sporadic amongst teachers than those who envisaged and designed the technology might have wished. Does this mean that it's not very good?

This is a question I've been asking myself quite a lot (partly because the Widget Store is something my own University has played a big role in). The problem is that when we design technologies in all hopefulness and expectation of how people will react to them, we don't really have "real" people in mind: we have 'abstract' people who tend to be little versions of ourselves: our "abstract people" get excited about our technology because we are excited about our technology. There is a kind of delusion that sets in with any technical design. In a project, this is a collective delusion!

What's interesting is that sometimes, despite delusions, things really do catch on. Whilst the delusion or abstract users will rarely be correct, something does set the public imagination alight. Nobody who originally designed Twitter, or many other  social software services would have imagined what they might become. Even YouTube's early ambition would not have dared to reach the extent of its business model now. But without the initial delusion, we'd get nowhere. Delusions are important.

But perhaps we shouldn't feel so bad if our ambitions are thwarted by 'real people' - or at least, we shouldn't feel so bad if that happens and we LEARN FROM IT. That is the real purpose of evaluation. (It would of course be dumb to not only be delusional in our design and idealism but to not learn from what happens!) Many brave technical experiments have gone this way: Apple have made many technical interventions far more expensive than ITEC's Widget Store before it caught onto something which grabbed the imagination of everyone: the Newton was ahead of its time; OpenDoc was a remarkable technical development. Google too are continually creating and discarding developments. We ought to be doing this in education, and it's work that ought to be funded properly. (Or we could let Google and Apple do it for us - but that takes us in dangerous water, I think)

The value of a technical development is complex. ITEC is a large project with many strands of intervention, from technical to pedagogical. Telling a simple story about the project has been much facilitated by the tools it has produced. The simple fact is that when explaining the project to a group of teachers, rather than ramble on about concepts of pan-european integrated education, ITEC could talk about its tools, and it can demonstrate them. Teachers can use those tools, and through using them, understand the deeper aims of the project. Tools like the Widget Store are particularly valuable in this regard because they are heterogeneous in their embrace of technology. That means that it is not just one technology that might be introduced to teachers (say, making a video), but also Etherpad, or a Shared drawing tool, or Prezi, or Poplet. Any one of these could be introduced individually, but that would concentrate the discussion on one tool. The Widget Store maintains an overview of possible technologies, whilst situating all of those technologies within the broader narrative and vision of the project. Even if people don't go on to use it in their daily practice, its discursive intervention can be significant in inspiring and changing practice in line with the aims of the project.

The way I'm beginning to think about this is through understanding the different discourses that interact when doing e-learning projects. There is always a group of technologists talking about technology. There is always a group of teachers who the technologists want to get to use their technology. There is always some kind of project management. These discourses are quite separate from one another, and the big challenge in ITEC has been to connect them. Sometimes it has been possible: but only under one particular set of circumstances: Where there is coherence in an identified 'need' or absence within one community with an identified absences in the other two communities. So when a pedagogic need coheres with a technical requirement which coheres with a project objective, then things work. If this isn't the case, they don't.

What I have learnt in the technical evaluation of ITEC is that absences really matter. Our challenge in evaluation (and project management) is to find better ways of identifying them. The delusions of software developers are, in a deep way, driven by absences. The better we get at orienting those technical absences to the real needs (also absences) of real people, the more virtuous the circle between design, implementation and management will become.

No comments: