In a recently published text titled “The future of Activity Theory” Engeström (2009) talks about (what in the US is apparently called) the “gold standard” of educational research:
The “gold standard” emphasizes the use of randomized controlled trials, the selection of valid control groups, and “scalability” implying large statistical samples and multiple research sites. The “gold standard” correctly sees educational research as interventionist research. The randomized control trials are meant to assess the effectiveness of educational interventions. The model of intervent’ion research is taken from fields such as medicine and agriculture. As one observer put it:
For instance, if I want to test the effectiveness of weed control measures, I randomly assign different plots of crops to the experimental or control conditions. Then, they all get treated the same otherwise as far as weather, fertilizer, hours of day light and other pests. The crops are monitored and observations are made throughout the growing season and a person might be able to see the result visually if the results are remarkable enough. But the telling evidence is in the yield, when the crops are harvested. If there is a significant difference in yield in all the experimental plots as opposed to the control plots, then we might attribute it towards the independent variable, which in this case is weed control. (http://specialed.wordpress.com/2006/02/10/educational-researchthegold-standard!)
The “gold standard” thinking in educational research starts from the
assumption that researchers know what they want to implement, how they
want to change the educational practice. In other words, the intervention and its desired outcomes are well defined in advance. Tne task of research is to check whether or not the desired outcomes are actually achieved. This predetermined and linear view of interventions is actually shared by much of the literature on design experiments
I find this a noteworthy aspect of Engeström’s critique. In fact, he emphasises the similarity of assumptions that seem to drive design experiments (and design-based research) and the middle-of-the-road “gold standard” approach to educational research. In Engeström’s (2009) words again:
The main difference between “gold standard” interventions and design
experiments seems to be that the former expect the design of the intervention to be complete at the outset while the latter, recognizing the complexity of educational settings, expect the design to proceed through multiple iterations of “refinement.” But even design experiments aim at closure and control:
Design experiments were developed as a way to carry out formative
research to test and refine educational designs based on theoretical
principles derived from prior research. This approach of progressive
refinement in design involves putting a first version of a design into the world to see how it works. Then, the design is constantly revised basedon experience, until all the bugs are worked out. (Collins et aI., 2004,p. 18; emphasis added)
Collins et al. (2004, pp. 18-19) compare educational design research
to the design of cars and other consumer products, using Consumer
Reports as their explicit model for evaluation. They don’t seem to notice any significant difference between finished mass products and such open ended, continuously co-configured products as educational innovations (for co-configuration, see Engestrom, 2008b; Victor & Boynton, 1998). A strange obsession with “completeness” runs like a red thread through their argument: Thus, in the jigsaw, all pieces of the puzzle come together to form a complete understanding. (Collins et al. 2004, p. 23; emphasis added) What this overlooks is that “one can never get it right, and that innovation may best be seen as a continuous process, with particular product embodiments simply being arbitrary points along the way” (von Hippel & Tyre,1995, p. 12)…
Later in the same text Engeström (2009) makes an important point that I also tried to get across in our recent Cascading Change Symposium ( CaCha09 ) at ASCILITE09:
… resistance and subversion are not accidental disturbances that need to be eliminated. They are essential core ingredients of interventions, and they need to have a prominent place in a viable intervention methodology.
This is a view I share completely. In fact, any intervention that does not produce a level of resistance among key actors probably doesn’t even attempt to go beyond the existing mode of operation… and thus remains within the horizon of first-order change at its best.
Reference: Engeström, Y. (2009). The future of activity theory: A rough draft. In A. Sannino, H. Daniels & K. D. Gutierrez (Eds.), Learning and expanding with activity theory (pp. 303-328). Cambridge: Cambridge University Press.
On Monday, December 7. we carried out our Cascading Change symposium at the 26th ASCILITE conference in Auckland, New Zealand. For various reasons the original group of nine contributors was cut down to four who actually made it to Auckland. While I certainly would have liked to meet all the folks who had committed to the proposal, I think we would have been (even more) in trouble with the time-slot of 60 min that we had been allocated. Even if you get nine contributors to limit their air-time to 5 min statements (which already is a hard job) you end up covering 45 min. An open, conversational format is simply not possible in such a time frame.
So, even with the four remaining contributors we covered about 30 to 35 min with a few words of general welcome and introduction and a series of short impulse presentations of about 5 min each, before we could even open up to the larger plenum. We had agreed upon this format in a lively, preparatory conversation on Sunday afternoon before the actual conference kicked off. In fact, I think we should have brought that conversation on stage… but more to that later.
Altogether, our slightly eclectic individual statements/presentations apparently worked as a conversation opener. There was clearly interest in the over-arching theme and present ASCILITErs were eager to chime in an voice their opinions. However, when things just started to get somewhat interesting we already had to wrap up the session and disperse the convention. I found this extremely unfortunate. So, in retrospect I should have never accepted the reduction of the original 90 min time-slot by the planners of the overall conference programme. On the contrary, I think I should have demanded two hours as a minimum to tackle a demanding topic in a conversational format with a (potentially) large group of people.
For me this is not a mere matter of delivering a good performance for an audience. I actually want to hear other voices and opinions on a particular theme and not only broadcast what I have already thought through and then finish that off with a little harmless question and answer ping-pong. The latter seems to be considered the height of audience participation in academic conferences these days.
This brings me to the physical space. The theme of ASCILITE09 apparently was “Same places, different spaces”. Unfortunately, our symposium was placed in an enormous, theatre-style lecture hall that can certainly be qualified as yet another example of the “same spaces” (as usual) that one generally encounters in educational conferences. No matter what you do in such a space… it does not create an egalitarian, conversational flow. There are some actors on stage… and there are spectators. Our attempt to compensate a little by dragging in some chairs from the coffee break area didn’t show much effect, I suppose. It only ensured that the contributors moved at least away from the central podium. An appropriate “space” is just another aspect that I simply should not (and hopefully won’t) compromise about. A symposium simply requires “different spaces” than a good old lecture hall… no matter how fancy and well equipped it is.
I had a few conversations with George Siemens and Rob Fitzgerald during the remaining conference days on the symposium, the presentation formats encountered, and the general failure to create real, genuine dialogue within the actual conference programmes… and not only during the breaks and social get-togethers. In the case of a symposium I am willing to do away with any kind of impulse presentations. I can easily imagine to simply start with a conversation among a group of informed peers on stage… that gradually draws in more and more participants. It would provide a hyperlink-cloud around the individual contributors to get an idea of where they are coming from, and possible end with recommendations on further readings… plus some form of mediated conversation and exchange beyond the event. No presentations, no lecture halls, no 60 min time-slots. Stay tuned… I will try this somewhere sometimes in 2010.
- May 2013
- March 2013
- December 2012
- November 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- June 2011
- May 2011
- April 2011
- March 2011
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009