— S e b l o g g i n g

In a recently published text titled “The future of Activity Theory” Engeström (2009) talks about (what in the US is apparently called) the “gold standard” of educational research:

The “gold standard” emphasizes the use of randomized controlled trials, the selection of valid control groups, and “scalability” implying large statistical samples and multiple research sites. The “gold standard” correctly sees educational research as interventionist research. The randomized control trials are meant to assess the effectiveness of educational interventions. The model of intervent’ion research is taken from fields such as medicine and agriculture. As one observer put it:

For instance, if I want to test the effectiveness of weed control measures, I randomly assign different plots of crops to the experimental or control conditions. Then, they all get treated the same otherwise as far as weather, fertilizer, hours of day light and other pests. The crops are monitored and observations are made throughout the growing season and a person might be able to see the result visually if the results are remarkable enough. But the telling evidence is in the yield, when the crops are harvested. If there is a significant difference in yield in all the experimental plots as opposed to the control plots, then we might attribute it towards the independent variable, which in this case is weed control. (http://specialed.wordpress.com/2006/02/10/educational-researchthegold-standard!)

The “gold standard” thinking in educational research starts from the
assumption that researchers know what they want to implement, how they
want to change the educational practice. In other words, the intervention and its desired outcomes are well defined in advance. Tne task of research is to check whether or not the desired outcomes are actually achieved. This predetermined and linear view of interventions is actually shared by much of the literature on design experiments

I find this a noteworthy aspect of Engeström’s critique. In fact, he emphasises the similarity of assumptions that seem to drive design experiments (and design-based research) and the middle-of-the-road “gold standard” approach to educational research. In Engeström’s (2009) words again:

The main difference between “gold standard” interventions and design
experiments seems to be that the former expect the design of the intervention to be complete at the outset while the latter, recognizing the complexity of educational settings, expect the design to proceed through multiple iterations of “refinement.” But even design experiments aim at closure and control:

Design experiments were developed as a way to carry out formative
research to test and refine educational designs based on theoretical
principles derived from prior research. This approach of progressive
refinement in design involves putting a first version of a design into the world to see how it works. Then, the design is constantly revised basedon experience, until all the bugs are worked out. (Collins et aI., 2004,p. 18; emphasis added)

Collins et al. (2004, pp. 18-19) compare educational design research
to the design of cars and other consumer products, using Consumer
Reports as their explicit model for evaluation. They don’t seem to notice any significant difference between finished mass products and such open ended, continuously co-configured products as educational innovations (for co-configuration, see Engestrom, 2008b; Victor & Boynton, 1998). A strange obsession with “completeness” runs like a red thread through their argument: Thus, in the jigsaw, all pieces of the puzzle come together to form a complete understanding. (Collins et al. 2004, p. 23; emphasis added) What this overlooks is that “one can never get it right, and that innovation may best be seen as a continuous process, with particular product embodiments simply being arbitrary points along the way” (von Hippel & Tyre,1995, p. 12)…

Later in the same text Engeström (2009) makes an important point that I also tried to get across in our recent Cascading Change Symposium ( CaCha09 ) at ASCILITE09:

… resistance and subversion are not accidental disturbances that need to be eliminated. They are essential core ingredients of interventions, and they need to have a prominent place in a viable intervention methodology.

This is a view I share completely. In fact, any intervention that does not produce a level of resistance among key actors probably doesn’t even attempt to go beyond the existing mode of operation… and thus remains within the horizon of first-order change at its best.

[Sebastian Fiedler]

Reference: Engeström, Y. (2009). The future of activity theory: A rough draft. In A. Sannino, H. Daniels & K. D. Gutierrez (Eds.), Learning and expanding with activity theory (pp. 303-328). Cambridge: Cambridge University Press.