I’ve been reading a lot of Nassim Nicholas Taleb of late, first Fooled by Randomness and now The Black Swan. The gist of both books is the same: we don’t know a lot more than we do know, yet we tend to think the opposite is true and behave accordingly. The logic of quantitative analysis exacerbates this problem. Statistical models (and Taleb is especially harsh on economists), are premised on estimates of error. A relationship is found to exist with a p value of .05, meaning that there is a 5% chance the relationship is attributable to chance or randomness. Even if that is true, what happens within that 5 percent can be vastly more important than the cumulative impact of everything that happens within the more predictably charted 95 percent. These low-probability, high-impact events are what Taleb calls Black Swans.
What does any of this have to do with education? The answer, in part, lies in what he calls the Ludic Fallacy. He introduces it in a critique of probability theory, but its application is broader. In essence, the Ludic Fallacy is the tendency to confuse games (simulations or models in which all rules and parameters are known) with reality (whose complexity is frequently beyond our comprehension). The greater the gap between games and the reality they’re supposed to represent, the more dangerous the conclusions we arrive at based on those games.
In education, the gap is large. We have become obsessed with studies that tell us what contributes most to student learning, as if we had a good way of thinking about such things. As Taleb says: “In real life you do not know the odds; you need to discover them, and the sources of uncertainty are not defined.” Yet we continue to study and design school reforms as if we knew the sources of uncertainty, or we understood the odds. As a result, we end up convincing ourselves that two hours a day of practice testing will somehow produce educated citizens.