A Unique Approach To Pre And Post-Assessments
by Grant Wiggins, Ph.D, Authentic Education
One of my favorite stories concerns the legendary basketball coach John Wooden. He always gave himself a research project in the off season. As recounted in the insightful and practical book You Haven’t Taught If They Haven’t Learned, one year Wooden’s UCLA Bruins had done a poor job at shooting fouls. What did Wooden do? Call up the coaches of the best foul-shooting teams as well as the best-shooting players to find out what they did in practice. He learned a vital lesson: too often, foul-shooting was not done under game conditions in every day practice.
So, he changed the routine: players would scrimmage, some would be subbed out. The ones removed would then run sprints, after which they had a set and brief amount of time to shoot free throws on a side basket while the scrimmage went on. Players gasping for breath, but only seconds to shoot – just like in a real game. Needless to say, the following year, his team led the league in free throws.
What’s your summer research project? What deficits do you need to ponder and research before school starts in the fall? Let me propose two projects and a general method for doing action research next school year.
Student misconceptions. We now know that students fail to understand essential (though often counter-intuitive) concepts. There is now a 30-year research history of such hard to eradicate errors in the sciences, for example: see here and here. What stubborn misconceptions did your students have difficulty escaping this past year?
Student self-assessment. We know from the research that students’ ability to predict their grades accurately, metacognitive ability, and self-assessment are highly correlated with great gains in achievement. How might you make student self-assessment more central to your work (and your measure of progress) next year?
Go For The Gain: Pre And Post-Assessment
The general method for doing useful personal research (in these two areas or others) is to construct a pre- and post- assessment system so that you can formally track how much progress you make next year. Indeed, the science misconception literature is typically based on a pre- and post-assessment using a test of misconception, such as the longstanding Force Concept Inventory in Physics. (Here are some follow-up interviews on the FCI). You can find other science misconception tests here. Here and here are some resources on math misconceptions. In other subjects, what might be a good pre- and post test using the same questions? A simple way to get started is to think of using a key Essential Question as the pre and post assessment question.
A more formal way of developing a baseline, ongoing, and final assessment of performance/understanding is to track the effect size of your teaching. Mathophobes, don’t freak: it is very easy to calculate. E = post ave – pre ave. /Stdv ave. i.e. Effect size = the class average in the post-test minus the class average in the pre-test, divided by the average of the standard deviations on both tests.
If you own Visible Learning for teachers by John Hattie, Appendix E offers a brief, easy to understand, and practical discussion of effect size and how to calculate it. He even walks you through the design of an Excel spreadsheet design to make it automatically calculated if you just input student grades/scores/times. There are other resources here and here.
A virtue of using such effect size calculations is that you can not only compare different tests composed of different amounts of questions, but you can compare very different kinds of tests and measuring systems. For example, history teachers and track coaches can have a common metric that permits progress to be compared across those two different measuring systems (decrease in running times vs. increase in grades).
Another virtue of using effect size calculations is that you can then compare your overall results to all the effect sizes in Hattie’s book and against the key effect size of .40. Why is .40 a key effect size? Because Hattie exhaustively studied all effect sizes and found that this is the typical gain in a typical class after a typical year of study. In other words, if you get an effect size of .6 or .7 you are achieving a far greater gain with your kids than would be expected. If you are getting gains of only .3 or .4 then your teaching is not making a significant difference in the area(s) you targeted. (You may also want to look at Hattie’s list of effect sizes of the most commonly used interventions in education to get a better feel for what effect sizes are typical and what great gains are possible. Some of the findings will surprise you and motivate you.)
The final virtue of looking at gain or effect size is that the assessment is completely fair and credible. You are looking at progress with the kids you have, on tests you choose; you are not comparing apples to oranges or holding your students to unreasonable expectations or stuck with dopey tests that strike you as irrelevant to your goals. As a result, as with computer games, swimming, and running you and your students will feel more in control of achievement and overcome the fatalism that infects so much of education.
When I taught briefly at Brown University, in my education course I used the same paper topic as the first and last assignment of the course. I also made them add an Appendix to the 2nd paper describing their reaction to looking back at the 1st version. Many of the Brown students said that the exercise was among the most enlightening and gratifying that they had experienced as students. One young man said it perfectly: I had no idea how much I had learned!
Have a thoughtful summer.
Image attribution flickr user rexpe