Teaching Evaluations Don't Tell Us Much about Teaching Effectiveness

University student evaluations of the "overall effectiveness" of their instructors play an important role in assessing consumer satisfaction with product that universities sell. I am skeptical about whether they measure anything else, and I have been ever since Geoff Carliner, Tom Romer, and I published an empirical piece many years ago showing that evaluations were not related either to the amount a particular instructor taught the students or to how lenient the instructor was [I don't have an on-line link, but see "Leniency, Learning, and Evaluations" (with G. Carliner and T. Romer), Journal of Educational Psychology 70 (Fall 1978): 855-863].

Now there is further evidence:

Students asked to watch five seconds of soundless videotape of a teacher in the classroom came up with evaluations of the teacher's effectiveness that matched those given by his own students after a full semester of classes.

[from Slate; thanks to Marginal Revolution for the link.]

If students can, in five seconds(!!!), provide instructor evaluations that are the same as the ones we get from students who have actually been in the course for an entire semester, there is a very good chance that the standard evaluations conducted near the end of each term are not measuring much more than presentability, performance, and atmosphere. Otherwise I would expect there to be much more of a difference between the 5-second evaluations and the 1-semester evaluations.

It is possible that professors who present themselves better within five seconds also teach better and teach more.

I doubt it.

My skepticism does not, however, mean I would favour abolishing student evaluations of instructors. As I said at the outset, evaluations are probably important for measuring consumer satisfaction with the product universities provide.