Let’s Try A New Form Of Teacher Evaluation

by Viktor Nordmark, Hubert.ai

Student evaluations of teaching have looked pretty much the same since they were first introduced, close to a hundred years ago.

To summarize what most think about them is pretty easy:

Nobody likes them.

They can’t accurately measure how effective a teacher is.

They are hard to design and analyze.

They are the best tool currently available.

So, let’s replace them with a better system.

Student evaluations have had a rough time recently, numerous blog posts and articles(1, 2, 3, 4, 5) describing the many flaws have gone viral in the teaching community. Some even argue that they should be completely abolished. There’s simply no reason to listen to student voices as they don’t know how to teach or how knowledgeable a teacher is in a subject.

Is this true?

See 30 Things Successful Teachers Do Differently

If we are to trust students to become the leaders of tomorrow and to invent smart solutions to problems created by previous generations, we must be able to trust them to, at the very least, provide meaningful feedback to teachers.

There may be unfair or low-quality feedback, but ‘bad data’ is a challenge in every industry. This is a solvable problem. There’s just no getting away from the fact that students experience the teacher’s expertise first hand, and are thus a prime source of information that can be used to develop better teachers.

Since student evaluations of teaching are, in fact, the most researched question in respect to teaching in higher education, there are many clues as to how a better evaluation system should be designed. For example, validated evaluation tools always treat student opinions as one data source that, together with others, can be used to make up the full picture. Stating that student opinions are, in fact, a data source and nothing else makes it easier to obtain clear data and to form an objective relationship between students and student evaluations of teaching.

What’s difficult in developing a better system is that effective teaching is extremely complex to reliably measure. There’s not even a general agreement regarding what constitutes effective teaching.

The most agreed-upon aspects to produce a holistic view of effective teaching include Student Attitudes, Teaching Practices, and Student Achievement (Benton et al, 2012). But there are plentiful examples of other aspects that could influence the symbiotic teacher-student relationship.

Once you understand what constitutes effective teaching from the teacher’s perspective is when you can start getting down to what is truly useful with performance feedback: How to actually improve your teaching.

A better way to understand how the teacher’s role impacts learning, what individual characteristics to enhance and reduce, and in the end, how to become the best teacher possible.

That’s what a new feedback system truly should aim for.

And of course, preferably remove some of the large pain points in the current system while we’re at it. Issues include questionable validity, present bias, low response rates, no censoring of vulgar language, time-consuming analysis, and more, just from the teacher’s perspective.

Boring, time-consuming, and purposeless, say students. So, how can we fix this?

Well, for a new system to be fully adopted, the most pressing issue that needs to be solved is measuring teacher effectiveness in a more reliable way that also provides feedback actually useful to the teacher. Today, most teacher evaluations take Student Feedback as the only source of input when assessing a teacher’s skill. A few have started to take Peer Observations into the greater picture to get a more extensive view.

That’s a good place to start, but there are more possibilities. Here are a few rolled into a plan.

A New Plan For Evaluating Teachers

The plan is to take in as many useful data sources as possible, mash them together, crunch the numbers, adjust for common bias, and finally spit out comprehensive recommendations on best practice methods that the teacher can choose to act upon. Over time, when the system is being used around the world, it’s even possible to cross-examine data to see trends in what methods work better for a specific subgroup or discipline.

To get started, the first obstacle is to design a system that permits students to easily leave constructive, qualitative feedback while keeping bias to a minimum and filtering out personal attacks and foul language at the same time. Achieving this in a plausible way by using a survey is impossible.

See also 8 Course Evaluation Templates To Get Feedback From Students

Could technology be the answer?

The latest technology advances could come really handy when designing an evaluation system for tomorrow. What I have thought of, is that a chatbot could act as a feedback intermediary between the teacher and the students and bring along a bunch of advantages over surveys.

Using a chatbot to collect feedback is the ultimate compromise between a qualitative and a quantitative research method. A chatbot can collect opinions through a conversational interface with the same advantages as a ‘real’ interview but with a fraction of the required work.

The conversation can be tailored according to the responses and personality of the student, ask follow-up questions and find out the reason behind opinions. Feedback is kept factual and relevant to teaching while filtering out irrelevant and insulting replies.

As more data is collected, the goal is to be able to provide teachers with insights on how to become more effective in their teaching based on student and teacher data from all over the world.

So far, there’s only a free beta version of the software out, but as more and more teachers join in on the action, it will be possible to create a more extensive version soon.

References

1. Anya Kamanetz, Course Evaluations Get an ‘F’. Available at: http://www.npr.org/sections/ed/2014/09/26/345515451/student-course-evaluations-get-an-f

2. Harvard Business Review, Better Teachers Receive Worse Student Evaluations. Available at: https://hbr.org/2014/09/better-teachers-receive-worse-student-evaluations

3. Marcotte, Best Way for Professors to Get Good Student Evaluations? Be Male. Available at: http://www.slate.com/blogs/xx_factor/2014/12/09/gender_bias_in_student_evaluations_professors_of_online_courses_who_present.html

4. Patton, Student Evaluations: Feared, Loathed, and Not Going Anywhere. Available at: https://chroniclevitae.com/news/1011-student-evaluations-feared-loathed-and-not-going-anywhere

5. Stark & Freishtat, An Evaluation of Course Evaluations. Available at: https://www.stat.berkeley.edu/~stark/Preprints/evaluations14.pdf

6. Benton, Stephen and Cashin, Student Ratings of Teaching: A Summary of Research and Literature. IDEA Paper, IDEA Center, 2012