October 26, 2021

The SETC Improvement Project: Teaching assessment and student learning experiences

The feedback students provide through SETC surveys can help instructors improve their teaching practice—but it shouldn't be used to assess teaching effectiveness. The SETC Improvement Project will educate students, faculty and academic administrators about appropriate uses of student perspectives.

At the end of every undergraduate course, SFU students receive an online SETC (Student Experiences with Teaching and Courses) survey asking for feedback about their learning experiences. There has always been confusion among students as well as some faculty and academic administrators about how this feedback is—or should be—used, especially in the context of teaching assessment. In Fall 2021, the university will launch the SETC Improvement Project to give students and those who use the surveys a better understanding of their purpose. In addition to an information/education campaign, the project will include a consultation to consider changes to the way the surveys are structured. Elizabeth Elle, associate VP, learning & teaching, answered questions recently about the project and its connection to teaching assessment. 

What is changing around teaching assessment?

Elizabeth Elle: With a new Collective Agreement, it’s timely to pay attention to how we assess teaching. The Centre for Educational Excellence has posted a number of resources that TPCs [tenure and promotion committees] can use, including peer assessment and self-assessment tools, both essential components of teaching assessment. Now we are working to improve the instrument we use to capture the third component: student voice. For this, we use SETC [Student Experiences with Teaching and Courses] surveys, and our goal is to help both students and instructors understand what SETC can and cannot do.

I thought SETC couldn’t be used for teaching assessment?

EE: SETC feedback is an indicator of the student learning experience and can be really useful for giving TPCs an understanding of that, and for providing faculty an opportunity to reflect on and improve their teaching practice. But it cannot be used as a direct measure of teaching effectiveness and has often been misused in that way, for instance by comparing numerical averages among faculty—the absolute worst thing to do! Addressing the confusion around this is part of why we are launching a project around SETC.

Why can’t average scores be compared?

EE: There are many things that contribute to the scores, and without that context, comparisons are meaningless. The context that should be considered includes whether the course is upper or lower division, required or elective, class size, and lots of other factors. The frequency distribution of scores can provide insight, but straight-up comparison of means leads to conclusions that are just not appropriate. Helping TPCs understand best practices is part of the larger educational component of this project.

I’ve heard that response rates are lower with SETC compared to the old paper evaluation forms.

EE: They can be lower. One of the items we will do is provide a report on how to improve them. For instance, as we return to in-person teaching, instructors can set aside time in class for students to complete the online surveys, just as they would have in the past with paper surveys. The literature suggests other options, such as allowing earlier access to grades for students who have completed SETC, but we need to understand whether this is something our community wants.

The current SETC forms are quite long. Could that be hurting the response rates?

EE: We are wondering the same thing. One item we are going to consider carefully is whether we keep the entire cascading framework—the core university questions, instructor questions, and those set by departments and Faculties. The latter two don’t always have a clearly defined purpose, so we will be working with Faculties and departments to understand whether it makes sense to keep them.

What else is planned with this project?

EE: When SETC was first introduced, it was suggested that we needed to do more to help students understand the purpose of the surveys. We’re planning to do that in the coming months. Engaging with students should lead to both better response rates and more thoughtful answers and is one of the first things we will launch this year.

Related links

Visit the SETC Improvement Project web page.