Much of what we know about observing and assessing teaching comes from our own experiences as learners, and many widely-held practices are not supported by best practices. We can adopt three broad principles in order to allow even non-subject-expert observers to provide meaningful—and legally defensible—assessments of teaching quality regardless of the subject, level, format, medium, or approach being used by instructors:

• Good teaching practices share seven core elements.
• Know when to do formative and summative evaluation.
• Much of what we observe isn’t actually teaching.

We Know Good Teaching . . . Right?

Many of us who are administrators in colleges and universities came to our current roles through the classroom, the lab environment, or field work. We are familiar with the give and take of teaching, and we now work with, or on behalf of, learners. So, we are well prepared to oversee, observe, and evaluate the teaching that our faculty colleagues are doing for our institutions.

Except when we aren’t.

Meet Francesca. She teaches chemistry courses at your institution. Or English composition. Or microeconomics. Imagine Francesca in your own discipline, whatever you teach now or used to teach (or study). Francesca is a new instructor, and she has been teaching her courses for a few semesters—some sections face-to-face, some sections in a blended formats with 50% live in-class meetings and 50% asynchronous online work, and even a few sections totally online. 

She is coming up on a formal decision point in your organization’s employment processes: perhaps you are observing her teaching as an adjunct instructor in order to determine whether to bring her back to teach more courses, or perhaps she’s on the tenure line and you are observing her teaching in order to determine whether she moves to the next stage of the promotion-and-tenure process.

Take a minute and think. What would you look for when you observe her teaching in order to make your decision? And which of her courses would you observe?

If you are like most of us, the second question is easier to answer: you will likely prefer (perhaps not consciously) to observe the in-class sessions that colleagues such as Francesca are teaching. Common answers to the question of what to look for include things like the instructor’s eye contact, voice tone, number of questions the instructor asks, how clearly she speaks, command of the subject matter, sense of humor, the amount of silence in the class period, presence throughout the space, and the number and quality of the handouts, slide presentations, worksheets, and other materials that the instructor is using. 

If your criteria for observing teaching can be summed up with “I know good teaching when I see it,” you are a) in good company and b) not as prepared for teaching observation as you could be.

These are all excellent indicators, but they do not correlate with teaching quality. Instead, some of them point to the personality profile of the instructor, or his or her performance style. Others contribute to learner success, but are not necessarily things that the instructor creates or controls. Further, some of these indicators don’t exist across all of the environments in which learning interactions take place. How can we observe tone of voice in asynchronous online courses, for instance?

Part of the challenge that all of us face, whether we’ve had long careers in the classroom or not, is that we tend to have been “good students” ourselves in the models that we experienced from our earliest days as learners. For most of us, that means that we flourished in a lecture-heavy, chalk-and-talk, face-to-face classroom environment—and even the most informed among us can be carrying around unconscious biases about what “good teaching” entails. The classroom has evolved from that standardized lecture-based model, and sometimes the classroom isn’t even a physical place any more.

A Word about Us

In higher-education administration today, there is a window that is slowly closing: the number of us who came into our administrative positions before online and technology-enhanced teaching was even a thing. There are fewer and fewer of us every year who can say that we were practitioners and instructors before the blooming of online and mobile learning in the mid 2010s. Even if you are still teaching (and good for you, by the way) or if you are a newer member of the leadership ranks, this article is still for you: at some point in our careers, all of us will find ourselves on the other side of the current-practices divide.

The lessons about observation and assessment in this article apply to any teaching practices and methods that we ourselves have never employed—be they online teaching, tech-enhanced inquiry, the flipped-classroom model, authentic learning, or any other approach. There is bound to be at least one approach that we’ve never participated in ourselves.

The silver lining to this situation is that we are not—and we don’t have to be—subject, practice, or skill experts in order to identify and quantify good teaching. Regardless of how our colleagues are sharing knowledge and supporting learning, we should definitely keep our “spider-sense” of what constitutes good teaching, while also finding data to support those hunches, especially when we are not personally familiar or experienced with what we are looking at in the first place.

I Fought the Law, and the Law Won

Observing and assessing teaching is a task that nearly all of us do in our administrative roles, and almost none of us get any formal training in how to do it. We should. But we usually don’t.

This is worrisome, because the decisions that we make as a result of teaching observations are legally consequential. Asking adjunct instructors to continue teaching for us (or not) and moving tenure-line colleagues forward for promotion (or not) are decisions that, if they are based on “I know it when I see it” or performative aspects of the instructor, are open to challenges in court.

That’s the heavy reason why we should learn how to observe and evaluate teaching, so that we can base our decisions on measurable, legally defensible criteria. The other reason that we should get some “secret boss training” in teaching observation and assessment is that doing it well makes our program-level decisions and processes run more smoothly.

Why is observing and assessing teaching important to get right? Yes, tenure and promotion for tenure-line faculty members and adjunct-instructor retention are legal decisions. But we also want to identify our best instructors. We want everyone to display a minimum level of competency in their teaching—and, yeah, we want to identify those who need more support and help in order to remedy gaps in their teaching skills. Having clear and measurable assessments, based on observation of teaching behaviors, supports all of these goals.

Teaching: It Ain’t What It Used To Be

Take a minute to think as far back as you can about the job of college instructors. What do we actually pay them to do? If you are thinking back ten or more years, your answer might sound like this: we pay instructors to select a textbook and other learning materials for an individual course, write the syllabus, show up to class meetings, facilitate the lectures and interactions, and provide feedback to learners in the form of assignment grades and office-hour consultations.

Especially since 2010, though, many of our colleges and universities are breaking up these tasks into separate roles in order to move toward a more consistent learner experience. Students should get similar knowledge and skills from taking Psychology 201, for example, regardless of whether it is taught by Professor A, Professor B, or Adjunct Instructor C. Content design, on the one hand, and the interactions between learners and instructors, on the other, used to be one job: we hired you to create and then teach a course for us.

Even for traditional face-to-face courses, however, these days the content design and facilitation interaction parts of courses are now separate and atomized. We hire a seasoned researcher or instructor to be the subject matter expert (SME), and we surround that person with an instructional designer, a media specialist, a librarian, a programmer/coder, and so on. 

Why is this change important to know? As observers, we can no longer assume that the person who is teaching is the same person who designed the materials and content for the interactions that we are seeing. Even when the instructor is also the course designer, we cannot use designed elements of what we observe in order to assess the quality of the teaching that is taking place. This happens a lot when we observe our adjunct instructors teaching (more on this below), but the short way to say it is that we can end up criticizing good instructors who are working from poorly designed content, and, vice versa, we can praise weak instructors who are supported by well-designed materials and interfaces.

Now, we should observe and assess the design of our courses, but separately from the teaching that happens in them. Most of the major course-assessment systems that exist (for example, Quality Matters, the Quality Online Course Initiative) focus solely on the design of content, materials, and interfaces—and have nothing to say about teaching practices at all, even though many colleges and universities are shoe-horning such instruments into service to observe teaching (and please stop that right now, if that’s you).

This is so important that two of my colleagues and I wrote a book about it for the online-teaching space, Evaluating Online Teaching, because there is so much confusion about what should “count” toward teaching observations—especially when the teaching takes place in a way, form, or setting with which we ourselves are unfamiliar.

The Three Secrets of Observing & Assessing Teaching

Because we cannot say that all instructors are the same people who have created the materials, tools, and designs within which they are teaching—and because we will never be experts ourselves in the subject areas, teaching methods, and tools that our colleagues use as instructors—we need to pare down our observations to some universal principles of what good teaching is (and what it looks like in various manifestations). So, without further ado, here are the three secrets of observing and assessing teaching, regardless of what is being taught or how:

• Secret 1: Good teaching practices share seven core elements.
• Secret 2: Know when to do formative and summative evaluation.
• Secret 3: Much of what we observe isn’t actually teaching.

These secrets help us to actually do less observing, set up our colleagues for successful assessment, and let anyone observe teaching anywhere, even in formats with which we are unfamiliar.

Secret 1: Good Teaching is Good Teaching

There are a lot of books and articles that argue how various teaching methods are unique and different from face-to-face teaching: teaching online, in the flipped-classroom model, immersive learning, learner-directed courses, you name it. And those resources are correct: all of these formats require different approaches to how teaching and learning interactions take place. But the first secret of observing teaching is that the skills that mark excellent instructors hold true regardless of the format or approach being used.

All competent instructors share these traits, first codified back in 1987 by researchers Arthur Chickering and Zelda Gamson:

• Encourages contacts between students and faculty.
• Develops reciprocity & cooperation among students.
• Uses active learning techniques.
• Gives prompt feedback.
• Emphasizes time-on-task.
• Communicates high expectations.
• Respects diverse talents and ways of learning. (p. 2)

If this sounds familiar to you, that is because these principles have stood the test under hundreds of newer studies, all pointing to these as reliable indicators of teaching quality. In your observations of teaching, look for evidence that the instructor actively strives to perform each of these seven principles, in whatever fashion. Be careful, too, to distinguish teaching actions from designed materials (more on this in Secret 3).

Secret 2: Avoid Surprises

By the time we are observing instructors in their classrooms or online learning environments, it is usually toward the end of the term. At many colleges and universities, instructors receive their first hint that they need some improvement, or that they are our rock stars, only at the end of their efforts. Re-think this framework, and design your assessment system to provide instructors with early and frequent feedback from many sources.

Set up formative assessments from students, peers, instructors themselves, and administrative colleagues well before you set virtual or actual foot in their learning spaces. These formative feedback instruments often take the form of mid-term or early-course quick-check surveys, with the format of one closed-ended and one open-ended question. 

Ask learners what percentage of the course prep (for example, readings, video watching, practice) students are actually doing (100%, 80%, and so on), and then an open-ended question like, “What is one thing that is getting in the way of you giving your best effort to this course?” If these questions come at the end of Week 2, instructors can actually respond to them and make adjustments in the pace, content, and interactions in the course. 

They will also get valuable feedback when their peers or administrators make time to look at instruction happening early in the term, when feedback doesn’t yet “count” toward those legally-binding purposes you saw earlier. Finally, provide self-assessment instruments to your instructors: knowing what questions to ask about one’s teaching provides reflective opportunities for strengthening practices. Yes, this requires a bit more time and energy than just walking in at the end of the term, but it pays us all back in the form of increased learner persistence, retention, and satisfaction.

Secret 3: Teaching Is Less Than We Think

So, if good observations and assessments are founded on an increased amount of formative feedback, where can we observers actually save time and effort? By not looking at behaviors that are not actually correlated with teaching quality. An awful lot of things that we can observe aren’t actually teaching, and the items on this list might be surprising to you: eye contact, voice tone, use of humor—all of those things that we listed at the very beginning of this article—do not correlate with teaching quality at all. As observers, we can get so focused on the quality of instructors’ performance, that we fail to see how razzle-dazzle hides poor teaching skills. Notice that in Secret 1, none of the effective teaching principles had to do with having a radio-host voice or keeping eye contact with learners. 

Rather, think about all of the interactions that happen in a classroom or online learning environment. The majority of those interactions are either sharing information (lecturing) or housekeeping (keeping conversations going, assigning grades). Further, we cannot infer good teaching by looking for its effects. Outcomes like student learning gains, learner satisfaction, and demonstration of accreditation standards are what the philosophers call “sufficient but not necessary” results. Good teaching could produce them, but so could a crop of self-motivated and well-prepared learners who make gains despite a poor teacher.

So, if good teaching is much less than what we can observe in a 50-minute class period, what is it, exactly? 

Good teaching is creating the conditions for learning to take place in the moment, facilitating the interactions that learners have with content, each other, their instructors, and the wider world. 

Lots of factors influence student learning: 

• Institutional resources (do we work in places with large staffs and lots of funds), 
• The program and curriculum (are they well-designed and robust), 
• Institutional context (are we a Research 1 focused place, or a teaching college), 
• Student characteristics (lots of first-generation learners, perhaps),  
• Technology access (both on campus and among our learner populations), 
• Teaching behaviors, and 
• Course content/design.

Notice that we cannot count any of the first five items in this list for or against individual instructors. For instance, if we are observing an instructor whose learners come predominately from first-generation student situations, we cannot fault the instructor for learners who struggle with the “hidden curriculum” of how to be college students.

In fact, only two of these factors have any bearing on our observation and evaluation of teaching: content and teaching behaviors. As we’ve seen above, content and design are the materials and resources for learning that are prepared ahead of time—things like slide decks, visuals, videos, assignment directions, resource lists, and other materials. These are all items that exist before Day 1 of our courses. We should not count these in our assessment of teaching quality.

Teaching evidence is all of the instructional interactions done in the moment of learning need: demonstrations, discussion facilitation, instructional supplement (for example, creating a quick 30-second video to respond to an online question from a learner), administrative tasks, and assessment and feedback. Teaching behaviors are thus the interactions that happen after Day 1 of our courses.

We can look for and assess teaching behaviors in four “big buckets,” like this:

By looking for measurable teaching behaviors that align with Chickering and Gamson’s principles of good teaching, we ignore a lot of inputs that might look flashy, but which do not actually correlate with teaching quality—making our task simpler and easier to measure.

A Word about Online Teaching

It should be evident by now that teaching online, although it requires a different skill set than teaching in a classroom, still relies on the same qualities and principles that Chickering and Gamson identified. Online education is often the tail that wags the dog, because we can observe interactions that would be difficult or impossible to observe in face-to-face environments, such as e-mail messages among learners and instructors, as well as the detailed feedback and grades that instructors provide.

This brings up one last piece of guidance for observers: just because we can observe an area of online courses doesn’t mean that we should do so. In fact, it is useful for us to limit our observations to similar time frames to the ones we use for our face-to-face observations, and ask online instructors to select their best units for us to look into, along with one more unit that we select randomly.

But what should we look for, specifically? Tony Piña and Larry Bohn performed a meta-analysis of online environments and recommend looking for the following measurable indicators of teaching quality online (note, too, how they map onto Chickering and Gamson’s principles):

• Assignment feedback speed & quality [against promised results in the syllabus or learning environment].
• Discussion frequency & quality [aim for 10-15% of total participation from the instructor].
• Student-question answer speed [against promised time frame in the syllabus].
• Announcement frequency [once or more per defined unit of the course].
• Personal & contact info [that contains multiple ways of contacting the instructor].
• Log-in frequency [on at least one more day per unit than students are required to log in] (p. 25-34).

For much more about observing and evaluating online teaching than we can discuss in this article, check your library’s holdings for Evaluating Online Teaching: Implementing Best Practices.

We Know Good Teaching. Right!

Now that you have a little bit of “secret boss training,” you can take away a few high-level ideas:

• Always focus your observations and assessments on measurable and observable interactions between instructors and learners. 
• A lot of what we can observe isn’t actually teaching, and we can start to focus on teaching behaviors by untangling our assessment of the quality of content/materials from the quality of teaching interactions by knowing what factors influence student learning other than teaching (and there are a bunch), and by looking specifically for the interactions that take place after Day 1 of the course. 

I speak and consult with colleges and universities about how to evaluate teaching, and I’d love to hear more of your story, if you’d like to connect. And a final word: this secret boss training should not be so secret, so spread the word among your colleagues, and become a secret boss trainer yourself.

---

About the author

Thomas J. Tobin is the Conference Programming Chair and Faculty Associate on the Learning Design, Development, & Innovation (LDDI) team at the University of Wisconsin-Madison, as well as an internationally-recognized speaker and author on quality in technology-enhanced education, especially copyright, evaluation of teaching practice, academic integrity, and accessibility/universal design for learning. He holds a Ph.D. in English literature, a second master’s degree in information science, and professional certifications in project management (PMP), online teaching (MOT), Quality Matters (QM), and accessibility core competencies (CPACC). Tom serves on the editorial boards of InSight: A Journal of Scholarly Teaching, the Journal of Interactive Online Learning, and the Online Journal of Distance Learning Administration.