My university just announced that we have joined a MOOC (massive online open course, for those of you who like me didn’t know what the acronym stood for). Specifically we joined EDx. The decision was made at a level far above my pay grade. No one asked my opinion, which is actually a good thing because I’m not sure what my opinion is at this point.

There are lots of good, altruistic reasons for higher education to take this step. The intent is to make high level education available widely at a very low cost to the consumer (not to the institution, by the way). We’ve seen how freer access to more information has put the fear of Truth into even the most dictatorial systems to the extent that they attempt to shut it down as fast as they can and find that they can’t. I like that part. I also like the possibility that freer access to information that is based on solid research and realistic thinking might empower people to be less subject to thinking based on superstitions and misinformation (not that it’s working that well here). More selfishly, I like the idea that really good teachers could be challenged to change the way they think about learning and put their talents to work finding new ways to structure learning environments that can handle the ever-expanding population of students with widely varying backgrounds.

But I think there may be lots of good reasons for us to take this step more slowly. Despite all the miraculous claims of how transformative technology can be, right now there is still a lot of work to do. Even the very successful Khan Academy is founded on a lot of delivery of information, granted by a very gifted deliverer. But information is not synonymous with understanding, and delivery is not synonymous with education. So before I rush into volunteering to create courses for our new MOOC, I thought as an educational psychologist I might take a crack at analyzing what I know about learning that needs to be considered in the process. Here it goes.

Learning means focusing attention on the key concepts in a topic. OK, online learning can take advantage of the magic of visual images and presentation strategies to highlight key ideas and put them in a form that will not only draw learner attention but create memorable images that will be easy to recall later. Check!

Learning means making connections with a learner’s prior knowledge. Hmmm, maybe not so easy here. Given the size and diversity of the audience for a MOOC, being able to make those connections in the presentation itself is going to be difficult. It’s possible that one of the affordances of technology—gathering and analyzing data—might make this possible. After all Amazon can keep track of the books and other things I buy and point me toward other books of the same genre or that other people like me (i.e., those who bought the same things I bought) liked. So it might be possible in the future for an online course to analyze the kinds of interests and background I have (from a population perspective) and offer me links from the information of this course to other examples on the web that are related to it. I’m pretty sure that doesn’t exist in the programming yet, but it could. It might take a while to build up enough of a database (computers ARE good at collecting data) to make those predictions, but it’s possible.

Learning means actively processing the incoming information, digesting it, working with it, summarizing, paraphrasing, applying it. Yes, that works online, too, provided the information delivery part of the course leads to opportunities for the learners to try out the ideas in various ways. Most online classes have finally gotten around to realizing that just listening to information is not the process of learning. As a result most of them include activities for the learners. That’s good, but here’s the stumbling block.

Learning also requires that the learners’ attempts receive guiding feedback. Uh-oh. That works pretty well in areas where there are “correct” answers that can be evaluated readily by a computer, but there’s a ways to go before it works in more complex content that involves decisions and judgments. I actually ran into this problem recently in a project to provide instruction in teaching to adjunct faculty using an online program. It was easy to respond to questions that had simple answers, like who suggested the idea of the Zone of Proximal Development (Lev Vygotsky). But evaluating the significance of the ZPD for teaching requires a whole other level of sophistication from the learners. Not so easy to anticipate all the things a learner might offer as important. This is not to say that it can’t be done, but rather that it can’t be done so easily YET. There are ways of providing electronic feedback to this kind of active learning. Our solution was to provide examples of answers that would fit the task and let the learners compare theirs. Not totally satisfying and sometimes not totally accurate.

There are some possibilities to consider when contemplating this issue of feedback. One is the “community of learners” possibility. I think in the vernacular of MOOCs this would be called “crowd sourcing.” It’s a more elaborate version of peer feedback, where the large group of learners respond to one another’s ideas in hopes of finding some kind of consensus. I think this probably works in an informed community of participants where there is a distribution of prior knowledge that can be drawn on. I think a community of novices still needs the guidance of a more informed individual or group of individuals.

Another possibility that has been around, but not perfected in education yet is artificial intelligence providing individual tutoring based on expert models and language matching. A new wrinkle that might make this more feasible is the idea of learning analytics and large databases. When this idea first came around, I was fascinated, but skeptical. I have seen too many really strange thought trails in my students writing to believe that even a computer could follow and guide them. But possibly the computer just needs more information to develop a case inventory of potential student ideas. I guess if they can create a computer that can win at Jeopardy, they can figure out how to interpret student responses to open ended questions. But I (and many other psychologists) believe that the essence of deep learning is in the interaction with others as we grapple with what we think we know versus what we really know. That’s the kind of online learning I’d like to see us build.

So am I going to try this thing? I’m conflicted. I see a lot of possibilities, but we are definitely not there yet. The technology world might be there, but those of us in education take longer to learn. Too busy giving guided feedback.

CONTACT:

msvinicki@mail.utexas.edu

A recent study (Bentley & Kyvik, 2012) found that faculty in the United States spend on average over 50 hours per week on the job, and of those hours, over 20 are spent in teaching activities. These hours can be much higher for faculty at certain stages of their career or at certain kinds of institutions, but regardless, we spend a lot of time at our work. But more isn’t necessarily better—we don’t measure productivity in academia in terms of hours logged. What are we gaining by the time spent? And are we finding the time we spend meaningful and rewarding?

What constitutes productivity in teaching is a point of debate, of course, but many of us agree that we want to facilitate student learning. When faculty are challenged to change traditional teaching practices to promote better student success, all we may see looming before us is additional class preparation time. The best kept secret, however, is how much more time-efficient some of these touted teaching practices are. Below I discuss three of these best practices and the positive impact they can have on the way we spend our time teaching.

Begin with the end in mind.

The principle of backward course design (Wiggins & McTighe, 1998) echoes one of the late Stephen Covey’s Seven Habits of Highly Effective People (1989). In this model, course design begins by determining what it is that we really want students to be able to do or feel or think long after the final exam is over. Then we make every other aspect of the course serve those goals. Once we have articulated our goals, whether lofty or pragmatic, our next step is to determine how students will demonstrate to us that they have indeed achieved the kind of learning we want of them (assessment). Lastly we turn our attention to the class format and activities that would facilitate that achievement. Aligning these three facets of course design (goals, assessments, and activities) builds in a coherence and synergy in the course that creates greater opportunities for students to learn what we want them to learn.

Clear course goals that communicate the nature of our disciplinary work to students tend to take the form of “At the end of this course students will be able to: explain concepts such as…, develop a thesis (or hypothesis), analyze data (or texts or images), contrast points of view on issues, or write cogent arguments based on research (or analysis).” We may assess students’ achievement of these goals through exam questions, papers, or projects, checking that the level of thought or skill that we want students to gain is represented in those assessments. Key to alignment, however, is making sure that we give students opportunities on lower stakes assignments or during class activities to practice the same skills we want them to ultimately demonstrate on our assessments.

The backward design approach helps focus our course efforts, not only generating better chances for students to learn what we want them to learn, but also saving class preparation time in at least three ways. First, we spend less time deciding what readings and assignments to include in our course because we now have targeted criteria to use to make that determination—our course goals. Second, we design our assignments around those course goals, so we spend less time grading or responding to assignments that don’t accomplish what we had hoped and are, in essence, busy work for our students and for us. And third, we are more apt to restrain ourselves from taking on too much in the course. Articulating our goals rather than masking them in a generalized descriptive statement (e.g., “In this course we will discuss the effect of global economics on world trade”) helps us see more clearly the demands we are placing on the novice learners in our disciplines.

Generate criteria or rubrics to describe disciplinary work for students.

Once we have clear course goals we can use them to generate criteria or rubrics, a time-efficient approach to grading. We faculty know what quality student work is when we see it—but our students do not. Disciplinary work is a mystery for students. As faculty we may have forgotten what it was like to be a novice learner in our field (a phenomenon known as expert blind spot), or we may have been more intuitive about these processes even as students. In all likelihood, we faculty were not representative of the other students in classes with us at the time. We are a self-selected group that shares little in common with the vast and diverse array of contemporary students. Providing students with criteria or rubrics gives them a glimpse into the way that we think.

Sharing with students the criteria that we will use to evaluate their work both models disciplinary thinking for them and helps them develop the ability to evaluate their own work. Although we may think that these kinds of guides “give it away” and make our assignments too easy for students, rarely is this the case. Instead these sets of criteria or rubrics can be a motivator for students. They make the assignment less of a mystery and make the students’ own success seemingly more under their control. For examples of rubrics in many disciplines see Walvoord & Anderson, 2010 (pp. 195-232).

Using criteria or rubrics to grade student work saves time by: helping students produce better quality work (and better quality work is both faster and more pleasurable to grade); allowing us to assign points more quickly and consistently as we grade; and providing clear criteria for us to use in talking to students about their grades. When a student comes to appeal a grade, we can ask her to explain how the work meets the criteria. So the session becomes less about faculty defending their judgment, and more about helping the student learn to evaluate work from a disciplinary perspective. Although it does take time to generate really useful sets of criteria or rubrics, we can use them over and over and adapt them to multiple purposes.

Embed “assessment” into assessments.

Generating criteria for student work also serves another purpose that is time-efficient—it helps us in our assessment of student learning outcomes for institutional purposes. Although assessment of student learning in terms of assignments, tests, and papers is second nature to faculty, “assessment” in the sense of tracking student learning outcomes is often considered a four-letter word. In reality, determining precisely what students are learning in our classes focuses our scholarly minds on our teaching. Just as we look for evidence to make arguments for our theses or hypotheses in our discipline, when we assess student learning outcomes we determine if our courses are accomplishing what we planned in terms of student learning. Based on what we learn, we can change our courses to make them more efficient in producing the outcomes we want.

Assessment processes have been criticized for consuming time without producing results. Because data collection often occurred at levels at the institution beyond the course (or possibly even department) level, it seemed removed from day-to-day course activities and needs (Hutchings, 2010). Some of the most meaningful assessment, however, is data that we as faculty collect about what activities engage students most productively, what concepts and skills students find most challenging, and what interventions advance student progress. The key to making the assessment requirement work for us is to embed our assessment of student learning outcomes into regular class assignments, exams, papers, and activities.

Faculty are accustomed to assessing student work with a grade. When we think about student learning, however, a grade represents a composite accounting of all the knowledge and skills we ask students to demonstrate on a piece of work. Assessing student learning outcomes requires us to deconstruct or unpack what that grade represents. What specific kinds of knowledge and skills did students demonstrate on a graded piece of work? For example, if our goal is to develop students’ critical reasoning abilities in our discipline, we may record the level of students’ performance on certain test questions that are specifically directed at that goal. These questions may be multiple choice or short answer, in which case we keep track of correct student responses. Or we can examine students’ performance on an essay using criteria (or a rubric) that capture the elements of critical analysis that we want students to demonstrate (see above). We then keep track of students’ rubric scores to determine what aspects of analysis they have mastered and what aspects they need to improve.

The information gained from monitoring students’ performance makes our teaching more time-efficient by directing our choices on class activities and assignments. For example, rather than lecturing on all aspects of course material, we focus class activities on those areas that students find most challenging. Likewise, we spend our preparation time designing and responding to assignments that are targeted more directly at developing key skills in students. The time we spend is more likely to produce the kind of learning we want in students.

Summary

Redesigning our teaching based on recognized effective teaching approaches does require an investment of time upfront. But that investment pays off every day, all year long. And our time is often spent in more intellectually satisfying interactions with students, increasing our sense of productivity, and making the time more meaningful and rewarding.

References

Bentley, P.J., and S. Kyvik, S. 2012. “Academic Work from a Comparative Perspective: A Survey of Faculty Working Time across 13 Countries.” Higher Education, 63: 529-547.

Covey, S. 1989. The Seven Habits of Highly Effective People. New York, NY: Simon and Schuster.

Hutchings, P. April, 2010. Opening Doors to Faculty Involvement in Assessment. National Institute for Learning Outcomes Assessment Occasional Paper #4. Retrieved from www.learningoutcomesassessment.org

Walvoord, B., and V. J. Anderson. 2010. Effective Grading (2^nd ed.). San Francisco, CA: Jossey-Bass.

Wiggins, G., and J. McTighe. 1998. Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.

CONTACT:

Linda C. Hodges

E-mail: lhodges@umbc.edu

We begin the fall semester with a Janus head, looking back in some ways and looking forward in others and trying not to lose sight of where we are at the moment. Mike Rodgers’ TECHPED, for example, looks ahead to a future in which online learning will certainly be even more a part of higher education than it has become already. At the same time, Mike stops to ask where are we today in this process.

All of our days seem rushed, overburdened, at times confused. New ideas about teaching (new buzz words about it, anyway) pop up every semester and committed faculty stop to listen to see if there might be something there to help them be more effective. Many good ideas have come along in the last twenty years. The cover feature by Linda ­Hodges of the University of Maryland highlights three of these that not only help improve teaching effectively, but also save faculty time. When I read Linda’s piece, I experienced a personal epiphany. Some of these approaches—rubrics, for example—had sometimes felt to me as though they were a kind of “talking down” to students. Suddenly I felt the scales fall from my eyes: I saw them as meeting students where they might be, that is to say, outside of the intellectual territory and habits familiar to faculty, and guiding them along with the kind of nourishing bread crumbs we all need when we are new.

Jill Collins White of the University of Wisconsin–Green Bay leaps into a bold new future when she elects to forego giving formal exams. The more closely she knitted continuous, embedded assessment into her classes, the more she saw the difference between how students learned and how they learned for exams. She sought the former and not the latter, and so she finally said goodbye to all that. For exams, students memorized and that wasn’t the kind of learning Collins was after.

Of course there is a place for memorizing, an old and honored place. Libby Jones’ (Berea College) article on the personal value of memorizing a favorite poem recalls some of the deeper human values nourished in some long-abandoned pedagogical practices. If I may confess (and brag) a bit here: when I was in fifth grade, we were all required to memorize and recite a poem. Naturally the race was on to find the shortest acceptable poem, but as I came from an emoting family and could not resist the impulse to show off, I stunned the class by reciting Poe’s “The Raven.”

Ah yes, looking back: in this issue I look back to second grade as well, because I learned some important lessons about teaching there, though they were learned from negative examples and thus taught backhandedly. I guess there was some looking forward embedded in those experiences too, however, in that I experienced a lot of AV long before it became completely expected in teaching.

And Marilla Svinicki’s AD REM . . . also looks forward. Marilla imagines a future in which we learn to know our students much better through the collection and analysis of data about them and their choices so that, in the end, we are able to teach them more effectively. “To know as we are known” has a familiar and hopeful ring to it. Perhaps “learning analytics” may bring us a bit closer to achieving our aims as teachers.