Lion F. Gardiner
Department of Biological Sciences
Rutgers University

The exercises and resources below draw on hundreds of studies of students and learning in colleges and universities carried out in the last two or three decades They provide practical ideas and tools to help us clarify what we're trying to accomplish as teachers and to understand how well we are accomplishing it. They can help us understand our student clients and thus help us help them develop societally and personally important abilities.

We in higher education respect research. Most of us have been trained as researchers in our disciplines, and many of us devote much of our lives to research and scholarly work. But, while the last 30 years of research have provided us with considerable knowledge that could enable us to be far more successful as educators, most colleges and universities do not use this research. They—we—continue to use the far less effective methods of the past.

Redesigning Higher Education pulls together the findings of hundreds of studies of students and learning in college so that busy faculty and administrators can have ready access to some of the most useful research.

The following exercises, tools, and resources come out of having written Redesigning Higher Education. Drawing especially on five Conditions for Educational Quality (p. 23) and seven research-based professional practices (p. 107), they sample a few of the methods available to help improve the quality of students' learning. They focus on students' cognitive development, claimed by most college faculty members to be their highest priority as teachers. Page references to Redesigning Higher Education refer to fuller discussion of each issue.

What are we trying to accomplish? Clear goals and objectives (pp. 24, 107)

Research tells us that an organization can greatly increase its productivity by being very specific about the goals it is trying to achieve. Today, we in higher education are being asked to substantially improve our productivity—to improve the quality of our results, our students' level of development.

One of the most helpful ways we can improve our students' learning is to clarify our intents. Most institutions, departments, and faculty members do not clearly specify their intended outcomes. We aim at whatever we think will be most useful, but may be pulling in different directions. Lacking clear guidance, students may have to guess at what they should learn.

Thinking About Outcomes: Two Exercises

1. If you have written statements of broad course goals, take a look at them.

If you do not have a written list of course goals, reflect on your course and list the four to six most important student outcomes you want your course to produce.

2. Look over your list and check the one most important student outcome. If you could only achieve one outcome, which one would it be?

3. Look for your outcome on the list of key competencies or outcomes society is asking us to produce. Is it there? If not, is the reason a compelling one?

4. Check each of your other "most important" outcomes against the list of outcomes. How many are on the list of key competencies?

5. Take stock. What can you learn from this exercise about what you are trying to accomplish as a teacher? How clear and how important are your statements of outcomes for your use and for your students'? Are they very specifically worded to avoid misunderstanding. Are they supporting important needs on the part of the students?

The Teaching Goals Inventory is a sophisticated tool for helping faculty clarify for themselves what they are trying to accomplish as teachers. Complete the TGI, which you will find in the Angelo and Cross (1993) reference. The authors explain how to use the TGI. What can you learn about yourself as a teacher from the TGI? If you work with graduate teaching assistants, try it with them, too. Use it as a basis for reflection and discussion, either one-on-one or as a group.

And How Successful Are We? Knowing Results (pp. 24, 60ff., 109, 125)

Research tells us that knowledge of results—feedback on how well we've actually reached our goals—is another key aspect of improving the quality any organization's results. How effectively are your students learning? Are they achieving your goals? How do you know?

Most of us design classroom tests to assess our students' learning. When we give students assessments such as paper-and-pencil tests, we are giving them mental measurements, and the stakes in accuracy of knowledge of performance, self-esteem, and grades associated with these measures are often high. They require careful design just as do professionally constructed assessments such as national standardized examinations.

How accurate are your assessments? Do they actually assess your students' learning? Do they test what you think we're testing (their validity)? Do they perform in a technically consistent fashion (their reliability)? How do you know? Research has shown that most classroom tests assess low-level cognitive material and have uncertain validity and reliability. Our interpretations of our tests' results are only as good as the accuracy with which they measure student characteristics. The books by Angelo and Cross (1993) and Jacobs and Chase (1992) can provide much of the help you may need.

How do you determine what you're asking your students to do intellectually? The most common formal method used by teachers to set the level of intellectual responses required by their students on questions asked in class, on assessments, and in assignments is the Taxonomy of Education Objectives. Developed by a committee chaired by Benjamin Bloom (1956), this straightforward and relatively easy-to-use classification of cognitive skills has six levels:

The first three of these levels are thought to build hierarchically on each other, and the latter three on them. Analysis, Synthesis, and Evaluation are often known as problem-solving skills. Their relationship to each other is less clear.

• Study Bloom's book; the Taxonomy can be a tool you use daily in your work.
• Ensure it is in your campus and department library, if not, then in your own.
• Using the book, prepare a handout describing the Taxonomy to help you teach it to your students to help them develop their skill at metacognition, or thinking about their own thinking, and in understanding and planning their own cognitive development.
• Use it each time you design a quiz, examination, or other type of assessment to determine what intellectual demands you are making of your students.
• If you've never used the Taxonomy, try this:

Who Are Our Students? Knowing Our "Customers."

Successful organizations of all types have long known that understanding their customers well is an essential step to meeting their needs and satisfying and keeping them. We in higher education have many customers, for example, employers, taxpayers, donors, governors, legislators. Although our students are different in some important ways from customers of, say, a supermarket or car dealership, we also exist to offer a service and we claim students' tuition and the people's taxes to educate them. As educators, they and their development are our major professional focus. As educators their development is our raison d'être. To serve them well we need to understand them well.

Research has revealed the importance of building relationships with students as a device to improve their retention on campus and learning. How well do you know your students? With how many of them do you have a "personal" relationship of some sort? What information do you have about them? How accurate is it? How do you use it to mold your professional behavior?

How much learning do your students do? Student quality of effort (p. 51)

When all is said and done, it is our students who do the learning, not their teachers. If our students are not working effectively, they will not learn. Student quality of effort is one of the most important variables related to the learning outcomes our courses and institutions are able to produce (CSEQ, Pace, 1990).

Here are four questions to ask that may help clarify your understanding of your students' learning effectiveness and improve their learning.

1. How long should your students study for your course outside of class sessions to achieve high-quality learning results? This effort can be expressed as hours per week, on average, or hours out-of-class per each hour in class.

2. How long do they study? How do you know? How can you find out? The easiest way may be to ask them! Cut printer scrap into quarters and pass out these slips at the beginning of a class period. Ask your students to think back carefully over the last seven days to the times when they studied for this particular course and determine the lengths of time they studied. Ask them to add these times together and write down on the slip the total time they studied to the nearest whole hour. They should not put their names on the slips if you want relatively accurate data. Although these are self-reports, the patterns that emerge from your classes will probably be robust, usable, and quite informative.

Collect the slips and calculate (a) the mean number of hours studied per week, (b) the percentage of the class who studied at or above your minimum number of hours for high performance in (1), and (c) the percentage who studied one-half or less of your minimum number of hours.

Sit back and reflect on the significance for student learning and your teaching effectiveness of the data you have collected. Do this several times or plot the data weekly to construct a graph for the semester. What patterns emerge? How do different classes differ? sections of the same class?: with the same, with different teachers?

3. How effective is your students' out-off-class learning? How do you know? Most of what students learn in a course is learned outside of class sessions. The time they invest and their motivation and learning strategies are crucial variables in their learning.

Using the paper slip method in (2) above, or a more formal questionnaire, ask them this: "Have you ever been taught how to study in high school or college, for example, in a special study-skills course or seminar or as part of one of your regular courses?"

Calculate the percentage of your students who claim to have had any instruction in how to study. The larger the number of students in your sample, the more confident you can be of the reliability of your results.

What are the implications of the data for your students' learning and your success as a teacher?

4. Reflect on your findings in 1 to 3. What is the current quality of learning effort expended by your students? What are the implications of your conclusions for your success in achieving the outcomes you said were most important in the first exercise?

How skilled at critical thinking are our students? Learning more about them (p. 9ff.)

Faculty often cite the development of their students' capacity for critical thinking--a careful, self-aware, truth-finding type of higher-order thinking (Paul, 1995),--as the number one outcome they are trying to achieve through their teaching. Yet they often believe their students are far less capable when struggling with ideas than they would like.

Dr. William Moore has developed an easily administered objective format instrument for assessing students' levels of epistemological development, the Learning Environment Preferences (LEP). The LEP asks students questions about the environmental conditions they prefer as learners. These preferences are strongly influenced by a students' current level of epistemological development. The LEP can help teachers understand their students' cognitive development.

When students receive their LEP scores they also receive a handout giving them suggestions on how they can improve their learning. The LEP can be used to help your students become more effective as learners. It does this by helping them understand their own cognitive development, thus becoming more metacognitive and developing into more critical thinkers.

Research shows large numbers of American college and university students believe in the reality of paranormal phenomena such as UFOs and alien invaders, the Loch Ness Monster, Bigfoot, reincarnation, the Bermuda triangle, and astrology. The book by Harrold and Eve (1987) contains several studies of the beliefs of college and university students. The Student Opinion Questionnaire used in these studies can provide a source of items for exploring your own students' or graduating seniors' beliefs and critical thinking and epistemological development. Or, you can replicate the studies on your own campus to investigate your students' cognitive development and your educational effectiveness. How many of your graduating seniors believe in alien invaders? Does it matter?

Although moral development has at least four major components (Rest and Narváez, 1994), skill at reasoning about moral issues to determine what is right action or what is the moral thing to do has been the most intensively researched. As with epistemological development, students need to be approached just beyond where they are in their development.

How can you determine where your students are in their development of moral judgment? Dr. James B. Rest has developed an objective-format test of level of moral judgment development that has been used in over 1,000 studies. The Defining Issues Test can be used to understand our students' current level of development in thinking about moral issues, situations, and problems and thus meet their educational needs more effectively. As with the LEP, giving them their results can stimulate discussion about moral development and behavior and help them become more self-aware and metacognitive.

A Quick and Simple Assessment of Students' Learning in General Education

Research shows that in planning and teaching our courses, our students' prior knowledge is a crucial foundation for what we do. Prerequisite knowledge we assume to be present—if absent—can seriously complicate and undermine learning and teaching. Research also consistently shows that when it comes to our students' learning, there is often less there than meets the eye.

Most of us lack much of the key evidence we need about our students' knowledge and skills. Our institutions often fail to provide us with input data they collect from entering students. Because we tend to slight outcome assessment beyond the level of individual courses, relatively few institutions systematically assess their students' learning in general education or their major fields.

How much have your students learned in school and college? How large is their fund of general knowledge on which you, as a teacher, can build? How well do they understand the examples you use in class? Redesigning Higher Education (p. 54ff.) reviews the results of available studies of the knowledge of college students and graduates. Here's a quickly and easily scored quiz you can use based on some questions from these studies. You can add your own items.

Before you administer the test, write down (a) what the acceptable responses will be and (b) your projection of what you believe acceptable results would be and what you believe they will be for each item. After your students have completed the quiz, summarize the data. If you have used the demographic items included, or added other, similar items of your own, you may be able to do some interesting statistical analyses. Many computers have the necessary statistical programs built into spreadsheet software bundled in "office" suites.

How closely did your projections match the results? What do the results for each of the items mean? What is the significance of the overall results for your students' future lives? For their learning in college? For your teaching? For the effectiveness of your institution's general education program? Are the results consistent with these students' grade point averages? Do any of your assumptions about your "customers" need revision?

Share the results with your students, and engage them in a discussion of the significance of the research for their own development and what you are trying to accomplish with them in class. If, as will be true in many cases, the results are especially interesting, use them as a device for stimulating discussion in your department or college-wide. Be sure colleagues and senior administrators receive copies of your analysis. A small pilot study in your courses may be profitably followed up with a broad-scale assessment and discussion of your effectiveness as an institution. And, if some action is suggested by the results, what, specifically, should it be and who will take it and when?

Quiz

About you

Number of years you have lived in the United States _______

Name of your home state where you legally reside _______________________

Number of years you have lived in that state _______

Your cumulative grade point average up to the beginning of this semester to the nearest tenth point (e.g., 2.7). (Credits earned at this college only.) If you're not reasonably sure, leave this space blank. _______

Number of credit hours you have earned at this college only up to the beginning of this semester _______

Total number of credit hours you have earned in college (all institutions you have ever attended) up to the beginning of this semester _______

About other things

2. Give the number of United States Supreme Court Justices.

3. Name the Chief Justice of the United States Supreme Court.

4. What is the Koran?

5. Give the 50-year period in which the U. S. Civil War occurred, to the nearest whole half century (e.g., 1600-1650).

6. The price of a jar of grape jelly is 12.6 cents per ounce. For a jar of 15 ounces the cost is $1.89. You do not know the price per ounce of a jar of peach preserve. You are asked to pay $1.99 for a jar of 20 ounces. Estimate how much the peach preserve costs per ounce. Write your estimate on the line below.
$________

Here is a list of those key outcomes taken from Redesigning Higher Education (p. 7).

• Conscientiousness, personal responsibility, and dependability
• The ability to act in a principled, ethical fashion
• Skill in oral and written communication
• Interpersonal and team skills
• Skill in critical thinking and in solving complex problems
• Respect for people different from oneself
• The ability to adapt to change
• The ability and desire for life-long learning

Conditions Necessary for Development of the Key Competencies

These are conditions in a college or university which research has shown foster the development of these abilities.

1. High-level intellectual challenge
2. A supportive environment
3. Active involvement
4. High expectations
5. Clearly defined outcomes and frequent assessment with prompt feedback

Contact:
Lion F. Gardiner
Department of Biological Sciences
Rutgers University
Newark, New Jersey 07102-1811
Phone: (201) 648-5450
FAX: (201) 648-5518
E-mail: gardiner@andromeda.rutgers.edu