A New Class

August is here and I am about to start teaching a new class, the same Math for Life Scientists that I have taught for the last four summers. This year, I want to deliberately implement thoughtful learning. Here are my plans, pristine before their collision with reality.

• I'm going to pause more. It turns out that the "what just happened?" processing time I started using last fall based on intuition is actually a technique called the "pause procedure" with a fairly solid research base dating back to the 1970s. (See, for example, this and this.) In the original studies, students were just asked to compare their notes with a neighbor's, although some instructors assign more complex tasks. Since time is an absolute requirement for thoughtful learning, the pause procedure fits this approach perfectly. I plan to mostly use pauses to have students compare notes or discuss confusing points, as in the original studies, and explicitly ask them to come up with questions.
• I'm going to avoid multiple choice questions and the use of peer discussion as a fallback when no one answers a question. These strategies resulted in the least participatory classes I ever taught and had no discernible effect on student learning. There's (probably) nothing wrong with peer discussion, but only if it's planned from the start. Otherwise, it rewards nonparticipation. If students are truly stumped by one of my questions, I can scaffold.
• I'm going to do even more retrieval practice, especially asking students to write summaries at the end of class. If it's not too much trouble, I might try the three-step "brain, notes, other students" color-coded approach, but that might be too much to fit into an already packed summer schedule.
• Finally, I'm going to try to give students more interesting conceptual questions to think about during class and breaks. This means explicitly asking them to not read the book before it's assigned. That shouldn't be a hard sell.

 We'll see how this goes.

May 2018 Link Roundup

A few items I've come across that merit being pointed out.

• While I'm somewhat biased against multiple choice questions, RetrievalPractice.org has a discussion of how they can actually be good learning tools. The key is that the incorrect alternatives must be plausible enough to make the student retrieve information about each one.
• If you really want to supercharge multiple-choice questions, try this idea from The Effortful Educator. Students comment on why someone might choose an incorrect answer, how the question might be modified to make an incorrect answer correct, and more. If you use clicker questions, this activity might make good homework.
• A promising type of math practice -- same surface, different deep structure problems.
• Finally, Diana Senechal as usual embodies thoughtful learning, this time in a post about praise in American and Hungarian schools.

What kind of praise is appropriate in the classroom? Those of the “growth mindset” persuasion often say that teachers should praise students for effort, not for ability or accomplishment. That strikes me as too rigid; different situations call for different kinds of praise. Sometimes students do need to hear that they have a particular ability or that their work stands out. What matters is that the teacher praise and criticize thoughtfully, not automatically, and that she avoid using praise (or criticism) as a way of exerting control. When students depend too much on teachers’ praise or take it too much to heart, they lose their own critical sense. A teacher’s praise should help students find their way.

I'm on a reading binge about motivation and curiosity, so expect posts about that soon.

Training for Flexible Teaching

For about the last year, I've been working with a speech and voice coach and recently started taking an acting class she teaches at UCLA Extension. Halfway into the course, I am most struck not by any particular technique or exercise but by how many ways there are to do one thing.

Several weeks ago, we started learning short monologues. We then explored each in three different ways ("body NRGs" in the jargon of the method we are using). Sometimes, the teacher asks students performing the monologues to do them in a different way, saying, "What if your director asks for something completely different?" Many of these experiments produce results that are unexpected but make sense. Even the ones that don't often reveal something about a piece that wasn't apparent otherwise. The point is to explore many possibilities before settling on one and be able to respond creatively to whatever happens.

In math and science teaching, at least in higher education, we often seem to look for the One Best Way to teach a topic. But life interferes. Sometimes, you write out careful notes and a student asks an insightful question, which leads to a twenty-minute discussion. Sometimes, your planned ten-minute review becomes the whole lesson because that's what the students turn out to need. Sometimes, you approach a student to help them and find that they are too frustrated or upset to focus. Being a good teacher means being able to respond to these circumstances in the moment, reacting flexibly while still accomplishing what you need to accomplish. This means that teacher preparation at every level, from formal coursework to writing out your notes the night before a class, needs to focus on developing flexibility. The point of preparation is not to know exactly what you will do but to be able to respond to whatever comes up.

Since I'm not teaching this quarter, I'm trying to implement a flexibility-building type of preparation with the undergraduate learning assistants I supervise. Right now, I'm just trying to ask them for multiple possible problems and solutions that might come up as they help students. In the future, I may develop more methods, but training for flexibility looks like a good concept.

Time to Think

What do the following two scenarios have in common?
1. A professor gives a dense, fast-paced lecture with lots of slides. Students scribble down notes, trying to keep up. They need to get all the key information down before class ends.
2. In a flipped classroom, students go from one clicker question to the next. They talk about each question with a partner, and once all the answers are in and the instructor has expanded on them, go on to the next problem. No question takes more than a few minutes to get through.

These scenarios are taken from styles of teaching that are typically held up as polar opposites, yet I would argue that they are more similar than different. In particular, they fail the same way. In neither classroom is deep thought occurring. And it is not occurring for the same reason -- lack of time.

Ben Orlin has a typically charming post on barriers to deep thinking in school. However, I think he missed one. Students do not think deeply in school because there is no time for them to do so.

The primary requirement for thoughtful learning is time because the primary requirement for thought is time -- whether for private contemplation or for a conversation to proceed beyond the obvious. I have been to too many teaching workshops where participants were given a question to discuss in groups and just as the discussion was getting good, just as learning was starting to occur, we were interrupted and had to go on to the next question. Using fewer questions might have worked better.

The humanist and educator Diana Senechal wrote about similar experiences during her teacher training in her book Republic of Noise:

Just as I started to ponder a topic, I had to move into my group and start working and talking. The work seemed superficial and rushed. It seemed, moreover, that the groups reached predictable conclusions about what they read or did. The instructor would move from group to group, listening to each discussion for a few minutes. When, at the end of class, she pulled together the insights of the day, it seemed that many of the finer points had vanished.

In order for our students to have a chance to think, we must slow down. If lecturing, remove some material that students can read on their own and give them time to process. In a math class I teach, I've experimented with pausing after a long derivation and giving the students a minute or two to think through what just happened in whatever way they need, whether doodling on paper, discussing or staring off into space. I plan to try explicitly providing time for students to come up with questions to ask after covering a topic.

Another potential strategy to give students more time to think, mentioned on Susan Cain's Quiet Revolution blog, is to ask them a question at the end of a lesson that will be discussed next time. Ideally, the question should be one that deserves the time and solitude this approach provides. This is quite similar to inquiry-based learning in math and may be one of the reasons I like that approach.

Let's take the time to think of ways to give our students time to think.

Two Ideas for Promoting Transfer

Often, one of the hardest things for students to do is apply a newly learned skill in a new setting, even one that looks almost the same to the teacher. The technical term for this is transfer and promoting transfer is one of the most difficult tasks in education.

The key to transferring knowledge from one situation to another is noticing that, despite superficial differences, the two situations are somehow the same on a deeper level -- in technical terms, they share the same deep structure. For example, an arms race and the ice-albedo feedback loop that enhances warming at the poles are both situations where a change in some quantity (the amount of weapons owned by country A, the amount of ice at the north pole) leads to a further change in the same direction as the initial one. Both are positive feedback loops. Transfer would involve a student who learned about positive feedback loops in the context of arms races applying their understanding to climate change, or vice versa.

Two recent papers have described promising results in promoting transfer. One is an elaboration of methods that many teachers already use, while the other is fairly new (although we sometimes use a very similar one in LS 30 and related courses).

The first paper describes something called concreteness fading. It's exactly what it sounds like -- starting with a concrete example of a topic and then gradually moving toward a fully abstract one. In this particular study, the researchers taught second- and third-graders about equivalence problems of the type 2+5+3 = 2 + __. The teaching was done either through concrete examples (sharing stickers and making balances balance), paper-and-pencil math problems, or a concreteness fading condition that started with stickers and balances, then moved to paper representations of these things, and then moved to actual problems with numbers.

 

 

 

From http://www.learningscientists.org/blog/2018/2/1-1

After the initial learning stage, the kids were presented with problems, including word problems, more complex than anything they had been taught. This was the transfer stage. The kids who were taught entirely using concrete methods performed worst, followed by those taught abstractly. The ones taught using concreteness fading did best.

What happened? Students who only see concrete examples have a hard time generalizing. They may not see the deep structure of what they are doing. (Using a variety of examples may mitigate this but is not always practical.) Abstract learning is general but often difficult. Concreteness fading may bridge the gap between the two, making the abstract learning more effective.

The other study gave undergraduates a classic problem that was analogous to a story they had read. Most people find the analogy difficult to see unless told to look for it. However, their performance improved substantially (from a 10% success rate to a 25% one) if they were asked to come up with a problem analogous to the one they were trying to solve before actually solving it.

This is a very practical result. In some cases, it may be enough to ask students to come up with examples of a new concept, which I already do (there are a number of such problems in Modeling Life) and could do more of. For more complex problems, perhaps including programming problems, asking students to come up with a problem analogous to what they are trying to solve could make sense.  At the very least, it's worth a try.

All Learning is Active

One of the most intellectually engaging classroom experiences of my life took place during my senior year at UCLA, in Rick Vance's Mathematical Ecology class. In a small classroom that was somewhat the worse for wear but had the advantage of ample blackboard space, Prof. Vance derived and analyzed models of ecological processes and the rest of the class and I followed along. A visitor to the class would have observed me doing absolutely nothing, not even taking notes (a physical disability makes me unable to do so). But I was concentrating intently, my brain firing on all cylinders, pushed to its maximum capacity for following a chain of reasoning. Thought, no matter how intense, gives no outward sign.

“…the term “passive learning” is an oxymoron. There is no such thing. If students are learning, then they are NOT passive, and learning does not always include moving or talking…” Yes.  Yes. Yes.  So well said, and sums up in less than fifty words what took me over six hundred words to say in this post on engagement.
 Engagement and learning is an exercise of focused cognition.  Without mentally attending to material/information, there can be no learning.  The idea of passive learning vs. active learning as an outward expression of engagement is very misleading.  A student can look ‘active’ with their learning because they are having a discussion with others or using a manipulative, but without assessment of the student’s cognition, we (students and teachers) should not   assume learning has occurred.  Conversely, a student can appear ‘passive’ in their learning because they are quietly reading; not in a collaborative group or creatively working with material.  In both instances, the student(s) may or may not be learning.

Thought is both the end and the means of education. We learn to think, but we also think to learn. Things we think about are remembered; things we don't think about are not. In the words of cognitive psychologist Daniel Willingham, "Memory is the residue of thought."

This is why we must shift the discussion from active learning to thoughful learning. The methods commonly referred to as "active learning" can be effective but outward activity is only a means of provoking and guiding thought. It cannot be a goal in itself and particularly should not be presented to novice educators that way. (I sometimes want to ask how Stephen Hawking would have fared in an active learning physics class.) Start with what you want students to learn, identify what they should think about, and only then decide how to evoke that thought.

Why I Have a Soft Spot for Inquiry-Based Learning (in Math)

Someday, I am going to get punched for saying, "That's a terribly designed experiment" or "How could they analyze their data this way?" one too many times. My long-suffering colleagues routinely listen to me rant about papers with avoidably confounded variables, uninterpretable multiple regressions (there's a paper in the works) and pseudoreplication in the education literature. If you claim to have data supporting something, I want to look at it, pick it apart and think of five alternative explanations for it. Not surprisingly, I'm a big fan of the Kirscher, Sweller and Clark paper "Why Minimal Guidance During Instruction Does Not Work" (I found this link by typing the title verbatim into Google) and definitely not a fan of discovery learning. However, there is one exception. Sort of.

This exception, as the reader already knows, is inquiry-based learning in math. Specifically, and this is important, it is inquiry-based learning (IBL) in upper division or majors-oriented lower division college math classes. What I am about to say is not meant to apply in any other context.

Inquiry-based learning in college math consists of having students learn math largely or entirely by working through sequences of problems and proofs. The version in which all proofs are done by the students, who are not allowed to use any outside references, is often called the Moore method. Less pure versions also exist and seem to be more commonly used.


The basic framework of an IBL math course has students work on proofs outside of class. Class sessions consist mainly of having students present their work and other students critiquing it as necessary. The instructor provides the problems or theorem statements and a bit of guidance during discussions but otherwise stands back.

An IBL math class embodies thoughtful learning in a way few other teaching methods in any subject do. Thinking is the entire point. Furthermore, since most of the actual work is done outside of class, students have time to think deeply and to grapple with serious problems. There is an alternation between solitary and communal thinking that takes advantage of the strengths of both -- the concentration possible alone and the error-checking and fresh viewpoints provided by others. Indeed, this is how real scientific collaborations often work.

Also, IBL fits its subject in a way that is rarely possible in the sciences. (It does bear some resemblance to seminars and writing workshops in the humanities.) While it may not be (and probably isn't) the most effective way to teach specific mathematical content because of the high cognitive load imposed by figuring out a proof and the very real possibility of proving something without understanding it, if teaching a particular way of thinking is an important goal, IBL succeeds admirably.

There is another, idiosyncratic reason why I have a soft spot for IBL. A few years ago, I was sent to a week-long workshop on the subject. Of that week, no more than 20-30 minutes were devoted to reviewing research, most of which was on active learning in general. The rest was looking at implementation and the details of what actually happens in the classroom. Rather than using bad data to try to show that a particular method of teaching was best, the workshop leaders in effect said, "Here's a way to teach we think is good and here are ways to do it". In keeping with the (apocryphal but frequently misattributed to Mark Twain) principle that "It ain't what we know that gives us trouble, it's what we think we know that just ain't so," no data can be better than bad data because it doesn't cause false confidence in the way bad data does. Perhaps grist for a future post?

Read more at: https://www.brainyquote.com/quotes/mark_twain_109

The First Rule of Thoughtful Learning

The first rule of thoughtful learning as I see it is that, short of abuse, pretty much any pedagical technique is sometimes appropriate. The point is to be aware of costs and benefits so that we can mindfully make tradeoffs and design coherent learning experiences.

When I critique a practice, it is rarely if ever my intent to say that no one should ever use that practice. Sometimes, I may use it myself. I may think the technique is oversold or has downsides that are rarely considered, but there is almost certainly still be a place for it somewhere, in some particular situation. If thoughtful learning means anything, it is shifting the emphasis from action to thought, and this applies to teachers just as much as to students.

Make Learning Objectives Honest

Lately, learning outcomes/goals/objectives have become all the rage in higher ed. (As this post is written in English rather than ed-speak, I will use these terms interchangeably.) These typically have the form, "After this lesson (or unit or class), you should be able to do X".  But does learning work that way?

As a practitioner of Brazilian jiu jitsu, I get twice-weekly immersion in the experience of being a student. During a typical lesson, the instructor explains and demonstrates a new technique and then has us try it out. Very often, the initial result is complete confusion. How do I get from the starting position to the intermediate one? Where do I grab? And which limb should I be using, anyway? (This is not just a result of my disability -- many physiotypical students go through the same thing.)

Even after the initial confusion abates and I can execute the move, I don't really know the technique. I may forget it by the next session. I may be able to do it on a cooperating partner, but what about one who's resisting? How do I set up the technique and when is a good time to use it? What small details make the difference between success and failure? Learning these things takes years. At what point can I be said to know the technique?

Academic learning is just as messy and multi-layered as physical. Coming to understand anything nontrivial takes time. Learning objectives that say, "After this lesson/activity/whatever, you should be able to do X" lie to students about the nature of learning. At most, a single learning experience or even a short sequence of learning experiences can make you slightly better at a skill or deepen your understanding of a concept. They should not promise to do more.

 Education consultant and blogger David Didau writes:

All too often our learning intentions are lesson menus; here is what you should know, or be able to do by the end of today’s lesson. Students are unlikely to do more than merely mimic the understanding or expertise we want them to master.

If instead we were to share our intention for students to struggle with threshold concepts, then we could tell them that it might take them weeks to wrap their heads around such troublesome knowledge. We could remind them that in this lesson they are making progress towards a goal and that there is no expectation for them to ‘get it’ in the next hour or even the next week...

Learning does not follow a neat, linear trajectory, it’s liminal. Students not only need to spend time in that confusing, frustrating in-between space, they need to know how important it is to stay there for as long as need be. If learning intentions rush or limit this experience then they might be doing more harm than good.

If you agree with this critique but still like the idea of explicit learning goals (or are required to write them), what else could you do? One possibility is to use thought-focused rather than action-focused language. I like the following possible goals for lessons or short sequences:

1. Introduce a concept or skill
2. Deepen your understanding of a concept or practice a skill
3. Connect a concept or skill to others
4. Extend a concept or skill to new contexts

If the purpose of writing goals is to structure a course and show students what they will be learning, consider using organizing questions. There can be a few overarching questions for a course -- one class I taught used "How do systems behave?",  "How can we use math to model biological systems?" and "How can we use models to predict behavior?". Each question can have subquestions and sub-subquestions that you can introduce at the start of a lesson. The virtue of this approach is that few things pique curiosity like a question.

There's nothing wrong with explicitly stating the purpose of a lesson. But let's do it in ways that don't mislead student about learning.

What is Thoughtful Learning? Take 1

The short answer: That's what I'm trying to figure out. Thinking out loud on this topic is why I started this blog (which would have been called "Thoughtful Learning" had that not already been the name of an educational publisher) .

The longer answer: The term "thoughtful learning" as I use it deliberately carries a twofold meaning: learning experiences that allow students to think and thoughtfully designed teaching. The two go together, as it takes a good deal of imagination and reflection to create such learning experiences. Thoughtful learning develops from coherent relationships between the teacher, the students, the subject matter and the environment in which learning takes place.

The ingredients of thoughtful learning are human connection, coherence, good questions, time and space to reflect, and a fit between the subject, the circumstances and the method.

I believe that to make thoughtful learning happen, we need a synthesis between (not just a mix of) traditional and active methods of teaching with attention to the context in which learning takes place.

The bullet points: Teaching for thoughtful learning is

• Teaching information in a coherent way, not as a series of disjointed facts to be memorized by rote, while recognizing the importance of knowing key facts.
• Recognizing that the deepest thought often takes place in silence and solitude, while appreciating the contribution of a lively discussion.
• Understanding how much guidance beginners need, while seeing Leonard Cohen's line, "There's a crack in everything; that's how the light gets in" as highly applicable to teaching.
• Valuing activity as a way of promoting thought, not an end in itself. 
• As teachers, aiming for the fluency to choose an appropriate technique for the situation rather than mechanically sticking to one approach.
• ???

What Are Grades For?

At almost all educational institutions, students get grades when they finish a course. Grades are primarily supposed to evaluate how well a student has learned the course material. They should tell other people (future instructors, potential employers, graduate and professional school admissions committees, etc.) roughly how well the student learned what they were supposed to learn in a given course. (This is one of the multiple reasons why grading on a curve is an abomination.)

As hard as it may be to assign a single letter or number to learning, most people would agree that a student who learns the material better than another student should not get a worse grade. Grades should be monotonic with respect to knowledge.

Modified from http://ars.els-cdn.com/content/image/1-s2.0-S0951832013000525-gr1.jpg

So far, the situation is simple. Grades evaluate performance. High grades are desirable because they improve your options later in life, so students typically try to get the highest ones possible, which means learning the material -- or at least the material that will be assessed. As long as instructors create reasonable assessments and students do not resort to cheating, their incentives are aligned and things more or less work out.

However, the desirability of high grades introduces a complication. As instructors, we want students to do things that will benefit their learning. We want them to come to class, do their homework, participate in discussions, and possibly more. Ideally, the desire to learn and to get good grades should be enough to get students to do these things, but sometimes they're not. Sometimes the end of the quarter seems too far away or the student is sure that they know the material and don't need to do all of those chemistry problems. So why not directly include those things as part of the course grade? Students want good grades, so if 10% of the grade is based on completing their homework, they will probably complete their homework. Incentives do work, at least in the short term.

Many, perhaps most, instructors engage in this practice to some degree. Even I do, as I was the freshman chemistry student who found it hard to do ungraded homework. (I do have a second grading scheme that excludes such things.) But there are several problems with it.

First, using grades as incentives to do specific work doesn't distinguish between the student who spends too much time partying and the one who works 20 or more hours a week and supports family members. The latter student is already at a serious disadvantage; making them do a certain amount of work, often at very specific times, no matter how much they actually know is just kicking them while they're down. Furthermore, if one class grades in-class clicker questions or reading quizzes and another one taken by the same students doesn't, students will tend to spend less time on the second one, creating an incentive for its instructor to do the same thing and leading to grading arms races.

The second, less frequently discussed, disadvantage of grading things other than assessments is compromising the monotonicity discussed earlier. The larger the proportion of students' grades based on work, the greater the chances of a student with a better work ethic or organizational skills (or more time) outscoring a student who knows the material better.


This problem could be solved by the kind of multi-category grading my middle school and high school used. For each class, you got both an academic grade and grades in "cooperation" (basically behavior) and work habits. Only the academic grade counted toward your GPA, but the others were still on your report cards and transcript.

The third, deeper problem with using grades to manage the details of student activity is that it very likely undermines the maturation process supposed to occur in college. At some point during development, a person should go from being driven primarily by short-term rewards and punishments to being able to pursue a long-term goal even in the absence of short-term incentives. For many people, this happens in college, where they grow into adulthood by managing the abundance of unstructured time higher education traditionally provides. Often, we learn this lesson the hard way, but we do learn it. However, as colleges start managing the details of what students do and when they do it as much as any high school, this unstructured time is being taken away. If young people don't learn to create their own structure by the time they finish college, when will they learn it? (The heavy reliance of many higher education reformers on external motivators is a subject I will take up in a later post.)

Teaching frequently requires us to balance competing values against each other.  While the mission creep of grades both undermines their original purpose and creates other problems, it can be effective in promoting learning. Every teacher must decide for themselves what to do about this tradeoff, but we can only do so with a clear awareness of what is gained and lost in each decision.

A Modeler Thinks About Assessment

Originally published on Perceiving Wholes.

As a lecturer and DBER fellow, I hear a lot about assessment. Instructors are told that they must align what they teach with the questions that are going to be on their exams, that students shouldn't encounter a question type on an exam that they haven't previously learned how to answer.

I think this is profoundly wrong for one simple reason: assessment is a modeling problem.


Think of your knowledge of some topic -- evolution or cell metabolism or ordinary differential equations --  as a network of related concepts, facts and techniques in your mind. The beginner's network might be missing some important connections and contain extraneous, misleading ones. The expert's network is rich but well-organized.

In assessment that goes beyond simple factual questions like "what are mitochondria?", we are implicitly trying to find out whether a student's network is more like the one on the left or the one on the right. The more expert-like the network, the better the student understands our subject.

Of course, we can't observe this network directly. To a teacher, a student's mind is a black box. Therefore, we poke and prod the black box by asking the student questions and use the answers to build our own models of the student's knowledge network. Particularly valuable are those questions whose answers are easy to figure out if subject matter knowledge is complete and well-organized but difficult or impossible otherwise. If a student answers these types of questions correctly, they probably understand the subject well.

Unless, of course, the student has explicitly learned how to answer the question that you asked without figuring it out. Then the process is short-circuited and we are left without a way of assessing what a student actually understands. Ben Orlin writes about this in Math with Bad Drawings:

Need to prove these triangles are congruent? Do this. Need to prove that they’re similar? Do that. Need to prove X? Do Y and Z. I laid it all out for them, as clean and foolproof as a recipe book. With practice, they slowly learned to answer every sort of standard question that the textbook had to offer.

Months passed this way. But something wasn’t clicking. I kept seeing flashes and glimpses of severe misunderstandings—in their nonsensical phrasings, in their explanations (or lack thereof), in their bizarre one-time mistakes. Despite my best intentions, something was definitely wrong. But I didn’t know what.

And, more worryingly, I didn’t know how to find out.

I’d already coached them how to answer every question in the book. How, then, could I diagnose what was missing? How could I check for understanding? For every challenge I might give them, every task that might demand actual thinking, I’d equipped them with a shortcut, a mnemonic, a workaround. The questions were like bombs defused: instead of blasting my students’ thoughts open, they now fizzled harmlessly.

Orlin is describing his mistakes as a novice teacher. But this is the inevitable consequence of the "alignment" being pushed by proponents of scientific teaching. They would probably say that the student should initially figure out the procedure instead of being taught it, and this might indeed be better (or not), but it remains true that when the exam rolls around, all we will see is how well the students remember what they were taught. We will have lost our tools for modeling their minds and assessing their understanding.

Silence is an Answer

Realization: when students don't answer a question in class, they are in fact saying something. They're saying that they don't understand the material adequately or don't feel confident in their understanding (or your question was vague). If that's the case, a constructive response would be scaffolding with simpler questions, providing more explanation, or asking for a question rather than an answer. Cold-calling just papers over the cracks.

Your thoughts?