JITT in IP&T

This is week two of reading the JITT (Just in Time Teaching) quiz results from my students.

Wow! I really like this approach to class preparation and lesson planning. After reading the JITT quizzes responses, I feel as though I have already held a mini-class with my students and I have an idea of which points from the reading are confusing for them, which are interesting to them, and which are clearly understood by them.

How does it work?

1. Kimberly (my wife and fellow instructor in this type of course) writes some quiz questions that are designed to (a) highlight the vocabulary that the program director feels are most important and (b) get students engaging with the most valuable ideas from each chapter. Three of the items are fixed-response (multiple choice or fill-in-the-blank, etc.) and one is usually an open-ended application question. We have a mis for obvious reasons: multiple choice are fast for students to do and cover a wide range of material quickly, but the open-ended questions lets us see how students problem solve and whether they understand the material at a deeper level.

2. Kimberly send me the questions and I review them, providing editing, content, and test-quality feedback. We make some revisions through discussion.

3. We post our questions online using the university's course management system (CMS) which has a quiz feature (a little cumbersome, but it works).

4. Our students are required to read the material and then answer the questions by a deadline (I have them complete the quizzes by midnight of the day before the day we meet in class), so that as instructors we then have time to analyze their responses and develop our lesson plan.

5. We download the quiz results. At first, we did this manually: we openned up every single response from every single student, and recorded the information from a web browser into an speadsheet file. This was horribly, painsakingly long (especially given the cumbersome nature of the CMS package). Of course after doing this the first week, I noticed that the CMS has a "Download Quiz Results" link. Needless to say, we tried this for week two and it was a huge timesaver.

6. We analyze the results. This is not a complicated process. Really, we are interested in seeing which questions (and hence which concepts) the students have a clear understanding of, and which concepts are problematic. Generally, I don't write any notes for questions in which all students got correct (or only one student got wrong - instead I would suggest talking to or emailing that student directly). However, in questions where there appears to be various different responses, I take notes on the most common misconceptions.

In the case of open-ended questions, students tend to respond with the most common, salient features from the text (most often the ideas that are highlighted in the end-of-chapter summary page) but few demonstrate understanding of the fullness of the concept as expressed in the text. I make notes of these deficiencies and I highlight responses from students whose answers indicate that they have grasped these finer points.

There is one final question that we analyze: the optional, open-ended feedback question. At the end of each quiz we include a feedback field for students to tell us how the reading is, any challenges that faced, and any other thoughts that they have about the reading, the quiz, or the class. At least half of the students skip this optional question, but the feedback from those who do respond is invaluable. Frequently students will explain how they had trouble with a multiple choice item ("I felt like there were two correct choices because...") or they will express concerns about a particular topic ("I'm still kind of fuzzy on the idea of..."). And about half of the responses are just postive feedback ("I really enjoyed this chapter since I learned something new that I can apply to..." or "I found this chapter much easier to understand than the last since I had already studied a book about..." or "Question 3 was a really good measure of the chapter since it pulled together all of the ideas into one practical situation.").

These notes (list of what students do not understand as well as their additional concerns and questions) serve as the guide for my class presentation.

7. We write a lesson plan. Although our class meets for a 3-hour block each week, we generally only try to take about 1 hour for our JITT presentation since we reserve most of our class period to student presentations (especially important since we teach pre-service teachers who need the practice of planning and presenting mini-lessons). So we design a presentation that covers the most difficult chapter concepts based on the JITT quizzes.

It's only week two, so I am hardly an expert at this. And I rely heavily on Kimberly when it comes to desiging a JITT presentation (aka lecture) since she has done an extensive review of the literature for one of her research projects. I really appreciate being able to work together on this experiment and I hope that Kimberly and I will be able to contribute to the JITT field by writing a paper that describes how to move from JITT quizzes to JITT lectures - a part of the JITT process that seems to lack publications.