clifflandis.net

Library Instruction Assessment is one of the most annoying parts of doing/coordinating library instruction.  Tallying paper surveys and grading quizzes is a pain in the butt.  So here at MPOW, we were looking for a way to assess our instruction skills without wasting a bunch of time.  We now have in place a pre- and post-test that can be tallied in about three minutes.  Here how you can do it at your library:

1. Figure out what you want to measure. We polled the reference librarians to see what they thought were the most important skills for students to walk out of the session with.  Those are the things we test.  Thanks to Emily for doing this!
2. Create a form for the pre- and post-test. Ours is here.
   1. We ask them for their ID numbers so that we can match pre- and post-test scores.  Since the librarians do not have access to the students’ ID numbers–and since the professors never see our assessment results– there is no fear of retribution for “bad scores.”  After all, bad scores just mean we’re doing a bad job of imparting the information!
   2. The form (invisibly) timestamps each submission, so you can tell the pre-tests from the post-tests by the time that each entry was submitted.
3. Have the results dump into a web-based text file. We use the ProcessForm script by MindPalette, which is free.  A big thank-you to Andy in Web Services, and Sherrida and Keith in Automation for making this happen.
4. Create a spreadsheet for analyzing the data. Ours is here.
   1. The spreadsheet’s calculations are based on the number of students taking the pre- and post-tests.  So if you had 17 students take the pre-test and 19 students take the post-test, you can plug those numbers in to see the percentage correct for the number of students taking each test.  This way, if you have stragglers, you’ll still get appropriate percentages.
5. Plug the data from the web-based text file into the spreadsheet, and voila!

My class of 19 "Intro to Communications" students.

Lessons Learned:

• We’re definitely getting more assessment done in less time, and with less fuss.  So that part is a complete success.
• Students will not remember your name (or even their professor’s name, or what class it is).  Write it on the board.
• We’ll be adding more options for unique (non-identifiable) numbers, since many students do not know their VSU ID numbers.
• The results get skewed if you have more people take the post-test than the pre-test.  Students who take the pre-test will naturally do better on the post-test because they will know what questions will be asked.  Since stragglers will do more poorly in the post-test, you can actually see a decrease in correct answers.  So if you have stragglers, they need not take the post-test (esp. considering they did not get the same instruction experience as those who are on-time and took the pre-test).
• Be aware of the language that you use in the instruction session, and how that will affect the results.  For example, the truncation question asks if truncating a term will search for words with the same 1) beginning, 2) middle or 3) end.  If you say in the session that truncation will search for words with the same root, you will get poorer results.

Please feel free to use this method, or tell me your own way of measuring your library instruction sessions!