Tools for Automated Grading? 100
Dont tempt me asks: "As all teachers and students are well aware, it is back to school time. As a math/computer teacher, I am constantly looking for ways to automate repetitive tasks. The one that seems to take up most of my time is grading. As is typical for us nerds, I find myself looking at handwritten tests and thinking 'there's gotta be a better way...' Since I can't find any related open-source projects, I have been thinking about creating one. I have been toying with the idea of using OMR (Optical Mark Recognition) to make my own scannable multiple choice tests. Is anyone doing this? If not, where would be a good place to start? In addition to teachers, this could be a useful technology for questionnaires, or other processes that require manual data entry."
Scantron (Score:4, Interesting)
The machines could NOT have been expensive. Using them was dead simple. We (the section leaders) wrote several tests, and rearranged each test to have different orderings for the choices. Thus, on test version A-1, I had answers (a) Sun, (b) Moon, (c) Earth, then on A-2 I had (a) Moon, (b) Sun, (c) Earth, etc. Then, we looked at their version of the test, and put in the right key.
This kept cheating to a minimum; at the least they had to memorize the answers instead of the answer key. And, memorizing the answers was kind of okay in a sense since they at least paid attention to the subject material.
Not just grading.. (Score:3, Interesting)
You could then automatically create tests with a certain percentage of 'A level' questions and so on. This would also let you more-or-less predict the curve... 10% will get an A, and so on.
Since the grading is automated as well, it would feed back into each question's score automatically
This may sound disturbing, but this is what the SATs do... those small sections at the end are just next years questions being tested
I also had a professor in college who did this, but it was through mental calibration over years. Yes, this does mean you can not give out the tests after for the students to review... but the test was surprisingly fair.
Re:It was a dark and stormy night... (Score:3, Interesting)
Several times if I didn't know the answer I would mark more than one (you don't want to mark all of them, it stands out too much) and I always got credit.
The teacher seemed like a smart guy too, I wonder if he was doing it intentionally to see if people would figure it out.
If you can automate, should you be grading? (Score:5, Interesting)
Assessment should be about the students knowing the material. Stuff like showing your work goes a long way. Math is the easiest to automate, but that would only show you that the student got the correct answer, not where the answer came from (like from a friend!).
To lower you work load, flip a coin on whether the students will hand in the work. If they aren't handing it in, trade with another student and grade it in class. Scantron only sends the message to your students that you are too lazy to look at their work, so why should they put any effort into it.
Scantron Exploit (Score:3, Interesting)
Well, a clever student decided to draw his own marks on the side of the exam. This managed to trick the idiotic machine into thinking all questions were one off & subsequently his test & all tests which were fed through after his had a low grade; the machine tested if question 1 had the answer expected of question 2 and so on.
Multiple Guess (Score:4, Interesting)
I've always had a strong dislike for multiple choice and true/false testing. Taking those tests is often more of an exercise in test taking than it is in the subject matter. A good test taker can eliminate a good portion of the answers right away and use fairly intuitive psychology to improve the odds of guessing correctly.
What ever happened to demonstrating competence in a field? Forget multiple choice and true/false. Ask your students to actually solve applied math problems or actually write some code (or pseudo code). Maybe you can't do as much testing that way and maybe you can't shorten the time it takes to grade the papers but at least you will be testing something worthwhile.
Sorry for the rant but after having survived more than a decade of "education" that consisted primarily of memorize foo and the regurgitate, I'm fairly traumatized by the horror that is the educational system. I learned orders of magnitude more useful information by simply reading everything and anything and trying to apply what I learned to my pet projects. I took one too many tests where I knew several multiple choice answers where justifiable and "right" depending upon unspecified information not contained in the question and having to guess what the test author thought the correct answer was. Multiple choice, true/false, and automated testing are big indicators of a "fast food" mentality and I'm firmly against that sort of foolishness. Grumble, grumble, etc.
Re:Scantron Exploit (Score:4, Interesting)
Back when I was in school, 15 years ago, my teachers were onto such tricks. It isn't hard to look at scores are you write them into your book. The teacher already knows 'Suzie' is smart, often getting a perfect score, so if he[1] misses most of the questions it is time to re-examine things by hand.
The most popular way to cheat was to mark the little box at the top that set this sheet to the master, which would re-program the machine to take your test as the correct answers.
None of these tricks were hard for a teacher to catch (if you knew about them it was easier, and the principal made sure they knew). Once you catch a student doing this you just write zero in for his score and re-run the tests.
[1]we miss you Johnny Cash
Comment removed (Score:3, Interesting)
Re:Not just grading.. (Score:2, Interesting)
But what really matters is the content. My wife teaches the courses in primary education (6 to 12y olds). She has a hard time making up questions for each course in each grade. If there was an openly licensed 'questions base' (maybe even a wiki) this would help her alot. She would be happy to contribute her existing material
You might want to start such an initiative along with the actual open source application.
Ohw... and *please* mind the localisation so there's a potential for every language speaking community in the world, and not only the english speaking community.