Tools for Automated Grading? 100
Dont tempt me asks: "As all teachers and students are well aware, it is back to school time. As a math/computer teacher, I am constantly looking for ways to automate repetitive tasks. The one that seems to take up most of my time is grading. As is typical for us nerds, I find myself looking at handwritten tests and thinking 'there's gotta be a better way...' Since I can't find any related open-source projects, I have been thinking about creating one. I have been toying with the idea of using OMR (Optical Mark Recognition) to make my own scannable multiple choice tests. Is anyone doing this? If not, where would be a good place to start? In addition to teachers, this could be a useful technology for questionnaires, or other processes that require manual data entry."
Scantron (Score:4, Interesting)
The machines could NOT have been expensive. Using them was dead simple. We (the section leaders) wrote several tests, and rearranged each test to have different orderings for the choices. Thus, on test version A-1, I had answers (a) Sun, (b) Moon, (c) Earth, then on A-2 I had (a) Moon, (b) Sun, (c) Earth, etc. Then, we looked at their version of the test, and put in the right key.
This kept cheating to a minimum; at the least they had to memorize the answers instead of the answer key. And, memorizing the answers was kind of okay in a sense since they at least paid attention to the subject material.
Re:Scantron (Score:1)
Re:Scantron (Score:1)
x=4/2
x=4
Yes, I got the answer wrong, but up till that point everything was correct, I got 1/2 marks
Or better, a VK (Score:2)
1.
You're in a desert, walking along in the sand when all of a sudden you look down and see a tortoise.
It's crawling toward you.
You reach down and flip the tortoise over on its back.
The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over.
But it can't.
Not without your help.
But you're not helping.
Why is that?
2.
Describe in single words, only the good things that come into your mind - about your mother.
Re:Scantron (Score:2)
If a person has correct answer but got it only by mistake then what good is it?
Well, it is still the right answer. Sometimes it is better to be lucky than good. I guess the answer to your question gets into some muddy philosophical area. Do you want to be on the plane with the expert pilot or the luckiest man on earth?
Enough with that mumbo jumbo though... I agree that multiple choice is not a very good method for determining how well a given student is doing at math or in CS.
Re:Scantron (Score:1)
Re:Scantron (Score:4, Funny)
Re:Scantron (Score:1)
Re:Scantron (Score:2)
The final exam that a course of teaching culminates in is usually the benchmark of achievement of the student, but in what way is this useful to a teacher given it does not help someone teach. No doubt tests are imp
Re: (Score:3, Interesting)
Scantron Exploit (Score:3, Interesting)
Well, a clever student decided to draw his own marks on the side of the exam. This managed to trick the idiotic m
Re:Scantron Exploit (Score:4, Interesting)
Back when I was in school, 15 years ago, my teachers were onto such tricks. It isn't hard to look at scores are you write them into your book. The teacher already knows 'Suzie' is smart, often getting a perfect score, so if he[1] misses most of the questions it is time to re-examine things by hand.
The most popular way to cheat was to mark the little box at the top that set this sheet to the master, which would re-program the machine to take your test as the correct answers.
None of these tricks were hard for a teacher to catch (if you knew about them it was easier, and the principal made sure they knew). Once you catch a student doing this you just write zero in for his score and re-run the tests.
[1]we miss you Johnny Cash
Re:Scantron (Score:1)
The Scantron Test Scoring Machine! [scantron.com]
Still VERY widely used.
Re:Scantron (Score:2)
What sort of teacher/student in the past 20 years hasn't used a scantron at some point or another?
the machines are dirt-cheap, and accurate enough for all intents and purposes.
of course, it does make the course a good deal less personal.
How about this? (Score:2, Informative)
It was a dark and stormy night... (Score:1)
There's a legend about the time when OMR was a new technology: a student had marked every single checkbox. The software gave him 100% points, because it only checked whether the correct answers were marked. :-)
Re:It was a dark and stormy night... (Score:2)
Not that I tried it, of course.
Re:It was a dark and stormy night... (Score:3, Interesting)
Several times if I didn't know the answer I would mark more than one (you don't want to mark all of them, it stands out too much) and I always got credit.
The teacher seemed like a smart guy too, I wonder if he was doing it intentionally to see if people would figure it out.
Re:It was a dark and stormy night... (Score:2)
Re:It was a dark and stormy night... (Score:2)
Hole Punch (Score:2)
Of course, you have to also look for students entering multiple answers (especially if they know how you're grading).
Re:Hole Punch (Score:2)
Re:Hole Punch (Score:1)
Re:Hole Punch (Score:1)
Don't bother with the OCR. (Score:1)
Ask Slashdot: Automated Homework (Score:4, Funny)
So... (Score:2)
> OMR (Optical Mark Recognition) to make my
> own scannable multiple choice tests.
Ooh! Ooh! Does this mean I can use a calculator when I take your test?
No? Hey, no fair!
Not just grading.. (Score:3, Interesting)
You could then automatically create tests with a certain percentage of 'A level' questions and so on. This would also let you more-or-less predict the curve... 10% will get an A, and so on.
Since the grading is automated as well, it would feed back into each question's score automatically
This may sound disturbing, but this is what the SATs do... those small sections at the end are just next years questions being tested
I also had a professor in college who did this, but it was through mental calibration over years. Yes, this does mean you can not give out the tests after for the students to review... but the test was surprisingly fair.
Re:Not just grading.. (Score:2)
I had a professor in college who did this, and he regularly gave out past exams for students to study from. He said before each test that even though he often reuses test questions, the result had always been a standard grade curve.
Re:Not just grading.. (Score:2, Interesting)
But what really matters is the content. My wife teaches the courses in primary education (6 to 12y olds). She has a hard time making up questions for each course in each grade. If there was an openly licensed 'questions base' (maybe even a wiki) this would help her alot. She would be happy to contribute her existing material
You might want to start such an initiative along with the actual open source application.
Ohw... and *please* mind the localisation so there's a
The problem with multiple-guess tests (Score:2)
My suggestion- get togehter with a programmer, and save class time by giving your tests on the web. You can code the tests in XML- the ASP or Java or python or PHP or whatever can randomize them for you so that no two students get the test answers in the same order. Students can log in from the computer lab to take the tests after school, or f
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Still better to randomize them- though that would do a pretty good job. Of course, the grand majority of answers would still be EATONRIDSH based if they were in English- making the most likely answer probably the first one instead of the 2nd.
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
Re:The problem with multiple-guess tests (Score:2)
And this will be an open book test? Or 3 people taking the test at once?
Re:The problem with multiple-guess tests (Score:2)
If so, they'll have to take it 3 times, and with the randomization factor thrown in, they aren't likely to get the same questions twice. Three times as much work.
And this will be an open book test?
All modern testing should be open book and timed. After all, in reality, that's the new mode of skills people need to survive- the ability to research solutions and discern among several potential solutions, while applying event-specific variables correctly.
Re:The problem with multiple-guess tests (Score:2)
With the one smarter kid supplying the answer to the other two all 3 times.
All modern testing should be open book and timed. After all, in reality, that's the new mode of skills people need to survive- the ability to research solutions and discern among several potential solutions, while applying event-specific variables correctly.
No. Some stuf
Re:The problem with multiple-guess tests (Score:2)
Thus teaching the other two the most invaluable lesson anybody can possibly learn- to listen to experts.
No. Some stuff you just need to 'know'.
And how much of that have you used in your career?
Especially at the high school level. If you merely look up the answer each time, you never internalize it. It never becomes a part of you.
But what you do internalize is where it exists- which is far more valueable than the fact itself
Re:The problem with multiple-guess tests (Score:2)
Or.."You're too dumb to ever actually know anything. Just ask the other guy."
And how much of that have you used in your career?
'Career' isn't everything.
Imagine sitting around with some friends, discussing events in the Middle East, or WWII. Without having a picture in your mind of where countries sit in relation to others...you miss a lot of the implications of why, who, and where.
- You never rea
Re:The problem with multiple-guess tests (Score:2)
That remark resembles more than 55% of the general population- and it's the reason computers and databases were invented to begin with.
'Career' isn't everything.
True, but it's one of the two main reasons society is sending you to school in the first place. Sitting around with your friends answering trivia questions isn't among those reasons.
Imagine sitting around with some friends, discussing events in the Middle East, or
Multitask (Score:2)
But to make it less of a time sink, just multitask; grade during your commute or tv or laundry or workout or whatever.
Stairs method (Score:3, Funny)
- Take all the submitted assignments and collect them in a big pile.
- Throw the pile down a flight of stairs.
- Everything that makes it to the bottom gets an A. For each step above the bottom take off 5% of the grade.
- That's it.
I've thought (heavily) about being a teacher.. (Score:1)
Re:I've thought (heavily) about being a teacher.. (Score:1)
If you can automate, should you be grading? (Score:5, Interesting)
Assessment should be about the students knowing the material. Stuff like showing your work goes a long way. Math is the easiest to automate, but that would only show you that the student got the correct answer, not where the answer came from (like from a friend!).
To lower you work load, flip a coin on whether the students will hand in the work. If they aren't handing it in, trade with another student and grade it in class. Scantron only sends the message to your students that you are too lazy to look at their work, so why should they put any effort into it.
Re:If you can automate, should you be grading? (Score:3, Insightful)
Re:If you can automate, should you be grading? (Score:1)
Gotta justify that teaching credential somehow.
Re:If you can automate, should you be grading? (Score:2)
Re:If you can automate, should you be grading? (Score:1)
I had a nother teacher that would twice a month give a quiz that was just page and problem numbers, if we did the homework we could anwser them, but it didn't make sense to do homework and then not go over it.
Re:If you can automate, should you be grading? (Score:1)
Why not remove paper altogether? (Score:2)
You could navigate back and forward, review your answers, and confirm completion. When you were done, the grade was instantly recorded and displayed to the student.
That was over 10 years ago! I see no reason why this shouldn't be possible (and inexpensive) in
Re:Why not remove paper altogether? (Score:2)
Well, it's true -- technology can't replace teachers. But it sure can give them more time to actually teach.
Re:Why not remove paper altogether? (Score:2)
Re:Why not remove paper altogether? (Score:2)
Re:Why not remove paper altogether? (Score:2)
seriously a custom designed system built to be rugged and safe for a classroom is much better than throwing a pile of money at the problem and hoping it goes away.
Re:Why not remove paper altogether? (Score:2)
I'm not sure what your point is with that "throwing money" remark, unless it's that we can't solve problems by buying technology without knowing what we're going to do with it. That's certainly
Re:Why not remove paper altogether? (Score:1)
Student Grading (Score:2)
If privacy isn't a concern, you can chop off the corner with the students' name & write a number on top & allow them to grade someone else's exam.
Multiple Guess (Score:4, Interesting)
I've always had a strong dislike for multiple choice and true/false testing. Taking those tests is often more of an exercise in test taking than it is in the subject matter. A good test taker can eliminate a good portion of the answers right away and use fairly intuitive psychology to improve the odds of guessing correctly.
What ever happened to demonstrating competence in a field? Forget multiple choice and true/false. Ask your students to actually solve applied math problems or actually write some code (or pseudo code). Maybe you can't do as much testing that way and maybe you can't shorten the time it takes to grade the papers but at least you will be testing something worthwhile.
Sorry for the rant but after having survived more than a decade of "education" that consisted primarily of memorize foo and the regurgitate, I'm fairly traumatized by the horror that is the educational system. I learned orders of magnitude more useful information by simply reading everything and anything and trying to apply what I learned to my pet projects. I took one too many tests where I knew several multiple choice answers where justifiable and "right" depending upon unspecified information not contained in the question and having to guess what the test author thought the correct answer was. Multiple choice, true/false, and automated testing are big indicators of a "fast food" mentality and I'm firmly against that sort of foolishness. Grumble, grumble, etc.
Re:Multiple Guess (Score:4, Insightful)
There are times for memorization, and there are times when you need to go farther. In math you always need to go farther and understand the concepts. In shop you must get 100% (no misses allowed!) on the safety memorization test before you are allowed to take the test on real tools. Of course knowing that the margin of safety on some saw is 10 inches doesn't mean you won't put your fingers closer to the blade, but if you don't know that number it means you will.
Memorization is important. Do not overlook the value of memorizing some things (even if you will never need to know that poem once you pass the class, it is still useful to do it). Though overall I agree that there is too much focus on memorizing (mostly on the wrong thing!) in school, that doesn't mean you can get rid of memorizing.
Re:Multiple Guess (Score:2)
My first thought when I started reading everyones comments was that they seemed to be missing the point that the first several years of mathematics are almost entirely memorization. Addition and subtraction are conceptual when showing children a bunch of apples and then taking two away. After that, it's memorization. Multiplic
Factual knowledge is a commodity (Score:2)
Anything worth knowing will be "naturally" memorized through use. Anything not worth knowing will be forgotten. Anything worth knowing that has accidently been forgotten can be looked up.
It's a beautiful system, and it's all natural!
Learn to read (Score:2)
You need to learn to read. Or maybe reading comprehension (Though I'll admit that it would help if I was a better writer). You can look some stuff up in a book, but sometimes you should memorise that stuff before you need it.
When your clothes are burning off your back there is not time to find a book to look up "Stop Drop and Roll", so you have to memorize it in advance - in fact most people would never need to know what to do in that situation, but everyone should memorize what to do about it!
There is
You're too insightful... (Score:2)
What ever happened to demonstrating competence in a field? Forget multiple choice and true/false. Ask your students to actually solve applied math problems or actually write some code (or pseudo code). Maybe you can't do as much testing that way and maybe you can't shorten the time it takes to grade the papers but at least you will be testing something worthwhile.
Sorry for the rant but after having survived more than a decade of "education" that consisted primarily of memorize foo and the regurgitate, I'
Re:Multiple Guess (Score:2)
It's a lot harder to grade, I realize, but the best tests in my opinion are the ones that have four essay questions, all of which require a great deal of thought on the part of the student.
Memorizing facts is nothi
Re:Multiple Guess (Score:3, Insightful)
Each of your 6 classes has 30-35 students.
Every time you give an assignment to you students, you get 180-
suggestions (Score:3, Informative)
For multiple choice tests you could use off the shelf survey software like phpsurveyor or phpesp. Keep in mind these wouldn't necessarily be great at grading but it would let you easily analyze the test results question by question.
If you are grading programming assignments, you could develop your own testing suites using the *unit family of testing suites: nunit (.net), junit (java), phpunit (php), and I'm sure there are others. I think there's even some tools designed to evaluate test coverage like jcoverage (never used). Maybe you could have advanced students write test suites for the novice student assignments and evaluate/fix them with jcoverage... then use the test suites to automate testing of novice students.
Of course, there are only so many things that are easy to test in an automated fashion. You may have to give students exact specifications on interfaces and that may not always be desirable.
Re:suggestions (ok, automate THIS) (Score:2)
It sounds like correctness as potentially evaluated by a testing suite going off of a specification would be at least as important as code cleanliness/efficiency/style. Using regression testing suites would not solve all problems, but it seems like they would solve some. If you could work in regression testing into the curriculum somehow, there would be a value add for using the technology.
Perhaps the true
Use an algorithm as the solution... (Score:1)
Create three variations of the test with each question yielding a slightly different mathematical answer. You could have it as simple as adding all the answer so they match a certain sum or have an algebra equation at the end that they plug the values into.
All you have to do is check the variation number of the test and the result of the algorithm.
To shake it up and prevent cheating you can use variations on the test (either variables or questions), you can use variations
Re:Use an algorithm as the solution... (Score:1)
A hybrid approach (Score:1)
Strange questions for Slashdot (Score:2)
Not sure if they have better or faster scantron machines today - but I would bet your school has something like it around somewhere.
Ask Slashdot - reinventing the wheel since the late 20th Century
Re:Strange questions for Slashdot (Score:2)
They're still the same. The latest scanatron is still reported to have trouble distinguishing the correct answer if there is even a trace amount of pencil on another cell.
If you make a mistake, you either need to erase heavily (potentially damaging the paper) or get another sheet.
Do less grading!! (Score:5, Insightful)
been there, done that (Score:2)
Maybe I've written some software that can help you. I'm a computer science teacher at the high school level, and I take my grading very seriously. But, I hate "busy work" and am equally serious about using technology to streamline my grading workflow.
Now, I assume you only have one PC in your classroom, whereas every student in my room has their own machine in front of them. And that's a huge advantage; if the work is in digital form to begin with, it grades much more quickly.
Anyway, I've done some t
Pimping LMS... (Score:2)
There are obvious limitations to online test taking, but it *does* provide automated grading, and with the right institutional commitment it can have a positive impact on student learning.
Just say no (Score:1)
I'd simply suggest keeping away from automated grading, as it is a very bad way of grading.
I have taught math and CS for quite a few years, and I have never ever given an exercise in an examination for which the point was the final result. Most of the time, I do not even look at the final result. Of course, it is a lot more work to actually read what students write, how they get to the results they get, how they argue if they argue, &c, but that is what I am teaching them to do, so that is what I am gr
Try Moodle and such. (Score:3, Informative)
All in all, it will allow you to make quizzes and lessons online that students can access. Questions can be auto sorted and even short answer questions with different possible answers. Its a beautiful system with the only flaw of facilitating a computer for each student to use. (I'm in an independant school so our kids have laptops at the ready, something we don't all have.)
The only other geek-oriented possiblity would be using scantrons or small LCD based devices, but from what I've seen nothing fits the bill. Possibly the best action might be changing how you grade and what work your students do (ie projects instead of tests and the similar). It works with a little imagination and there's alot less grading!
Make them grade each other... (Score:2)
Works especially well if you don't happen to have a lesson plan that day.
This happened all the time in my high school. And de
Re:Make them grade each other... (Score:3, Informative)
Besides concerns about cheating, that practice is likely a violation of the student's right to privacy. The Family Educational Rights and Privacy Act [ed.gov] prohibits the release of any information from a student's education record. Because test scores make up a large part of any final grade, sharing these results with other students is probably a no-no.
This happened all the time in my high school.
Yeah, high school is a good place to stomp all over kid's rights. Somebody has to put them in their place...
Re:Make them grade each other... (Score:2)
Re:Make them grade each other... (Score:2)
However, I don't think the GP is correct - nobody objects to students working in groups, etc., and I fail to see how peer-graded homework that isn't handed in constitutes part of a student's "record." Nobody would do that for a major test or final exam.
Re:Make them grade each other... (Score:2)
Did you read the original question or comments?
The article specifically asks about grading tests, with no mention homework:
I find myself looking at handwritten tests and thinking 'there's gotta be a better way...'
Then tktk writes:
Collect the tests
...
And I respond:
Because test scores make up a large part of any final grade, sharing these results with other students is probably a no-no.
Then you write:
I fail to see how peer-graded homework that isn't handed in constitutes part of a student's "
Re:Make them grade each other... (Score:2)
We used to do this as well, and it was a very educational experience. I can imagine it now... "Jimmy, what is the answer to question six?" "I'm sorry, the Family Education Rights and Privacy Act prohibits the release of any information from my education record. Since my knowledge of the answer constitutes
What a dick. (Score:1)
Re:What a dick. (Score:1)
Exactly. So you and I both agree. Teachers should use their brains, rather than having some crappy piece of code automating the process.
And don't whine to be about the hard life of a teacher and the low salary. It's not like it's a fucking new situation. If you've gone into educating as a profession at any point in the last 30 years, you knew what a shit job it was and didn't have a problem taking it - supposedly because you enjoy the actual job itsel
The traditional way (Score:1)
Re:The traditional way (Score:1)
No shortage of Open-Source solutions (Score:1)
Moodle and LON-CAPA are more g