Are Written Computer Science Exams a Fair Measure? 728
me! asks: "I seem to have this inability to write substantial chunks of code (500+)
in exam conditions (for uni). I have been
writing code for years for open source and commercial applications, so I know a thing or two. There is just something about exams and code that does not work for me. I find that I need to be sitting in front of a computer to
get a problem out, to get in the 'vibe', have you will. I have done exams on computers (closed environment) that involve coding, and it work so much better for me. So what I am asking is...how do people tackle exams that involve solving problems on the fly, on paper, in exams?" I have this exact same problem, and I've never thought written tests were a fair way to measure someone's knowledge of coding. It's fine when you are asking questions about design and structure, but when you need to write code it falls way short. How do you feel about it?
Error checking by compiler (Score:2, Insightful)
I feel the same way (Score:3, Insightful)
Granted, you can never design a perfect test. But you can at least do away with the totally obvious imperfections like asking people to do things in ways they never do and never will do in the future (except on other tests).
The few times I was subjected to these tests, I sure felt primitive erasing entire chunks of code after having realized I needed to insert something in the beginning. And it's a hard thing to avoid; I don't think any reasonably experienced programmer writes code entirely from top to bottom. Just a simple thing as placing braces becomes a hurdle in writing unless you know the length of the code you need to put between them!
I don't mean to sound mean.. (Score:2, Insightful)
This is A Big Problem... (Score:3, Insightful)
I am much better at solving a programming problem in front of a computer.
Also, they are nothing like real world conditions... Do you know any programmers without reference books on their desks? That have the whole Java API memorized?
No need for large portions (Score:2, Insightful)
Testing the ability to write large chunks of code is not reasonable on a written exam, and then, usually your grade is determined by your handwriting speed and neatness
Writing code (not typing it) is *hard* (Score:5, Insightful)
This is why it's an outstanding way to test people's knowledge. Anyone can make a program work given a compiler and sufficient time. If you can do it with just pen and paper, and remember the syntax without having other code in front of you, then you know your shit.
This is why it's a great test for interviews. You'd be amazed how many "senior C developers" we interviewed at my last company who couldn't write push() and pop() for a linked list.
Learn this skill (Score:5, Insightful)
It seems that you've associated syntax and semantics with the visual cues of a computer. That's not the same part of the brain that answers exams. So you need to retrain yourself. Here are some things you can do to relearn programming:
A) Stop using IDEs. Use the plainest text editor that you can find to write your code. Turn off any syntax highlighting and code completion.
B) Design your code before you write it. Use UML, flowcharts or whatever, but design it first. Then when it comes time to code you will know where you are going.
C) Write code away from a computer. Use a pencil and paper. I didn't have a computer when I was in college (very few people did). So I coded by hand on paper, then went to the lab to type it in.
Re:simple (Score:3, Insightful)
It was the total mind-opening academic scene that surrounded me 24/7 for 5 years, the chance to really go out and party my brains out to relieve the stress, the hour long conversations with professors who are true geinuses in every sence of the word.
You might get paid a lot of money, but if you see college as simply a means to that end, I strongly suggest you take a year off and attend full time. Because you certainly do not get the point.
And, through so many experiences, I learned that your final career and salary doesn't define you, your worth, or how much you can get out of life.
but congrats on your job, though. Pretty sweet deal.
Good programmers use tools, not memorization (Score:5, Insightful)
But when I am asked questions about the same thing away from a computer and my books I start drawing blanks even about things I know in detail. This is why, IMHO, computer programming exams and even job interviews should be 'Open Book'. Memorizing details doesn't make you a good programmer -- knowing how to use the tools does.
Jack William Bell
Re:I also (Score:4, Insightful)
Just because you have a calculator is no excuse not to learn how to add.
Other disciplines, people just live with it (Score:3, Insightful)
In the field of computer science, specifically, you can make the claim that only the ability to express yourself in type - personally I can write by hand anything I could type, but that's me - matters professionally, so that's what you should test on. Certainly, those students who "choke up" when presented with a computer terminal shouldn't be in computer science courses, so you don't have the same problem you'd have giving exclusively oral tests in the humanities.
But I don't think it's true that you can totally neglect the ability to scribble code; there are situations where (for social reasons) I've had to do development work by hand-writing on printouts. If you're teaching someone else, it is often easier to scribble on a slip of paper than to talk and gesture at a computer screen.
In CS, it would be reasonable to have all the tests on computers. However, there is a certain justification for making sure people can write code by hand, and it isn't so difficult for most people to learn. I don't see it as a fairness question - the skill of writing code by hand is not terribly important, and your department may place disproportionate emphasis on it. That's unadvisable, but it's perfectly fair; it's the same skill for everyone, but some people can't do it. Too bad for them, but they've not suffered any injustice.
Now, personally, I think it's daft to have students write significant code during exams. If you're teaching programing, have the kids write programs and submit them. A little pseudocode on an exam ought to demonstrate the abstract understanding for which the written exam should be testing.
Re:simple (Score:1, Insightful)
sociology, and all those core classes totally sucked but those were the ONLY classes that i took that ever had hot chicks...
Re: code on exams (Score:3, Insightful)
It's Friday at 5:00 and you're leaving on a weekend trip at 7:00. The software is handed off to SQA on monday. There's one more bug in the queue and it belongs to you. Question: A) Do you bail on your trip? B) Do you sneak out the back door? C) Do you go fix that bug in 57 minutes?
Re:I feel the same way (Score:2, Insightful)
You have to
1) Remember definitions.
2) String statements together in logical, consistent groups.
3) Reach the proper conclusion with the hypotheses given.
4) Justify interesting or tricky steps.
Sounds a lot like creating code in an exam situation. A reasonable professor (or TA more likely) will read the code/proof looking for the the correct ideas and flow more than perfect pedantic correctness. At least that's what I did when I graded for upper division classes.
rbb
ps - use scratch paper to avoid erasing.
Re:Hate it! (Score:3, Insightful)
The only language I can write 100% on paper is COBOL, and I hate COBOL.
With C I need a compiler to catch the few typos or what ever I made. But I don't need all day, one or two compiles does it.
Luckly at my school the COBOL tests were on paper, and the C test was on a computer (and timed).
I liked my OS class. The UNIX tests were held in front of terminals. I couldn't remember the quite how to use the non-GNU `sort` command. So `man sort` saved me there. I told the teacher what I did after the test, he said, "excellent."
Re:Hate it! (Score:2, Insightful)
But written code tests something very important, which is whether you can design a solution and describe the solution in essentially correct code. That's something very different from just slowly crufting a solution together because you can build it up. I've seen so many programmers commercially who can't design, describe, or communicate their approach to solving a problem, and I think quality suffers directly.
Obviously there are other important things to test that require prac exams to test, but asking for written design, pseudo-code or even code in some exams is, imo, a good idea.
benefits of paper programming (Score:5, Insightful)
One of the things that universities are supposed to do is give you a chance to express your knowledge in new contexts. It's relatively easy to apply things you've learned in familiar contexts, but how good are you at applying them in unfamiliar environments?
Making you work with pencil and paper forces you to plan ahead and architect your solution, not just jump in and start writing code.
When you write by hand, the logic and aesthetics of your code are especially important -- since you (and the reviewers) can't run the code to see whether it works or not, you need to make an extra effort to make it transparent and comprehensible.
I might take issue with an exam that was nothing but one big handwritten coding assignment, but I don't think this is at all out of place as a part of an exam or course. It's a little like essay questions in a soft-science class; they're unrealistic because in the "real" world you have access to reference materials and time to spell-check and so on... but at the same time they are a useful gauge of your ability to articulate a subset of your knowledge, to think on your feet, etc.
Also, in courses I've graded where such exercises where required, we usually didn't worry about things like minor syntax errors that didn't obscure the intent of the author. The goal was to look for an understanding of and ability to solve the problem, not to be human compilers.
It depends what they ask and how they mark (Score:5, Insightful)
Write some code to rotate/scale an image.
Fine an element in this sorted list (of say 10000)
Give this xxx write a class model for it.
Now as long as they don't expect perfect code, eg. forgeting to +1 because of a rounding error, or a binary search can result in a high or low result(for a non-perfect match), these seem fair enough questions to me because:-
They all test your abilty to write good code, not just code that works.
I know loads or 'programmers' who don't know how to turn a recursive function into a loop.
Would use sin/cos to rotate each point on the image, instead of using an interger line drawing algo to scan accross the image to rotate it (or even floating point)
Would write a n order search instead of a binary search.
And would use a poor or unflexable design pattern.
So long as they mark your programming skills and not how perfect your code is, i don't see a problem.
Debugging is also a key skill, but is very hard to test in an exam, a good debugger uses a hell of a lot of past experiance and insperation to locate, fix and check that there are no simila bugs.
Answer: Not fair. But we have to do it anyway. (Score:5, Insightful)
I used to teach a class to engineering freshman, and part of the course was writing code in Matlab. There are in fact a bunch of matlab questions that you can ask that aren't just writing code, but you really don't get complete coverage unless you have the student put some code on the page.
Additionally, there probably was some cheating going on with the homeworks (this was a large class taught to all freshman, with relatively small programming assignments. It would be difficult to prove someone was cheating.) We needed a controlled environment where this couldn't happen.
However, we all realized how crappy this was for the student, and we graded the assignment more like it was psuedocode. If the code had obvious typos, we ignored those. If they were close on syntax, but a little unclear on the API, we usually let it slide, too. (We understand when online you can look that stuff up with the online help.) The thing we stressed was the overall algorithm itself, as well as a demonstration that the student knew how to use matlab. (After all, as a TA, I'm taking programming courses myself; I have some idea of what's reasonable to expect and what's not)
If you do have a written CS test, and the prof does mark you off for syntax errors, I would probably go in and (politely) complain. Say, "I made a typo which would have been caught by the compiler, but I got the gist correct." If you demonstrate that you know what you're talking about, and you're not a jerk about it, and the professor's not a total jerk, I bet you'll get most of the points back. (We would do this all the time, especially if the student could clearly explain what they were trying to do.)
If the professor or TA is a jerk, all bets are off, though. But if they're a jerk, they aren't interested in being fair anyway, so you were screwed from the start. We've all been there. Sorry.
Re:Learn this skill (Score:2, Insightful)
Bad grammer on an English test? Gee, that sounds like arithmetic errors in a calculus test!
Re:AP Computer Science (Score:2, Insightful)
When I took the AP computer science test in high school we had to code on paper! I'm just like you...I get my vibe when I am in front of the computer coding, not with a pencil and paper.
Well, I will be taking AP Computer Scince next year. I have seen copies of the test, it seemed pretty resonable, IMHO. How did they grade your exam? Did they give you oppritunity to contest the grade if you felt it was graded incorrectly?
DESIGNING code (not typing it) is *hard* (Score:3, Insightful)
I can't write a java main() statement. Just can't remember how - don't do it very often. But my emacs expands when I type 'main', so it's not a problem
I couldn't write push and pop and get it to compile, either. But I know how. And I can communicate the design.
If you're talking about "can't design push and pop", then my sympathies. If you're talking about "can't get the {}'s right", then I don't want to work with you anyway.
Re:I feel the same way (Score:3, Insightful)
Indenting is relevant to this for the same reason that Python uses only indenting, not braces or begin-end, to indicate block structure.
Yes, one school of coding in a text editor is to place the open/close braces first and then fill in the code. As you say, that's tough to do on paper. It's also a crutch.
Another school says indent your code properly and the braces are superfluous, just there for the compiler (for non-Python languages).
If you follow the latter train of thought, you just put in the closing brace when you're done filling in the intermediate, indented code.
overrated? (Score:2, Insightful)
Re:Error checking by compiler (Score:2, Insightful)
You're supposed to write perfect code the first time around? I agree that it's an interesting test of your knowledge of a concept, but it's completely unrealistic from the standpoint of what you would do in real life (as much of academia is, I'm afraid). I can write code that is pretty close, debug through compile and runtime errors, unit test for missed functionality, etc., and have something working in half the time it would take me to attempt to do all that on paper, analyzing each statement "by hand." Is this not the way that 99% of all developers work?
and second, you should not be looking up stuff with man if you ever get a job... just wait untill the boss finds out that your spending your payed hours learning how to code when school should have taght it to you...
I use man as a reference, as I would imagine most people do. I don't keep every last detail of every function or program I'm going to have to use in my brain.
I know that both of these arguments are somewhat counter to the purpose of academia. I have an HP calculator that will do integrals for me too, but can obviously see where that wouldn't be allowed on a calculus test where you're demonstrating your ability to integrate. But as for writing code on paper, it just never made much sense to me. It's completely outside the medium for which it was intended, and completely counter to the process which most developers use to write code.
If you want to test someone's knowledge of a computer concept/algorithm/etc., have them write detailed flow charts. Get 'em used to it. Make UML requirements. These are the kinds of things that not only make for good development practices, but also will have a practical application when students actually get out of school and start working.
My $0.02
Are written plumbing exams a fair measure.... (Score:5, Insightful)
Once again, Slashdot continues to perpetuate the (Computer Science == Programming) myth.
Programming IS NOT "computer science" any more than telescope design is Astronomy (with apologies to Dykstra).
Programming is a trade skill. Certainly, a challenging and rewarding one.. but it is not a science.
Being the first guy to figure out quicksort... thats science... thats research. Coding a quicksort in LISP is not.
I've ranted about this before, but it upsets me to see such a misleading headline.
I think *computer science* exams are quite fair... if you understand the data structures and algorithms associated with B-trees.... you shouldn't have any problem describing them on an exam. Likewise, if you understand the halting problem, reducing a simple contrived example should be doable on an exam.
On the other hand, I very much disagree with the "code this up in PL/I, using proper syntax" type of exam question. What an enormous waste of time...
The bottom line is that if you've been through a computer science program and understand the underlying principles, I should be able to give you a manual for 'trendy language x', a few examples, and 48 hours. After that, you should have no trouble coding in Jav^H^H^H 'trendy language x'. Will you screw up the syntax occasionally? You bet. Is it fair to dock points on a formal exam for being human and forgetting a semicolon? I sure don't think so; and I've *never* done that to my students.
But I stray from my point. Once more, for the record, since no matter how often I rant about it, no one seems to listen:
COMPUTER SCIENCE != PROGRAMMING.
Re:Learn this skill (Score:5, Insightful)
I find that people's performance over the long haul has nothing to do with a 30 minute interview coding session. Some people don't work their best when under pressure, especially when they are interviewing. A lot is riding on that interview, and the stress does very odd things to people.
Re:Learn this skill (Score:5, Insightful)
Most likely? I doubt it. Just because you may have done it doesn't mean everyone does. I've never had to write code on a whiteboard in a job interview. I *have* had them give me a program spec and say "go home, make this work, and email it to me" That was even better in my opinion.. there wasn't a time limit, so they can judge you on how fast you work when there's no motivating deadlines, and the project can be a lot more complex than "write a 5 line routine on a whiteboard"
Now I'm the one doing the hiring, and that's exactly what I do.
Re:Error checking by compiler (Score:2, Insightful)
Well, I don't think that's fair at all... You are paid to do a job, and it's easy to be effective without memorizing tedious information. Compilers are better at finding missing semicolons or parentheses that you are. Why not let them do it? You might know what a library function does, and how to use it, but maybe you forgot the order of the arguments. Sure, a guess probably won't compile, but it doesn't mean you are a bad programmer. I'd rather work with someone who has a strong grasp of OOAD and can learn the specifics on the fly than someone who can write code that compiles every time out of the gate, but doesn't "get" the whole "objects" thing. (At least in my case, where we work in an OO environment.)
Part of being a good programmer is using the right tools to maximize your effectiveness. The compiler is a great syntactic and semantic validator. Let it do that for you. The manual is much better at retaining information about specifics. Let it do that for you. As long as you can use these tools to craft quality code, then you are being effective. There's insignificant cost to running the compiler twice as often, or looking something up in a manual or on the web. You tend to memorize the things you use really often, but what you use can change from project to project, or release to release. It's hard for an employer to find someone who already knows everything exactly that they want... and when they get them, they may not be as effective/productive as someone who has to learn part of it on the job.
If it's a class in Specific Language X Syntax, then maybe that makes sense, but that's a useless class. At my school, we did a lot of hand-writing code on tests, but usually pseudocode was sufficient to get full marks, and you could get good partial credit if you had the basic idea. The focus was not on details like syntax, but on the concepts, ideas, paradigms of the subject (compilers, operating systems, computer graphics, or whatever).
You are hurting yourself if you stop learning once you are out of school. Technology is always moving and changing, and what's available and how to use it might change on you, you have to adapt. You should always be learning, 1 year after school, 10 years after school... why stop? No one I know at anyplace I've worked ever expected anyone to know everything about what they were supposed to do. They care about results... providing those requires being able to learn effectively on the job, because who knows what you might be doing next release, next project, next company... Being versatile will get you exceeding expectations all the time. Success is rarely about what you know, but how quickly and how well you learn, and whether you can make use of it.
-If
Comment removed (Score:2, Insightful)
Whats up with these places... (Score:3, Insightful)
The coolest professor(this was a EE class) I ever had did it this way - he made the tests extremely hard, but allowed open book, open notes, and calculators. His philosophy was that on the job you would use these tools, so you might as well use them on the tests. You were competent in the subject if you made it out with a C, and perhaps 2 people in a class of 20 would get an A.
Actually, I think CS tests are pointless, period. They have very little to do with your skill as a programmer or knowledge of the subject. Its much better to give hard assignments that require lots of work before they are due(i.e. a two week assignment that is virtually impossible to finish the weekend before it is due). I can see concepts on tests, and perhaps a few functions to demonstrate knowledge of certain algorithms, but a 500 line program is ridiculous.
Re:Writing code (not typing it) is *hard* (Score:4, Insightful)
Ugh. You've got everything backwards; I almost can't believe you're serious.
It's fine to have to resort to a reference book, but the thing you should be forgetting and having to look up, is how to use your library (e.g. the names of functions, the order of their parameters, etc), not how elementary data structures conceptually work.
If you can't implement a list or stack, it does show an inability to solve problems. Here's the problem: solve it! Can't? Then you're not a problem solver.
It's ok to forget libraries and syntax; seanadams is in the minority in thinking that matters. But stacks and linked list aren't things you should have to remember -- they're something you should be able to derive and re-invent on-the-spot. If you can't whip them up ex nihilo, then your problem solving skill is ..um.. frumious.
Re:simple (Score:1, Insightful)
Step 2: ???
Step 3: Profit!!!
These tests are wrong. (Score:3, Insightful)
As a side note, having learned touch typing, it's really hard to write on paper. Thoughts flow much faster than handwriting, and you end up losing yourself sometimes. I'm positive that also affects other coders when doing written tests.
You need to be able to do this (Score:3, Insightful)
First, it's something that will come up during an interview, and you can look really bad if you can't do it. I have given more than one interview when an otherwise impressive candidate was turned down because they couldn't write a relatively simple algorithm. I've also been on the other side of the table, where it really gave me an opportunity to distinguish myself. (Btw, from an interviewer's perspective, what's more important there is the approach, not the end results. I wouldn't be surprised if many of your teachers feel the same way.)
Also, it's often necessary to be able to write out code when you're working in a team. It's not always enough to just describe your approach, especially if another programmer wants to go a different way. It's like any visual aid, it helps you get your point across.
Finally, it can be useful when working on your own. I sometimes find that it's helpful for writing complex programs if I sketch it out longhand first. It helps me guarantee I haven't left anything out, so I don't have to go back over my code as much.
So, is having students write code on an exam a good way to test their coding skills? I don't know. But maybe coding skills aren't all they're testing.
Paperboy.. (Score:1, Insightful)
When you get to a certain maturity level at programming you should be able to do it in a methodical manner - paper and pen first, before coding and the keyboard.
Start coding with discipline. 'on-the-fly' coding might seem far quicker, but wait till you face large complex problems.
Programming is a necessary tool (Score:4, Insightful)
Learning the exact syntax and idioms of a particular language in an introductory class is necessary, as it teaches you how to learn a new language as much as it teaches you the basics of programming.
Re:Error checking by compiler (Score:2, Insightful)
Exactly =) Programming, as a career, is _not_ memorizing all the man pages, but knowing how to use them quickly, and learn on the fly. Every task you do is likly to be different, so why bother memorizing everything? i, for one, use them for function argument order, which i never remember because all the libraries i use have slightly different orders.
I suppose it'd be better to think you get it right, only to find a logical error somewhere down the line when you provide the source and dest structs in the wrong order..? right
Re:ABSOLUTE BULLSHIT!!! (Score:4, Insightful)
Wow, I'd really like to know what kind of programming you've done lately. Perhaps in procedural programming there is a set place to put things, but as far as OOP goes, I'd say there's quite a few ways you could arrange things. People have different reasons for coding things differently, both syntactically and in organization. One person may find it more convenient to store something in a simply ASCII text file, someone may find a small but efficient database system better.
I agree, you shouldn't use more resources than you have to, but there are multiple ways of accomplishing things, each with their own benefits and drawbacks. If you can't see that, I'd really hate to see what happens when you have to work together with someone on something.
Re:Hate it! (Score:2, Insightful)
But if they knew enough about coding to do that, they'd have a real job and wouldn't be grading written tests.
Oh well...it was a nice idea.