Voice Recognition and Programming?

dsaxena asks: "I'm suffering from repetitive-stress injuries to my wrists/foreamrs from typing too much. Luckilly it hasn't progressed into carpal tunnel syndrome yet, but it will if I'm not careful about it. I've been looking at alternative input methods and was wondering how many people have tried using speech recognition for programming? I've seen demos of speech recongition software (Dragon and ViaVoice), but it's awlays been in the context of writing letters or reports. Would it work as an efficient input method when trying to code in C?"

"It's all in the grammar."

Voice recognition still has a ways to mature before it is robust enough for coding since a lot of programming involves the use of symbols, not words. Sure, you can assign "names" to these symbols and start speaking away, but it would be cleaner if voice recognition engines are made aware of the syntax of specific languages and would do this for you (many modern editors have this feature). Is voice recognition moving in this direction, or do coders have a while to wait before they can start using their keyboards as paperweights?

IBM Viavoice for Linux

[G Money]

While I haven't tried using it for programming, IBM's speech recognition software has so far been far more acurate than Dragon's Naturally Speaking. The Linux version is solid (the java frontend is a little kludgey, but getting there) and it is designed so that you could add it in to existing systems fairly easily. Just putting some hooks into emacs would at least let you dictate into it. You could voice enable much of your system by putting hooks for it in KDE or Gnome. But as for programming, I'll leave that for the experts to decide.....

Re:Function names?

[Zan Thrax]

because. Do you expect open to create [, {, or (?
On another note, isn't times quicker than multiplied-by?

Voice strain

[exitzero]

It is worth noting that case studies have shown that the vocal chords can suffer the same kind of injuries that hands and wrists can suffer when used in an abnormal fashion, like when speaking strongly and clearly to a speech recogniser.

So speech recognition may not be the ideal answer you are looking for.

Nuance as an ASR for voice programming

[johnmark]

Actually this sounds pretty practical. I'd use something like the Nuance speech reco engine, which has a grammar based recognition step that constrains the text conversion. The grammars are word based regular expressions (and a bit more.)Conceivably one could write re grammars for fragments of the language that you are programming in. You'd have to write these specialized grammars, though.

Re:how about handwriting?

[Mr. Sketch]

Hmmm, I'm not sure if this would actually help or hurt. I personally find hand writing to be a lot worse on my wrists than typing. After about 2 hours of hand writing my wrists are hurting, but I can code all day (and I often have to) without bothering my wrists at all. But I suppose for some people it might be more natural and easier to write by hand instead of type, but not for me.

Re:Function names?

[icezip]

Very much agreed. I program PHP most of my day, and I can't imagine speaking 'for open bracket cash i equils zero semi cash i less than four semi cash i plus plus close bracket open brace..' all day long. I would spend most of my day thinking of how to say what I want rather then just typing.

I remember seeing on television a year ago a input device that fits right into your hand, has 5 buttons and using combonations you can make the chars you want. Don't know if this did brackets, semicolons or braces though.

Re:Function names?

[.sig]

Actually, ( is open parenthesis, [ is open bracket, and { is open brace (or 'that squggly little bracket thingey,' as a friend calls it :-)
It'll probably be awhile before something as abstract as coding can be easily done with voice recognition software. Until then, just invest in those wrist pads and a trackball... works for me...

Re:Been there, done that, gave up

[twisteddk]

I can understand that. Even though a voice recognition software programme MIGHT be usefull when entering the programme. IT's certainly not very efficient for editing existing programmes. Which. I know most coders spend a HECK of a lot of time doing (most spend more time editing programmes than making new ones, as editing is usually required to satisfy custumer needs, testing, etc.).
So even though You might find a piece of Voice software that was made for C/Pascal/SQL/whatever You would still have to do a heck of a lot of typing. So don't count of being able to "ease" Your way around this problem. My solution was simple. I just stopped resting my arms/wrists on the desk. It takes a bit more effort, and a new mouse. But I no longer have problems with hurting/numb hands or arms at work (at home is another matter entirely).

Step away from the computer...

[RhetoricalQuestion]

When your hands and wrists start telling you that they don't like coding non-stop everyday, it's time to stop coding non-stop everyday. It's hard to believe, but life exists away from computers.

The poster above made an excellent point about compulsiveness. There are students at my school who, after their 4 years of undergrad, are so crippled by RSI and carpal tunnel that they can't work in the field they studied. The first day of the first year programming class is now a lecture on RSI and how to avoid it.

A friend, who doubted the existance of RSI, once asked me how it is that RSI exists now, but didn't exists years ago, because otherwise secretaries would have suffered from it as far back as the 40's and 50's. I pointed out that secretaries in the 40's and 50's worked 8 hours a day, and didn't type when they got home.

In addition to recognition software and therapy, you've got to learn to chill....

I'd love to,

[human bean]

but they keep emailing me spec changes...

Very hard to do

[chris_lamb]

Programming languages are not designed for input by speech. They often have "words" that sound very similar, and there also have irregular white space - do you want a space after the word you just spoke or not? While speech recognition will one day work effectively for inputting spoken languages, I doubt it will ever be particularly useful for programming languages that are not designed to be spoken.

he he!

[madmooer]

reconation is funny. i once had one.

Re:Function names?

[madmooer]

how about int i equals sizeof open int close multiplied-by four semicolon

Re:Function names?

[madmooer]

Not really. What if the elements of speech where higher level, e.g. for i pause set counter zero pause counter greater 4 pause inc counter, or even for i, zero, 4, 1.

C provides shorthand for common operations such as incrementing a variable by one, why can't speech to code engines do the same? A bit of syntactical sugar would sweeten everything up.

Re:Sore throats, and how do you pronounce '}'

[madmooer]

good question

Re:Function names?

[madmooer]

or even int i equals sizeof int multipled-by four [implict-semicolon, denoted by pause]

works for me

[fat_hot]

I was diagnosed with tendinitis/carpal tunnel about nine months ago, and was forced to learn to program using voice recognition software. I program entirely hands-free, using Dragon Naturally Speaking Professional; while I haven't tried other systems, I don't imagine I could do without its programmable macros.

I'm not sure whether I'm less productive; I used to just puke code out in great volume, but now I'm slower and, I think, more deliberate. one absolute requirement, though, is PRIVACY; I used to have a cubicle, but my neighbors and I drove each other crazy once I started talking to my computer. Fortunately my manager was able to find me an office.

There's some good information on the Web for people like us. You can try http://www.codevox.com [codevox.com] or http://www.voicedeveloper.com [voicedeveloper.com] to start with.

Not appropriate for vocal input

[Tet]

With a bit of work, voice recognition systems may be taught to recognize the syntax for some programming languages. However, all this means is that you can dictate a program to the machine. You can't actually program it because programming is not linear in the same way as writing prose. You'd need to be able to tell the editor to go up 3 lines, or to copy the next 7 lines into the do_more_stuff function. Voice recognition just isn't an efficient interface for doing that sort of stuff. The keyboard will be with us for quite some time to come (even if it may evolve into more efficient forms, such as a stenograph machines -- unlikely, but possible).

how about handwriting?

[josepha48]

It would be possible to write your program with a stylus and touchpad. It would need to be able to recgonize your hadnwriting too. I am wondering why systems like these are not making there way to the pc. I mean if you can do it on a palm, you can do it on a pc as well. This would make more sense if you ask me. It would not have to deal with all the symbols differently it would be another character. You can usually write pretty fast and once you get used to it this may work. You may even be able to get rid of the keyboard as well.
~~~~~~~~~~~~~~~~~~~~
I don't want a lot, I just want it all ;-)
Flame away, I have a hose!

Function names?

[DeHackEd]

I think the biggest obstacle to overcome would be the funny names programmers assign for their variables and functions. While I use dragon naturallyspeaking (which come with Corel WP Suite 8 Win9x) and it does support saying things like "OPEN BRACKET", a function name like vsprintf would be difficult to choose how to put that into normal words. And then you have to type in the name of the word into the add to dictonary feature and then say the word three times to program it. Seems like a lot for work for the whole LIBC.

Also, I feel that programming languages were really meant to be written by keyboard. Saying int i equals sizeof open bracket int close bracket times four semicolon just seems like too much work compared to entering it on a keyboard. Even then i will most likely be misinterpreted as eye.

I think its possible to write code in such a way, but perhaps too impractical. It would either involve creating a whole new, large dictonary for your programming needs or maybe your very own #include file to convert things like equals to =.

Sore throats, and how do you pronounce '}'

[human bean]

When I am programming, I often crank ten screens worth per day. By the time I finish editing these, I have probably typed ten times that. I would hate to speak all of this, especially if working in the classic "open cube farm".

This could be worse than those annoying jackals from the marketing department, who when cubed near us insist on monitoring their voicemail by speakerphone, and always at top volume. Grrhh! Or the admin from hell who insists that everyone benefits from decibelic exposure to retro radio. "All disco, all the time..." Smash! Whump! Mrphf! (sound of small boom-box radio receiver being forced up human orifice)

Maybe you could get a better keyboard, and some macros for it?

Hybrid speech/stylus interfaces

[hatless]

Man, the signal-to-noise ratio here just keeps getting worse. Dvorak keyboards and "good" keyboards ain't gonna stop RSI. Dictation software alone isn't well suited to coding, what with the massive amounts of critical punctuation and formatting. And handwriting and gesture recognition is just plain slow compared to typing or speech.

What I'm surprised by is how little imagination the handwriting recognition and speech-recognition developers have. Dictation software is usually used by people who *do* have the use of at least one hand. So why not have hybrid interfaces?

The tedious process of editing material in a dictation environment could be simplified and sped up greatly if you could simultaneously use a stylus to jump the cursor around and scriblle out or mark errors--or make corrections. You could also be presented with floating popups showing choices for ambiguous words and phrases the dictation engine is having trouble with even as you continue to dictate.

I think a hybrid interface like this--using handwriting, gestures and speech together would allow for quicker and more precise and efficient dictation, and would make dictating things like source code or scientific data and equations at least as quick and fluid as keyboarding.

It really is about time the keyboard and that other vile RSI torture device, the mouse, started to go away for most applications. Shaking developers out of the misguided focus on 100% hands-free dictation and all-or-nothing handwriting recognition would help immensely.

Just one question

[jaa]

since I can't really answer yours:

are you getting this (early ct syndrome) professionally treated? In the words of jwz:

"Do not fuck around. If you are experiencing any kind of pain, get to a doctor and get it diagnosed [jwz.org]."

Looking for alternative solutions is great, but don't do it to the exclusion of professional help.

Best of luck.

alternative Keyboard layouts

[Jordan Block]

You cold awways try out a different keyboard layout. The Dvorak layout is supposed to minimise repeditive strain injuries.

I had a mild bout with the same thing not too long ago. I went through some Psyiotherapy and stuff to deflame (is that a word??) the nerves in my elbows that were causing my hands to go numb.

Now I'm just more careful about how I rest my elbows on my desk, and I'm doing much better.

Been there, done that, gave up

[mouseman]

When I was finishing up grad school, I had to deal with RSI. I tried physical therapy, funky keyboards, dvorak, and, yes, dictation software. I wrote much of my dissertation using dragon dictate, and did some programming with it as well. I had voice macros for common symbols, keywords and emacs commands.

Did it work? Yeah, sort of. For entering normal text, dictation software has gotten to the point that it is definitely usable -- by someone who absolutely can't get by without it (95% accuracy seems really good until you realize it's getting one in 20 words wrong. and your spellchecker won't be any help). But editing programs is harder -- you spend a lot of time making changes to existing text, which is hard to do by voice. After leaving grad school, I gave NaturallySpeaking(TM) a try. I found it better in some ways, worse in others. I think it is far worse for programming than dragon dictate. at least the version i used was (maybe it's been improved), because you could only dictate into a dedicated window, and there was no support for voice macros (IIRC).

So where am I now? Pain-free most of the time. I have flare-ups from time to time and fully accept that this is a lifetime condition. I still use dvorak; it *feels* better, but that's about all I can say. Since leaving grad school, I spend less time in front of a keyboard, by virtue of spending more time in meetings. :-( That, plus (somewhat) better habits has made a world of difference. I think of RSI as a physical symptom of a psychological condition: compulsiveness. A normal, healthy person does not keep doing something that causes pain. I think the key to dealing with RSI (note: not recovering from -- it's too late for that) is to listen to your body and stop doing things that cause pain. Dictation software might work as a stopgap, but if you don't make a fundamental change in your habits, you'll just end up with a strained voice in addition to your strained wrists.

good luck!