Computer Vision Applications and Linux?

blackcoot asks: "I'm about to start work on a summer project with my advisor, part of which involves getting a lab set up for real time (or as close as we can get) computer vision applications. Currently, the only machines in the lab with digital video cameras attached to them are Win2k boxen, and that's only because we haven't been able to find reasonable drivers for the firewire cameras that we have. My advisor and I would much rather *not* write our own driver for these cameras; right now my advisor has the budget for a couple reasonably inexpensive cameras. This leads me to a couple questions: are there firewire (or equivalently high bandwidth bus) based digital video cameras (not framegrabbbers) out there that have reasonably solid Linux or Video4Linux drivers; have any Slashdot readers tried to build a vision type application under Linux (something that does more processing than Xaw TV); Am I setting myself up for a whole world of pain trying to make this happen under Linux, or is it doable? Any words of wisdom? Your help is much appreciated."

Linux?

[ObviousGuy]

What's the problem with Windows?

Wise man say: If you need two fish and already have one, do not throw the one you have back to fish in unknown waters.

Ask me in about three months

[flikx]

I'm with the SAE Walking Machine Challenge team at the University of Utah, and part of the goals for this upcoming year is to incorporate a vision system with rudimentary object recognition into the existing walking machine platform as part of bringing the machine to full autonomy.

I've utilized RT-Linux on an onboard 'normal' PC as the control platform, and have no intentions of changing it. Unfortunately, I cannot be of much real help at the moment, since I'm at the same stage as the article submitter. But if anyone is interested in details about the system, I may be of help in the future.

one word, Mac

[norwoodites]

A word an iMac would do the job, Quicktime would be the api you could use and process it as it comes in.
Use Mac OS X and get UNIX for free.
It would cost you about $900 US for an eMac. About $700-$800 for an older iMac And about $1300 for a new flat screen iMac.
Also you get codecs that you can use that are stable.

Did you look in the kernel sources?

[zenyu]

I've written a vision program in linux and later one in Windows, I can't see ever doing the windows thing again. By the end of the project I had spent much more time on Windows annoyances than it would have taken to write a linux driver.

Why not use a solid framegrabber like the Osprey? Do you have some super high res progressive scan cameras or something? Most vision research is done on prerecorded video so I'm assuming your doing some real-time tracking. If this is a short 2-4 month project and you need the firewire cameras sticking with the working Windows setup might be the answer. If it's longer there is at least one good book on writing linux drivers (there are pointers in the kernel Docs.)

Look At iRobot

[Aix]

You might check out iRobot [irobot.com]. I work at the MIT AI Lab [mit.edu] and we use the B21r for my project, which has similar goals. This robot was originally designed by a company called Real World Interface (RWI) that was ultimately purchased by iRobot. In any case, the robot runs Linux and uses V4L as far as I can tell, with what appears to be an off-the-shelf Hauppage WinTV card or equivalent. I don't know much about the camera, but conceptually, you should be able to set something up very similarly.

Now, as far as actually writing the code for real-time vision... that's up to you - good luck - it is a difficult problem.

Re:Look At iRobot

[fille]

Looks great! Especially the Magellan seems cool for "domestic use". How expensive are these things anyway?

Re:Look At iRobot

[Aix]

I really don't know. The group already had the robot when I arrived. I think they bought it in 1999. It is pretty loaded full of stuff, so I'm guessing they aren't cheap. It has two computers, a 180-degree laser range-finder, two sets of sonar range finders, IR range finders, a pan-tilt camera, CORBA-based software to control it all and enough lead-acid batteries to keep it running for about six hours between charges. (Not to mention all the drive servos and various shaft-encoding systems) Uses 56K wireless also. I think it came with a laptop, also. I would guess it was around $25K, but I really have no data with which to back that number up.

Depends on what kind of camera

[Omega Hacker]

If you're referring to webcam or scientific-style cameras on the Firewire bus, they are supported by programs like Coriander through the libdc1394 interface and corresponding kernel drivers.

If you're talking about consumer-grade video camcords (aka MiniDV), you'll use the dv1394 interface and corresponding kernel modules, plus libdv [one of my projects] to decode the images (on a fast enough machine, about 700MHz PIII pegged still).

I haven't heard of any other types of firewire cameras, so if you have something that doesn't fit into one of these two categories, you got shafted IMO.

Depending on your processing requirements as far as data-flow, you may want to look at GStreamer, another of my projects, to plug together the DV device interface, DV decoder if necessary, and your processing components.

All of these are found on sourceforge under the names given, putting that many URLs into the comment when I'm this awake would be bad ;-)

Re:Depends on what kind of camera

[isaac_akira]

There are lots of consumer-grade firewire webcams out there, and they should all work with linux (yay standards!). They use the same libdc1394 interface as the scientific cameras do. I'm using the Fire-i cam from unibrain.com, which cost about $130, but there are several others at that price range.

This site has a lot more info on linux and firewire:

http://linux1394.sourceforge.net/hcl.php [sourceforge.net]

p.s. it actuallys work. ;-) i'm running 320x240 YUV422 video into my image processing app at 30 frames a second. 640x480 works too, but it looks a bit grainy (the fault of my camera's lens and ccd) and that's too much info for my app to process anyway.

Write the driver

[inerte]

You have an itch, so scratch it. Write the damn driver and everyone (including people you will ever know they existed), happy.

check out intel library

[Anonymous Coward]

it may not be entirely related to your post.

by the way, intel IPL library does support linux.
it's a pretty cool library (written in assembly for optimization purpose). So you can expect a lot of optimization.

Also, openCV, open source computer vision, which is built on top of IPL is also available.

What's your application?

[jd_esguerra]

If this is strictly a summer project (for you) and you have very little computer vision experience, I recommend spending your time learing basic computer vision algorithms/processes. They are pretty much the same from platform to platform. Regardless of the OS, get a student copy of Matlab and the Image Processing Toolkit. It won't do real time stuff, but it makes algorithm development and experimentation REALLY easy. Once everything has been fleshed out, convert to C,C++. It sounds like you are building a generic rig for a variety of experiments. For real time vision, a framegrabber is recommended. This combo might be a bit more expensive up front, but the framegrabber software will get you up and running more quickly than having to write your own driver. (I doubt you will learn much computer vision writing device drivers.)The Mutech MV-1000 has decent linux support, and does not cost too much. ($1500?) Look at Matrox too. Other practical things to consider: The "linux comfort level" of people who will be taking over your project in the fall, interchangable lenses, GOOD lighting (very often overlooked), color or grayscale, quality of images/data. I'm working on a stereo vision system for autonomous mapping/navigation. I do much of my initial algorithm development in linux/Matlab using images from 2 defective USB webcams. Once the rest of the $$$ comes in, almost everything will be converted to C under win2k/matrox. The non real-time stuff, like camera calibration, and NN training will remain Matlab. Summary: If you are planning on doing a lot of computer vision stuff, go ahead and spend time playing with your hardware/software options. If this is a 3-4 month fling, start writing/playing with computer vision routines as soon as possible.

Xvision

[randombit]

Xvision is a library written at JHU (originally at Yale; the main people moved over here a few years back). I used to work on it a little bit (theoretically, I was the sysadmin, but kinda I got roped into working on it for a while a couple summers back). It's not bad, though the code is pretty dirty in places. It's used to program the robots for that robot soccer competition.

Hrm... a URL... Ah, here:

Xvision2 [jhu.edu] page.