Ask Slashdot: Tips On 2D To Stereo 3D Conversion? 125
An anonymous reader writes "I'm interested in converting 2D video to Stereoscopic 3D video — the Red/Cyan Anaglyph type in particular (to ensure compatibility with cardboard Anaglyph glasses). Here's my questions: Which software(s) or algorithms can currently do this, and do it well? Also, are there any 3D TVs on the market that have a high quality 2D-to-3D realtime conversion function in them? And finally, if I were to try and roll my own 2D-to-3D conversion algorithm, where should I start? Which books, websites, blogs or papers should I look at?" I'd never even thought about this as a possibility; now I see there are some tutorials available; if you've done it, though, what sort of results did you get? And any tips for those using Linux?
Here's a tip (Score:4, Insightful)
Don't do it.
Re:Here's a tip (Score:4, Insightful)
Re:Here's a tip (Score:5, Insightful)
Exactly. People with the proper equipment and money fail at this regularly.
I think it is important to arrive at the correct mindset. This has never stopped people from snapping pix at weddings and sporting events and tourist traps, even if their pix look like garbage compared to a pro photo on a postcard or whatever.
If you want to do it for fun, heck yes go for it. Go Go Go. You don't need help just try it.
If you think you'll turn out something that means anything to anyone else in the world, you'll probably be disappointed. Insert stereotype of goans when someone wants to show you old fashioned slides of their vacation. Although that old tech is getting kind of retro cool now.
Re: (Score:2)
Hollywood has released big-budget movies costing tens of millions of dollars to produce, using 2D-to-3D conversion, and the results have been terrible. Hollywood may suck at coming up with original storylines and good plots, but their skills at technical effects are unequaled anywhere on earth; if they can't do it, no one else can either. The whole thing is a bad idea. If you really want 3D imagery, you need a 3D camera.
Re: (Score:3)
I may be nitpicking, but ugh it's important... Cameras are only 2D. For good stereoscopic output, you need a stereoscopic camera. The recent "3D movies" are really called stereoscopic in the industry, "3D" is strictly a marketing term. 2D to 3D conversion, which adds a Z-buffer (the third dimension), still outputs a stereoscopic image in the end (which would be 3D stereoscopic, being generated from a 3D model).
Let me elaborate on that (Score:1)
Don't do it.
Really, just don't do it, please!
Re: (Score:3)
Re: (Score:2)
I'd second this. I hope that 3D is just a gimmick that falls out of style rather quickly.
Most certainly not. Converting 2D footage to 3D is a horrendous endeavor and should be stopped - or at least left alone. But well-done stereoscopic footage has an added value IMO.
Now, they've improved resolution and stereoscopic aspect. What I'd like to see next is the framerate. I find 24fps vastly insufficient to relay the feeling that "you're there", and whenever there's a big tracking shot, I find it choppy at best.
I've watched 100fps footage and it does make a heck of a difference.
Now, there's plenty o
Re:Here's a tip (Score:5, Interesting)
You're in luck... Peter Jackson is pushing 48fps over 24 in the cinemas, stating enough digital projectors are capable. He's shooting the Hobbit at 48fps, and shooting it in 3D from the get-go. I'm more interested in the content of the movie, but I'm expecting it'll be one of the best, if not THE best, attempts at 3D so far (Jackson Explains "Hobbit" 48FPS Shooting [goo.gl]).
He's trying to encourage future film productions to step up to 48, too.
Re: (Score:2)
Wow, thanks for the tip. I didn't know it was that close.
Re: (Score:3)
Re: (Score:2)
Sucks for the folks who bought 120Hz TVs in an attempt to eliminate telecine judder; now they'll have to upgrade to 240Hz.
Re: (Score:1)
Most significant comment I've seen so far. I'm happy to do without 3D TV. I'm not happy with the idea of buying a TV with a higher refresh rate to eliminate this annoying effect.
Hopefully the industry will settle on 60fps, instead of forcing yet another round up TV purchases.
Or... frightful thought.... is this another standards war brewing, in the style of Blu-ray versus HD-DVD? Where are the content producers other than Peter Jackson on this issue?
Re: (Score:2)
Sucks for the folks who bought 120Hz TVs in an attempt to eliminate telecine judder; now they'll have to upgrade to 240Hz.
It's OK, TV's today don't usually last more than 5 years. Which is fine - they're quite nice and dirt cheap for their size and price.
Re: (Score:2)
I sure hope you're wrong; I bought an LED-backlit TV specifically for maximum longevity.
Re: (Score:2)
I sure hope you're wrong; I bought an LED-backlit TV specifically for maximum longevity.
I did too - a 37" for $400. My last TV was a 27" purchased in 1993 for $730. On an inflation-adjusted basis if I get 5 years out of it the yearly cost will be about the same.
The TV repairman who used to fix my CRT does most of his business with flat panels and gave me the 5-year metric to work with. I bought the extended warranty for $20 to bump it from 1 yr to 3 yrs.
Re: (Score:1)
Re: (Score:2)
Someone is going to have to explain to me how theaters are going to project this, because the DCI stereoscopic standard (pretty much THE stereoscopic projection standard if you're not IMAX), used by RealD and others, stores 24 FPS stereoscopic movies as 48 FPS and alternates left/right eyes. 48 FPS stereoscopic would mean handling 96 frames per second, which is far above the capacity of most already-installed digital cinema projectors.
Re: (Score:3)
Be glad you haven't seen that yet. Those "upscale" (for the lack of a better word for it) algorithms make the picture look absolutely horrible, much like a cheaply produced VHS Porno. Now, there's nothing wrong with pr0n, but I'm not especially a fan of the cheap 80s porn esthetic. I'd rather scratch my eyes out than watching a complete
Re: (Score:2)
I'm not sure if I should mod this as offtopic or insightful, because it's somehow both at the same time.
Re: (Score:2)
No, seriously. I absolutely love 3D. There isn't a bigger 3D fanboy in the world.
And I reiterate: Don't do it.
2D converted to 3D is what is wrong with the current 3D. It's absolute garbage. It makes me want to pull my hair out.
Re: (Score:2)
well, it can be done. if you have an hour of time per 4 secs of video where you decide that some 3d effect works and is worthwhile.
some scenes you can do pretty ok automatic detection of what is background and what's foreground(by motion). but it'll still be more like a puppet show than what real 3d should/would be.
Re:Here's a tip (Score:5, Insightful)
Don't do it.
Well, there is a way to do it, a very elegant way even. One that can be, for all purposes and intents, as good as you can get with the raw material; even to the point where the average human will not be able to tell the difference.
The thing is: That solution has a big catch. How big? Well, to put it mildly, you will most likely win the Turing Award in the process of doing so and will at some point end up with a Nobel Prize in your hand, too. As you can imagine, the solution is: Artificial Intelligence; and if you want to really do it, only strong artificial intelligence will do.
The fact is, as others have quite succinctly pointed out, that the issue is in determining what is "in front" and what is "in the background" on top of how far away everything is. This is, quite simply, impossible to do right if you approach it as a purely algorithmic picture-to-picture problem. There is just not enough information inside the frames/movie to do it well enough even at the best of times.
So, what do you do? Easy, you import external information. Things like: "This is a tree; That is a human. A tree is bigger than a human. Both take up the same space in the picture. Assumption: The human is closer than the tree. Proof: The tree casts a shadow on the human and the only light source is behind the tree. Angles point to a distance of 20 meters between human and tree. Etc. pp."
This line of reasoning imports lots of information from the outside; essential things like "What does a tree/human look like?", and "What are their relations to each other size-wise?". But if you grant that this information can be derived and used by an AI, the result can be a very precise derivation of the distances between objects.
It is exactly the same line of reasoning the human brain uses for large distances (where the parallax of your eyes is too small, focus is unimportant and difference between eye positions negligible), or when you have lost vision in one eye (or just plainly covered it). Even though your brain suddenly has only half the information, it is capable of giving you a good feeling for distance and depth.
Of course, it doesn't always work, as far too many optical illusions like the Ames-Room show, but it works significantly better than a "pure" picture-to-picture approach and is the sole reason why almost everyone here feels that 2D-3D conversions are so horrible:
Their brain tells them, that what they see just can't be correct, even if their eyes have actually seen it.
But of course, just using 2 cameras is much simpler. So good luck with (strong) AI. I would be surprised if you solved this issue all by yourself. :)
Re: (Score:2)
No, if you do it that way you just end up with flat cardboard scenes with a bit of depth. If I can't observe parallax, say a slight rotation of someone's head, when I switch eyes, it's not real 3d.
Re: (Score:2)
No, if you do it that way you just end up with flat cardboard scenes with a bit of depth. If I can't observe parallax, say a slight rotation of someone's head, when I switch eyes, it's not real 3d.
That's why I said you need most likely strong AI to get it really perfect. You need to know not only the relations between things (like my example; a tree vs. a human), you also need to know the relations of things to themselves, i.e. the properties of a human face.
Imagine, that instead of lifting whole objects from the 2D-plane, you lift individual pixels, just like a modern computer game calculates the lighting, texturing and shadowing on a per-pixel basis to give you things like Normal mapping [wikipedia.org].
After all,
Re: (Score:2)
Re: (Score:2)
Not really as bad as you think. All it does is show frame n in one eye and frame n + 1 in the other, stretched (and cropped to preserve aspect ratio) a bit to exaggerate the depth. So things that do not move, they are assumed to be in the background, moving things seem to be closer. It's not as bad as you say, no resetting one key frames for example, but yes the effect is strange, often not right, as well as neat.
new generation, new suckers for '3d' (Score:3, Insightful)
we all were suckered. we tried it, hated it and moved on.
each time they try to re-invent this, its still just an effects gimmick.
you'll soon grow bored.
don't invest anything in this. its a reocurring cash grab due to industry boredom.
and as a fulltime glasses wearer, I'd never be caught dead with cardboard glasses over my regular ones. an absurd concept if there ever was one.
Re: (Score:3)
and as a fulltime glasses wearer, I'd never be caught dead with cardboard glasses over my regular ones. an absurd concept if there ever was one.
Word. Pay twice as much to wear ANOTHER pair of glasses and watch something that will more than likely give me a headache? Where do I sign up!
Until they have fully immersive holography, count me out.
Re: (Score:2)
Until they have fully immersive holography, count me out.
Ditto. I'm typically the sour grapes guy of the group that always resists seeing the newest 3-D blockbusters at the theater because I can't stand it, it looks like overly dim, out of focus crap and 9 times out of 10 I leave with a headache to boot.
That being said, there is only one film that I want to see in 3-D, and that is the Hobbit films. I've resigned myself to the headache, but in seeing the behind the scenes footage of how they're shooting the film, [youtube.com] I'm very interested to see how it compares to the
Re: (Score:2)
Sure, it's a gimmick. But the saturday night movie with my kids has a whole new level of "excitement" for them, especially when they have sleep over guests. That alone is well worth it. But my Nvidia GTX on my PC works with it, and playing Left4Dead 2 or Ba
Re: (Score:2)
It seems its decision is based upon movement. The more an object moves, the more it jumps out. It is logical, but you can imagine where that doesn't work.
That's what comes from basing their 3D algorithm on T-Rex vision.
Re: (Score:2)
Given the amount of studies and remarks about the potentially dangerous effect of 'fake 3D' on the brain/vision development of youngsters, I think I'll keep this technology away from my kids for as long as I can. Sure, the occasional movie or fun-park attraction is OK, but having this in the living-room and/or gaming-computer(console) simply is asking for day-to-day use.
Might well be the effects are largely overrated I prefer to play on the safe side with this. It's a fun gimmick and it does have a 3D-ish e
Re: (Score:2)
You could just buy another pair of glasses with coloured glass. More classy than cardboard anyway.
Re: (Score:2)
When I wore glasses (had lasik) and I did research in 3D (like 20 years ago) I actually had a pair of flip ups that attached to my regular glasses.
Not that I'd recommend that for anybody that didn't want to look like a complete dork, but it was convenient for me at the time.
Re: (Score:2)
Re: (Score:2)
you're wrong. a movie is mostly in the mind (and pure audio; ie, your cd collection, is even more so like that). imagination and the brain's ability to connect dots and extrapolate does more for the entertainment than your tech tricks.
I'm an audio guy and spend a lot on design an implementation but I realize that when a tune comes on the radio, my mind gets the same memory enjoyment from it as if it was on a high end system. the content and my extrapolation of it (on poor playback equip) completes the ex
Re: (Score:2)
Just because you cannot enjoy it doesn't mean I can't. Personally, I don't enjoy dubstep, but I don't go around making out like "dubstep sucks" is a matter of fact, when it is a matter of opinion.
Re: (Score:1)
http://xkcd.com/941/ [xkcd.com] FTFY
This sounds familliar (Score:5, Funny)
I'm interested in converting 2D video to Stereoscopic 3D video
George Lucas, is that you?
Special FX (Score:5, Informative)
It sucks, it's mostly manual, get over it.
Re: (Score:2)
Re: (Score:2)
In theory you can expand the image to 3d by clipping each object and vertically slicing it into layers (for a tree, or horizontally for a bench) then adding 3D effect to the layers, filling in where there is gaps and overlaying where needed, for each object in each frame of the film, then composit all the objects and masks back on to the scene. Now that you've spent ~100 hours on that your first frame is done, time to do the next, but now it's harder because you need to account for motion so your clipping
Re: (Score:2)
Waitaminute... (Score:3)
Dammit, leave the original trilogy alone! The digital "remaster" was insulting enough!
Re: (Score:1)
Red/Cyan glasses (Score:1)
I was personally very surprised at how well red/cyan works. Of course the colors get a little muddle, but not as mu
You're missing critical information (Score:1)
You can't turn a two-dimensional photograph into 3D because the original has lost all the phase information that conveys needed info (e.g., "depth"). Similarly, you can't restore 2D sound to 3D, because the essential information isn't in the source recording that you'd need to "position" all the sound sources in 3D. In general, you can go from (N+1) to (N) dimensions, but you lose information. That means you can not automatically go from (N-1) dimensions to (N) without restoring that lost information...w
Re: (Score:3)
With the use of video in such a case, the depth information can be pretty accurately inferred from the parallax effect, due to the fact that your car (and camera) are moving along the road. It's a difficult problem, but by comparing frame with frame, an algorithm might piece a somew
Re: (Score:2)
Re: (Score:1)
As the edges and shading change, this reveals the 3D structure.
Even in a single frame, inference of object depth/texture is possible by application of [inverse] shading models.
That said, it needs a bit of a render farm
Re: (Score:3)
However, the easiest way is to just shoot the thing in 3D in the first place.
Re: (Score:2)
Let's say you have a video camera poked out of the side window of your car, and you're driving down a road alongside a wide field. The field is sparsely populated with trees, and there are mountains far off in the background. With the use of video in such a case, the depth information can be pretty accurately inferred from the parallax effect, due to the fact that your car (and camera) are moving along the road.
This is sort of how the Pulfrich effect [wikipedia.org] works anyway. You have a camera moving sideways while pointed at a relatively static scene. Any two frames a moderate fraction of a second apart will thus effectively have been taken from slightly different parallax positions, like twin stereoscopic 3D images.
The key is when watching this film on an ordinary 2D television you view it through a pair of glasses with one lens darker than the other. Due to the way the brain processes images, the eye viewing through the
Re: (Score:2)
"Arduino sucks" (Score:3)
Contra my provocative subject, Arduino is an excellent choice for serious hobbyists. And similarly, there is nothing wrong with playing around with 3D video techniques and even being willing to try rolling one's own algorithm.
Get a (homebrew friendly) life, slashdotters!
(If the OP clarifies that he's working on a big Hollywood title, I'll take this back. Until then...)
Creating "3d" (Score:5, Interesting)
Any motion picture where the camera pans side to side gives an opportunity to create a "3d" image. If an object moves across a still camera, you can also derive 3d information. (Also if it spins)
An interesting exercise would be to process a film, and make stereoscopic only what what can be done properly, and leave the rest flat. A scene would start out flat, then people and things would begin to jump out at you.
Re: (Score:2)
Also (while I seriously doubt this applies to the OP), the world of 3D will be an interesting place when the CV and AI academic communities start recognizing a bunch more objects and create the ability to much more accurately annotate and infer 3D within a scene. Currently 3D can be added by a difficult manual process, which is certainly too time intensive to do thoroughly and well for anything movie-length, hence the annoyingly partial jobs we've seen up to now. We haven't yet seen the theoretical
Re: (Score:3)
Re: (Score:2)
But that's how stereographic photos work in general, whether one is using one camera or two (or more). This wasn't a new technique.
I think the real trick were those 3-D system they smuggled in, where they could pan around the area in virtual 3-D that let them see the V-2 rockets as they were on their launch pads and get a feel [sic] for them.
this can be done easily with ffmpeg and imagemagic (Score:5, Informative)
Re: (Score:2)
THIS. Somebody mod parent up please.
Re: (Score:1)
You dont simply want to filter the channels red vs green/blue. That creates terrible ghosting. Instead look up Dubois alforithm, its a linear projection from 6d colorspace to 3d colorspace, optimized for minimal ghosting using MSE. Finished matrices exist fro both red/cyan, green/magenta and amber/blue, available from Dubois homepage. Recently used this for a project, works great.
3D trickery... (Score:1)
my blueray player can. (Score:1)
My blueray player can simulate 3D from any 2D source (Panasonic DMP-BDT210) although I'm not exactly sure how it does it, or how good it looks. (no 3D tv) You might be able to talk someone into connecting one up at your favorite bigbox store for you if you acted interested in buying the blueray player, and wanted a demo of its conversion capabilities. This would at least give you a firsthand idea of how it will look to see if YOU think its worth it.
imagemagick (Score:2)
The convert utility in the imagemagick package does a good job of it with still images. I'd consider dumping your frames out as a series of images, running the convert utility on them, and then re-creating your movie.
I've also thought that taking that code in convert, merging it into VLC, and setting up VLC to grab from 2 cameras at once... with enough CPU and RAM, it could be come very close to real time 3d movie.
Luminosity (Score:1)
How to convert your 2D display into 3D (Score:3)
1. Display 2d images on a flat panel tv facing you
2. spin the display 45 degrees so that one edge is nearer to you the other edge
3. That's it --notice how pixels on one side are closer to you when the ones on the opposite edge are futher away from u spetially)you display is in 3D now.
More likely to make people sick (Score:1)
Have to love Ask Slashdot... (Score:2)
Standard Template:
I want to do _something_, but I do not know how to do _something_, how do I do _something_, provided I don't know how to or do not want to waste my time using Google.
Then a barrage of responses by people that don't really know how to do _something_, but surprisingly have a lot of opinion about _something_.
And then, of course, a smart *ss like myself pointing this out.
Re: (Score:2)
2D to 3D Algorithms (Score:2)
In a few words: if you only have a 2D video, then it is a very hard computer vision problem [wikipedia.org], that has not been solved on the research side.
There is an active benchmark [middlebury.edu] of disparity estimation algorithms (full bibliography at the end of the page). Those algorithms take two pictures and estimate a depth image. From this depth image, it is possible to reconstruct the scene in 3D (but you cannot see what's beh
Re: (Score:2)
Algorithm? Um, no. (Score:1)
Despite what some PR hustling excitables might claim, stereoscopic conversion cannot be effectively automated at this time. Do people try it? Yes. Does it generate watchable results? Sometimes by accident, yes.
The thing is, a stereoscopic conversion done painstakingly frame by frame by a highly skilled compositing artist looks pretty bad. Any automated conversion process will be orders of magnitude worse.
What you need is a ton of really excellent rotoscoping (I send my jobs out to work farms in Russia) to s
1D to 2D (Score:1)
Monoprice 2D to 3D HDTV/DLP Converter (Score:2)
You cant. (Score:2)
You can't just run a 2D video through an algorithm and magically get a 3D video.
You have to run the video through a compositing program (think Photoshop for video) and use that to chop and mask each scene and introduce parallax effects. Then (if your compositing program supports 3D space) you output the streams from two different virtual cameras so that you have 2 final videos that are synced and are from two different angles (one for each eye). At that point, it's trivial to encode them to whichever 3D vid
Re: (Score:2)
Also, if you really think you're up to the task of writing an algorithm, the place to start is reading up on all of the various SIGGRAPH research papers on image composition analysis and video processing.
Re: (Score:1)
realtime conversion? no such thing (Score:2)
This is basically impossible, or will have horrible artifacts.
The current crop of movies with 2D-to-3D conversions still took significant human and artistic effort to achieve, even though the results are mediocre. For a given frame, for every pixel in 2D, SOMETHING has to decide how far away the subject depicted must be. That is, it has to INVENT the third dimensional value. Then this value is used to calculate two new 2D frame with parallax involved.
There's no co
3DTV.com (Score:1)
I friend of mine (former CEO of a startup I founded) asked me to write one.
He called and kept offering more each time. I actually spent some time investigating this and decided that it was a good way to give my self a stroke.
It's hard enough implementing and getting things right when you know what to do, with 2D to 3D there isn't even a clear algorithmic method to use, few papers and no examples of a good automated conversion. DDD seems about the best.
I must admit I've seen some decent human with software a
Re: (Score:2)
Are you kidding, someone moderated this as Troll !!!
I certainly never intended to be negative about anyone or thing, just that it's a hard problem.
DVDFab? (Score:1)
You can try DVDFab from Fentao and see if that works for you: http://www.dvdfab.com/
The only viable algorithm is called "interns" (Score:2)
I work in post-production, and while some of the stereo-handling algorithms are impressive from a technical point of view (like the stuff in Eyeon Dimension and The Foundry's Ocula), and while I think stereo 3D is here to stay for video games (at least after consoles add some improvements to head tracking), I doubt it will be more than a passing fad for movies. It's simply not compatible enough with human vision, even when done properly (head movements spoil the effect, the difference between convergence po
Maybe possible... (Score:1)
For the best results... (Score:2)
I write post-production software used to do this (and it runs on Linux!). The best results I've seen involve manually breaking each shot into dozens of layers, using rotoscoping. Each set of layers is exported as masks and imported into a compositing application where the images for the layers are projected onto the masks in 3D space. In some cases they build rough 3D models and project the layers onto the respective models. Now they can add a virtual camera and render the scene from both views. Then they b
YouTube has a 3d tool (Score:1)
Gimp (Score:1)
Bundler + PMVS (Score:2)
OpenCV (Score:2)
If you look on Google under "OpenCV stereo vision" you will find links showing how the code runs. There are video examples using two web cams that run in real time at around 5 frames per second. If you record and run off line you can get reasonable playback frame rates.
This code generates a depth map for the scene, so each pixel is assigned a distance from the came
Re: (Score:2)
The company DDD has built hardware to do this; it "works", after a fashion. It is, indeed, incorporated into a number of recent 3D TVs.
Basically, there are a number of algorithms in the box, and it chooses the one that is most appropriate for a given sequence. If the system sees blue in the top of the frame, it assumes that it is sky, and puts it in the back. If the camera is trucking from one side to the other to generate parallax, it uses that to generate depth. If I recall correctly, there are some 2
Re: (Score:1)