Ask Slashdot: Synchronizing Sound With Video, Using Open Source? 103
An anonymous reader writes: I have a decent video camera, but it lacks a terminal for using an external mic. However, I have a comparatively good audio recorder. What I'd like to do is "automagically" synchronize sound recorded on the audio recorder with video taken on the video camera, using Free / Open Source software on Linux, so I can dump in the files from each, hit "Go," and in the end I get my video, synched with the separately recorded audio, in some sane file format. This seems simple, but maybe it isn't: the 800-pound gorilla in the room is PluralEyes, which evidently lots of people pay $200 for --and which doesn't have a Linux version. Partly this is that I'm cheap, partly it's that I like open source software for being open source, and partly it's that I already use Linux as my usual desktop, and resent needing to switch OS to do what seems intuitively to be a simple task. (It seems like something VLC would do, considering its Swiss-Army-Knife approach, but after pulling down all the menus I could find, I don't think that's the case.) I don't see this feature in any of the Open Source video editing programs, so as a fallback question for anyone who's using LiVES, KDEnlive, or other free/Free option, do you have a useful workflow for synching up externally recorded sound? I'd be happy even to find a simple solution that's merely gratis rather than Free, as long as it runs on Ubuntu.
A Clapboard (Score:4, Informative)
What you want is a simple Hollywood style clapboard. Use any two track + video editor and visually line up the audio spike to the video frame where the board is closed.
Re:A Clapboard (Score:5, Informative)
However, if you want something very polshed and fancy and automatic....take a look at Davinci Resolve 12 Beta [blackmagicdesign.com].
This is the free version that contains about 98% of the $1K paid version...what's missing likely you won't miss either....it is now a full blow NLE that compete with Premier and FCPX and others...
You can do some amazing sync'ing of multi-clip video and sound with this thing..take a look at the demos on site.
Unfortunatley, the Linux version is only available for Paid version, but hell, who doesn't have a mac or windows box laying around? I mean, if you have money for a decent video set up, you've already poured out some decent cash (camera, lights, software....hell, lenses ALONE)....surely you have laying around a win/osx box or can buy one.
But, do give Davinci Resolve 12 a look...they also have out Fusion [blackmagicdesign.com], which is IMHO, going to be a competitor to Adobe After Effects too for SPFX....worth looking into too.
Re: (Score:2)
While I do have a good video camera that will accept a microphone, more than 90% of the time I use a separate device for audio recording. It is very rare that a microphone attached to the camera can be optimally positioned. A separate audio recorder is the answer to many vexing problems.
I don't have a clapper board and if I did, I usually would still not have an assistant around to do the "take 3 (snap)" bit. But asking your subject to hold up his hands in the camera's view and clap smartly works. They als
Re: (Score:2)
What about using a wireless microphone setup and running it through the camera's inputs? You'd still get the benefits of a separate device without cords to trip over, but all recording would be done on one device, with one clock. I can only imagine this becoming a problem if you seek to keep every microphone on a separate track, rather than a real-time mix down to stereo or mono.
Re: (Score:2)
A wireless mic would solve a lot of problems.
I probably will not buy a wireless mic since I've got a method that works. Also there are occasions when I want the sound track of interview to continue while I splice in a different video clip, and I think that would be more difficult if I was recording the audio on the camera.
Re: (Score:2)
A wireless mic would solve a lot of problems.
I probably will not buy a wireless mic since I've got a method that works. Also there are occasions when I want the sound track of interview to continue while I splice in a different video clip, and I think that would be more difficult if I was recording the audio on the camera.
Are you planning on doing your editing IN the camera?!? If not, what you say makes no sense.
Just use a NLE suite (I know they all suck on Linux; but, that's what you want), and if you can't "eyeball" the sync (your brain is QUITE sensitive to even small video/audio sync mismatches; so you SHOULD be able to pretty quickly get to an acceptable result), then do what another poster said, and use a "clapboard". Or just frickin' have someone stand in front of the damned camera (fairly closely) and clap their da
Re: (Score:2)
I've been trying without success to make sense of the post I'm replying to. My best guess is that the poster is unaware that he does not know as much about the tools available in Linux as he thinks he does. And that his reading comprehension was, for some reason, not very good when he was looking at earlier posts.
Blender runs on Linux and contains a very good NLE (and a fantastic suite of CG tools as well). Audio clips brought into Blender's NLE can be displayed as waveforms, which makes synching the spike
Re: (Score:2)
While I do have a good video camera that will accept a microphone, more than 90% of the time I use a separate device for audio recording. It is very rare that a microphone attached to the camera can be optimally positioned. A separate audio recorder is the answer to many vexing problems.
Actually, if you have seen any of the pro-level portable cameras, most of them have a separate wireless transmitter module that can accept a balanced mic/line level through an XLR, and usually a 1/4" (TRS-balanced?) input as well. This gets transmitted back to the camera, in lieu of the built-in mic signal.
Re: (Score:1)
There may also be a need to adjust the speed of one track. I recorded a few songs on a concert with video from a compact digicam and audio from my smartphone (because the camera only records audio at 16kHz). When lining up the start of a song, at the end it was asynchronous by a few seconds. Audio was iirc 0.7% faster or slower than video, and this was the same for all recordings. I resampled the audio using Audacity before merging it with some video editor.
Clap Clap Clap (Score:4, Informative)
Sorry but you are screwed (Score:2, Informative)
Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.
So you either have to have something that can do an advanced auto-sync and make sure the sync gets corrected in multiple places, or you'll need to do it manually. Depending on how long the recording is that may not be too bad and you may not need to
Re: (Score:1)
Shh, he's still splicing things together with a reel-to-reel tape recorder, obviously.
Re: (Score:3, Informative)
He is correct. AV is my job, and I do a lot of it on Linux.
My multi-track audio recorder and my video claim to run at the same speed - but there is variance between any two clock sources that are not linked.
I use Harrison Mixbus (an excellent Linux/Mac/Windows DAW) and XJAdeo, both slaved to the Jack clock. You can shuttle audio and the video tracks your position according to the audio elapsed time.
The two clocks drift apart very slowly. For the purposes of syncing live music footage, you only need to make
Re: (Score:2)
audio/video drift sync is totally machine correctable. http://forum.videohelp.com/threads/49752-Fixing-Audio-Drift-Without-Postprocessing [videohelp.com]
Re: (Score:2)
Re: (Score:1)
Yes sync in a longer take would be an issue, but I doubt his equipment has a clock or genlock input. So without getting into the vagaries of wordclock, blackburst, or AES11 I suggested a simple clapboard. It'll do the trick well enough for what hes doing. He could also run LTC into a spare audio track on both recorders, but I doubt he has a SMPTE generator or software to lock the two tracks together. Besides, modern digital recorders (both audio and video) have pretty stable clocks nowadays. I mean, it coul
Re: (Score:1)
That sounds pretty buzzword compliant.
Re: (Score:2)
This works well with speech, where the audio frequency shift is minor and much less important than the lip synch. I'm not sure how well it would work with music. As I think someone else has already mentioned, you might have to resample the audio track with Audacity to get a singer's lip synch right without screwing up their pitch.
Re: (Score:2)
I'm not sure how well it would work with music.
Most time stretching (or compressing) these days can do an excellent job of minor adjustments without effecting pitch or formant.
Re: (Score:3)
Sorry but you are screwed. Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.
Uh, the fact that he links to a piece of proprietary software which can do it in the summary should be a pretty good hint that he's not screwed. He might not be able to do it on Linux, but clearly you know even less about it than the submitter.
Re: (Score:2)
Uh the fact that I mention that in my post (Score:2)
should be a pretty good hint that you didn't read it and just responded to the title.
He's screwed for what he wants: No cost Linux software that does auto-sync. There is none.
Re: (Score:3)
Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.
Unless it is a rather long edit, they'll stay in sync just fine. The clapboard method isn't automagic, but unless the recording has individual edits that are going to be 20 minutes or longer, its going to be an exercise in wasted money. I've even used background sounds in videos and audios to sync together. Not at all recommended, because its a pain in the ass, but remarkably effective.
Avidemux (Score:1)
I haven't used it in a long time, but would Avidemux [fixounet.free.fr] do the trick?
Avidemux (Score:1)
FFmpeg (Score:1)
I think FFMPeg can mux audio and video from separate files. If not, search for muxing tools, and you'll have your answer. Of course, bear in mind that sample rates can drift. so the longer the video, the less likely this is to work without resampling. And you'll need to edit the start points to be precisely the same.
Open Broadcaster Software (Score:5, Informative)
Blender Video Sequence Editor (Score:1)
The video editor / compositor in Blender is quite nice. I'm sure with a little setup in Blender (and using a ClapperBoard [wikipedia.org] at the start of every one of your videos) you could automate this process down to just having to drag an audio clip back and forth to match a spike in the audio with a frame of video.
If you REALLY have to do a LOT of videos and don't want to take the time to do it manually, then maybe you should think about dropping the $200 for PluralEyes. I have never seen or heard of any free / open
Re: (Score:2)
Seconded for Blender.
I do my (admittedly, not much) video editing in Blender.
Blender's interface is weird, but there's a tutorial series on Youtube about using the video editor for non-Blender users by a guy going by "Mikeycal Meyers." There's an old series and a new series - just watch the new one.
Plus, if you know anything about 3D, you can make animations and whatnot. Blender's almost a whole-pipeline-in-a-box type tool - Blender, an audio program, and Photoshop/Gimp are enough to make an animated movi
Sync to the audio (Score:5, Informative)
Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.
Re: (Score:3)
Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.
I don't do video stuff, but I had the same thought. Record crappy audio in the camera and it would be a simple correlation algorithm to sync to the good audio.
Re: (Score:2)
It will sync up the crappy audio from the camera with the fancy audio from the recorder (and also can sync multiple video streams based on their audio).
His problem is that it costs $200 and doesn't run on linux. Also, probably that he has a single-camera setup and thus the features that sync video frames are not really necessary for him.
He isn't asking for some magical software that lip reads his video and sy
Re: (Score:2)
I read the post. I didn't get the mechanism of the $200 software. I figured it was being clever and matching on-screen movements to sound or something.
The logical thing to do is to start programming a solution if you don't like the $200 solution and you don't like the free options.
Re:Sync to the audio (Score:5, Informative)
The above is totally correct, I use a separate recorder all the time so that I get better quality audio than my video recorder will supply, and then use Kdenlive to align the audio to the camera's reference audio.
I've actually made a video that includes how to do it (among other features), and the video itself uses the Align Audio to Reference feature:
https://www.youtube.com/watch?v=eIO73t228k0
Here's a short text version:
https://userbase.kde.org/Kdenlive/Manual/Timeline/Right_Click_Menu
Re: (Score:2)
Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.
You can get 1 video frame off accuracy this way. I did the old clapboard method a gazillion times with no discernable sync issue. Keep the clapboard at the images point so that an echo won't give you an issue, but even that is overkill for most applications.
If I was going to get at all fancy with it, I'd build a clapboard with an led on it that would light the moment the clap noise happened, to get a visual reality check. But that's even overkill.
Re: (Score:2)
You can get 1 video frame off accuracy this way.
Why?
Also bear in mind you'll be 1 frame off sync for every 11 metres your clapper is from the camera...
I think this is why humans notice less when sound is delayed than when it is in advance - because that's the normal way of things.
Re: (Score:2)
You can get 1 video frame off accuracy this way.
Why?
Also bear in mind you'll be 1 frame off sync for every 11 metres your clapper is from the camera...
I think this is why humans notice less when sound is delayed than when it is in advance - because that's the normal way of things.
Another autocorrect or keyboard bounce attack 1 video frame of accuracy this way was what I tried to type.
Yes, the speed of sound and pickup placement is important if we are trying to be as accurate as the video will allow us to be.
Re: (Score:2)
I haven't done the specific thing he's suggesting, but I've had PVR recordings that were out of sync. I just used one of mencoder's 27 million options and used trial and error to get the right input value.
mencoder (Score:5, Informative)
A/V sync command line (Score:5, Informative)
https://github.com/allisonnico... [github.com]
OpenShot (Score:2)
Re: (Score:2)
If you want the software to "automagically" do its job, it is indeed not a simple task. You'd need to correlate what you hear with what you see, this is generally hard. But if you let the user to the "hard work" and associate e.g. the image of clapping hands with the clap sound, or other visual occurences with their audio counterpart, the software only has to calculate an interpolation, with the easiest setup being splines.
I'm not sure about doing it live, but... (Score:4, Informative)
I use avidemux [fixounet.free.fr] and audacity [audacityteam.org] to add Rifftrax [rifftrax.com] to movies. Basically copying out the existing audio, merging the rifftrack with the movie audio in to one track and then putting it back in as a separate audio track so the original audio is there too if you want to watch the movie without jokes (pretty rare for me actually, but it's nice to have options ;)).
Someone else already suggested OBS, I've used that too for recording gaming video. It can be pretty intensive and it takes some fiddling to make it workable but it's not bad.
Oh I forgot (Score:2)
If you're doing this for surveillance, I use iSpy [ispyconnect.com] you can define a range of your camera that if it detects motion will start recording and you can choose how far in advance the buffer is and how long it waits before it stops. It can be triggered by audio too, I've found it to be very nice for watching maintenance at my apartment and people trying to break in to our cars.
They do charge for remote access features but you can "roll your own" for that if you need it (I didn't). It can use an external mic as a s
Re: (Score:2)
The FOSS program motion [lavrsen.dk] works well for this too. I have it set to contact me when it detects movement, along with a few frames of the captured video.
Kdenlive works reasonably well (Score:2)
Open Source Syncing of Video with Audio (Score:1)
Clap, tap (Score:2)
I can't help with the automagic part, but...
I capture with this method too, except when working with pre-recorded studio performances. I highly recommend rolling the audio and video capture device for a few seconds before "action" to give you time to clap or tap an object in front of the camera, as close to the sound source (not necessarily the microphone) as is practical. you will then have a definitive reference point in your audio and video streams against which you can synchronize in your editor of ch
A Different Persepective (Score:2)
Have you thought about modding the device to add a mic input? If you check the web, someone might have already done it, or the schematic might be available, or it might be obvious if you take it apart.
Cinelerra (Score:1)
I'm no expert at video editing, but did you look at Cinelerra [heroinewarrior.com] ? It exists for years (I mean, before VLC existed, before mplayer, even before Xine if I remember correctly... Who remembers aviplay ?) and apparently it is still (kind of) maintained.
For Debian it's available on Christian Marillat's multimedia repository.
reaper under wine? (Score:1)
VDub (Score:2)
If you haven't tried VirtualDub you need to go do so. That nobody here even speaks of it is mounting testament to the declining mental health of this community.
Re: (Score:3)
That nobody here even speaks of it is mounting testament to the declining mental health of this community.
The fact that we successfully read TFS and so aren't listing software that doesn't work on Linux is in fact a testament to our mental health.
Don't be a little bitch (Score:3)
Don't be a dick (Score:1)
Could you not have just answered the question like a normal person instead of being a douchebag about it? Or were you worried you weren't going to feel superior to anyone today?
Glad the sync problem isn't just me (Score:2)
Clocks drift at varying rates (Score:2)
Different device clocks drift at different rates, mostly due to heat. Oven-controlled crystal oscillators [wikipedia.org] are employed in more expensive devices to provide more reliable clocking.
Sources that are simply synchronized at the start will typically quickly drift and lose sync. Unfortuantely the drift is often not linear. So if you sync them at the start, and then stretch or shrink one source via re-sampling, they may sync at the start and end, but not in the middle.
A good solution for the OP's problem would comp
Re: (Score:1)
Re: (Score:2)
There's a reason so many are "stuck" using Windows, no matter how onerous Microsoft gets (or OSX, no matter how walled garden Apple is) - some industry standard software simply doesn't run on other systems. Compared to the cost of someone to do the work for you, buying a dedicated Windows machine and the software to run it is less than a single edit session.
Or, as they say, use a clapperboard and do it yourself.
Re: (Score:2)
There's a reason so many are "stuck" using Windows, no matter how onerous Microsoft gets (or OSX, no matter how walled garden Apple is) - some industry standard software simply doesn't run on other systems. Compared to the cost of someone to do the work for you, buying a dedicated Windows machine and the software to run it is less than a single edit session.
Or, as they say, use a clapperboard and do it yourself.
Just one of the big reasons I use OSX. Even iMovie can make a surprisingly professional movie. A lot of times when under insane time pressure, I used it instead of FCP.
And I used a clapboard. Quick and efficient. And it works.
caveat: if I were to be making a long single cut video, I'd find a professional tool to keep everything in sync. But that never happened.
Re: (Score:1)