Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
Open Source Software Input Devices Linux

Ask Slashdot: Synchronizing Sound With Video, Using Open Source? 103

An anonymous reader writes: I have a decent video camera, but it lacks a terminal for using an external mic. However, I have a comparatively good audio recorder. What I'd like to do is "automagically" synchronize sound recorded on the audio recorder with video taken on the video camera, using Free / Open Source software on Linux, so I can dump in the files from each, hit "Go," and in the end I get my video, synched with the separately recorded audio, in some sane file format. This seems simple, but maybe it isn't: the 800-pound gorilla in the room is PluralEyes, which evidently lots of people pay $200 for --and which doesn't have a Linux version. Partly this is that I'm cheap, partly it's that I like open source software for being open source, and partly it's that I already use Linux as my usual desktop, and resent needing to switch OS to do what seems intuitively to be a simple task. (It seems like something VLC would do, considering its Swiss-Army-Knife approach, but after pulling down all the menus I could find, I don't think that's the case.) I don't see this feature in any of the Open Source video editing programs, so as a fallback question for anyone who's using LiVES, KDEnlive, or other free/Free option, do you have a useful workflow for synching up externally recorded sound? I'd be happy even to find a simple solution that's merely gratis rather than Free, as long as it runs on Ubuntu.
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Synchronizing Sound With Video, Using Open Source?

Comments Filter:
  • A Clapboard (Score:4, Informative)

    by Anonymous Coward on Thursday September 10, 2015 @06:29PM (#50499235)

    What you want is a simple Hollywood style clapboard. Use any two track + video editor and visually line up the audio spike to the video frame where the board is closed.

    • Re:A Clapboard (Score:5, Informative)

      by cayenne8 ( 626475 ) on Thursday September 10, 2015 @08:08PM (#50499821) Homepage Journal
      Yep, the good old fashioned way of doing this is looking for the "CLAP" on the waveform and lining it up with the video.

      However, if you want something very polshed and fancy and automatic....take a look at Davinci Resolve 12 Beta [blackmagicdesign.com].

      This is the free version that contains about 98% of the $1K paid version...what's missing likely you won't miss either....it is now a full blow NLE that compete with Premier and FCPX and others...

      You can do some amazing sync'ing of multi-clip video and sound with this thing..take a look at the demos on site.

      Unfortunatley, the Linux version is only available for Paid version, but hell, who doesn't have a mac or windows box laying around? I mean, if you have money for a decent video set up, you've already poured out some decent cash (camera, lights, software....hell, lenses ALONE)....surely you have laying around a win/osx box or can buy one.

      But, do give Davinci Resolve 12 a look...they also have out Fusion [blackmagicdesign.com], which is IMHO, going to be a competitor to Adobe After Effects too for SPFX....worth looking into too.

    • by hvdh ( 1447205 )

      There may also be a need to adjust the speed of one track. I recorded a few songs on a concert with video from a compact digicam and audio from my smartphone (because the camera only records audio at 16kHz). When lining up the start of a song, at the end it was asynchronous by a few seconds. Audio was iirc 0.7% faster or slower than video, and this was the same for all recordings. I resampled the audio using Audacity before merging it with some video editor.

  • Clap Clap Clap (Score:4, Informative)

    by Lars Kumbier ( 4176113 ) on Thursday September 10, 2015 @06:31PM (#50499247)
    That is what the movie claps you see in documentaries are for. You produce a clap with your hand, resulting in a spike in your external mic and in the video cameras mic. That way, you know where to sync the two in your favorite video editing tool and you're fine. No need for special software or something similar.
  • Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.

    So you either have to have something that can do an advanced auto-sync and make sure the sync gets corrected in multiple places, or you'll need to do it manually. Depending on how long the recording is that may not be too bad and you may not need to

    • by Anonymous Coward

      Yes sync in a longer take would be an issue, but I doubt his equipment has a clock or genlock input. So without getting into the vagaries of wordclock, blackburst, or AES11 I suggested a simple clapboard. It'll do the trick well enough for what hes doing. He could also run LTC into a spare audio track on both recorders, but I doubt he has a SMPTE generator or software to lock the two tracks together. Besides, modern digital recorders (both audio and video) have pretty stable clocks nowadays. I mean, it coul

      • That sounds pretty buzzword compliant.

    • by Kjella ( 173770 )

      Sorry but you are screwed. Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.

      Uh, the fact that he links to a piece of proprietary software which can do it in the summary should be a pretty good hint that he's not screwed. He might not be able to do it on Linux, but clearly you know even less about it than the submitter.

    • Unless the devices themselves have some kind of common sync like wordclock or the like, they will drift out of sync. So sync the audio at any given point, it'll be out of sync later. That's why studios have all kind of gear to slave everything to a master clock.

      Unless it is a rather long edit, they'll stay in sync just fine. The clapboard method isn't automagic, but unless the recording has individual edits that are going to be 20 minutes or longer, its going to be an exercise in wasted money. I've even used background sounds in videos and audios to sync together. Not at all recommended, because its a pain in the ass, but remarkably effective.

  • by Anonymous Coward

    I haven't used it in a long time, but would Avidemux [fixounet.free.fr] do the trick?

  • See Subject. It's not completely automagic, but it can probably do what you want.
  • by Anonymous Coward

    I think FFMPeg can mux audio and video from separate files. If not, search for muxing tools, and you'll have your answer. Of course, bear in mind that sample rates can drift. so the longer the video, the less likely this is to work without resampling. And you'll need to edit the start points to be precisely the same.

  • by JohnnyComeLately ( 725958 ) on Thursday September 10, 2015 @06:34PM (#50499273) Homepage Journal
    Your issue is very similar to what Twitch streamers go through with delay between audio and video. I'd suggest checking out OBS [obsproject.com] and there are quite a few how-to videos on YouTube to show you how to sync.
  • by Anonymous Coward

    The video editor / compositor in Blender is quite nice. I'm sure with a little setup in Blender (and using a ClapperBoard [wikipedia.org] at the start of every one of your videos) you could automate this process down to just having to drag an audio clip back and forth to match a spike in the audio with a frame of video.

    If you REALLY have to do a LOT of videos and don't want to take the time to do it manually, then maybe you should think about dropping the $200 for PluralEyes. I have never seen or heard of any free / open

    • by spauldo ( 118058 )

      Seconded for Blender.

      I do my (admittedly, not much) video editing in Blender.

      Blender's interface is weird, but there's a tutorial series on Youtube about using the video editor for non-Blender users by a guy going by "Mikeycal Meyers." There's an old series and a new series - just watch the new one.

      Plus, if you know anything about 3D, you can make animations and whatnot. Blender's almost a whole-pipeline-in-a-box type tool - Blender, an audio program, and Photoshop/Gimp are enough to make an animated movi

  • Sync to the audio (Score:5, Informative)

    by linuxwrangler ( 582055 ) on Thursday September 10, 2015 @06:42PM (#50499333)

    Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.

    • Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.

      I don't do video stuff, but I had the same thought. Record crappy audio in the camera and it would be a simple correlation algorithm to sync to the good audio.

      • Did you guys even read the post? That is exactly how the $200 software he mentions does it.

        It will sync up the crappy audio from the camera with the fancy audio from the recorder (and also can sync multiple video streams based on their audio).

        His problem is that it costs $200 and doesn't run on linux. Also, probably that he has a single-camera setup and thus the features that sync video frames are not really necessary for him.

        He isn't asking for some magical software that lip reads his video and sy

        • I read the post. I didn't get the mechanism of the $200 software. I figured it was being clever and matching on-screen movements to sound or something.

          The logical thing to do is to start programming a solution if you don't like the $200 solution and you don't like the free options.

    • Re:Sync to the audio (Score:5, Informative)

      by Fone626 ( 6793 ) on Thursday September 10, 2015 @06:58PM (#50499435)

      The above is totally correct, I use a separate recorder all the time so that I get better quality audio than my video recorder will supply, and then use Kdenlive to align the audio to the camera's reference audio.

      I've actually made a video that includes how to do it (among other features), and the video itself uses the Align Audio to Reference feature:
      https://www.youtube.com/watch?v=eIO73t228k0

      Here's a short text version:
      https://userbase.kde.org/Kdenlive/Manual/Timeline/Right_Click_Menu

    • Don't sync to the video, sync to the audio. Your video recorder records the audio "reference" track and you just sync your externally recorded audio to the reference track. Kdenlive has this feature. Other editors may a well.

      You can get 1 video frame off accuracy this way. I did the old clapboard method a gazillion times with no discernable sync issue. Keep the clapboard at the images point so that an echo won't give you an issue, but even that is overkill for most applications.

      If I was going to get at all fancy with it, I'd build a clapboard with an led on it that would light the moment the clap noise happened, to get a visual reality check. But that's even overkill.

      • You can get 1 video frame off accuracy this way.

        Why?

        Also bear in mind you'll be 1 frame off sync for every 11 metres your clapper is from the camera...

        I think this is why humans notice less when sound is delayed than when it is in advance - because that's the normal way of things.

        • You can get 1 video frame off accuracy this way.

          Why?

          Also bear in mind you'll be 1 frame off sync for every 11 metres your clapper is from the camera...

          I think this is why humans notice less when sound is delayed than when it is in advance - because that's the normal way of things.

          Another autocorrect or keyboard bounce attack 1 video frame of accuracy this way was what I tried to type.

          Yes, the speed of sound and pickup placement is important if we are trying to be as accurate as the video will allow us to be.

        • I haven't done the specific thing he's suggesting, but I've had PVR recordings that were out of sync. I just used one of mencoder's 27 million options and used trial and error to get the right input value.

  • mencoder (Score:5, Informative)

    by carlhaagen ( 1021273 ) on Thursday September 10, 2015 @06:42PM (#50499345)
    mencoder will do exactly what you need, muxing separate video and audio tracks into one container file of your choice, with tunable offsets for each track.
  • by mungewell ( 149275 ) on Thursday September 10, 2015 @07:08PM (#50499493)
    I had a recent project which needed to sync Audio from recorder with video from several cameras. I found this project on GitHub which has a command line tool to measure the delta between the high quality audio from recorder and the low quality from the cameras, and then I could just put this offset in my video editor when inserting the clips.

    https://github.com/allisonnico... [github.com]

  • I've done this with OpenShot. It's fairly easy and intuitive to use. Save often -- it crashed frequently for me, but when it worked, it worked well.
  • by waspleg ( 316038 ) on Thursday September 10, 2015 @07:32PM (#50499609) Journal

    I use avidemux [fixounet.free.fr] and audacity [audacityteam.org] to add Rifftrax [rifftrax.com] to movies. Basically copying out the existing audio, merging the rifftrack with the movie audio in to one track and then putting it back in as a separate audio track so the original audio is there too if you want to watch the movie without jokes (pretty rare for me actually, but it's nice to have options ;)).

    Someone else already suggested OBS, I've used that too for recording gaming video. It can be pretty intensive and it takes some fiddling to make it workable but it's not bad.

    • If you're doing this for surveillance, I use iSpy [ispyconnect.com] you can define a range of your camera that if it detects motion will start recording and you can choose how far in advance the buffer is and how long it waits before it stops. It can be triggered by audio too, I've found it to be very nice for watching maintenance at my apartment and people trying to break in to our cars.

      They do charge for remote access features but you can "roll your own" for that if you need it (I didn't). It can use an external mic as a s

      • by Trogre ( 513942 )

        The FOSS program motion [lavrsen.dk] works well for this too. I have it set to contact me when it detects movement, along with a few frames of the captured video.

  • I've done that a couple of times with kdenlive for some amateur skydiving videos. It probably wouldn't work so well for production quality stuff, but it works fine for what I'm doing. Most of my videos are less than 10 minutes, though.
  • http://thinkinghard.com/blog/H... [thinkinghard.com] http://www.hecticgeek.com/2013... [hecticgeek.com] http://alien.slackbook.org/blo... [slackbook.org] There are lots more blogs, website etc that have solutions
  • I can't help with the automagic part, but...

    I capture with this method too, except when working with pre-recorded studio performances. I highly recommend rolling the audio and video capture device for a few seconds before "action" to give you time to clap or tap an object in front of the camera, as close to the sound source (not necessarily the microphone) as is practical. you will then have a definitive reference point in your audio and video streams against which you can synchronize in your editor of ch

  • Have you thought about modding the device to add a mic input? If you check the web, someone might have already done it, or the schematic might be available, or it might be obvious if you take it apart.

  • I'm no expert at video editing, but did you look at Cinelerra [heroinewarrior.com] ? It exists for years (I mean, before VLC existed, before mplayer, even before Xine if I remember correctly... Who remembers aviplay ?) and apparently it is still (kind of) maintained.

    For Debian it's available on Christian Marillat's multimedia repository.

  • reaper has added video editing capability recently. while windows/osx only, the devs have made efforts to make it run well under wine. should allow for syncing/editing your audio into your video. i believe it acts largely as a front end for ffmpeg for a purpose like this. there should be a lot of help at the reaper forum. the demo is fully functional and not time-limited. BabaG
  • by Khyber ( 864651 )

    If you haven't tried VirtualDub you need to go do so. That nobody here even speaks of it is mounting testament to the declining mental health of this community.

    • That nobody here even speaks of it is mounting testament to the declining mental health of this community.

      The fact that we successfully read TFS and so aren't listing software that doesn't work on Linux is in fact a testament to our mental health.

  • by atari2600a ( 1892574 ) on Friday September 11, 2015 @05:16AM (#50501285)
    Hit record x2, CLAP, sync in post. You really can't zoom in & line up two tracks like the big boys?
    • Could you not have just answered the question like a normal person instead of being a douchebag about it? Or were you worried you weren't going to feel superior to anyone today?

  • I have a little Sony handycam and made a video of a headless 8 string acoustic guitar I built. Using the Blender video sequence editor, the audio kept drifting. So I just split the audio into two minute chunks and sync'd each one. I figured I was doing something wrong, but it appears this is a widespread problem.
  • Different device clocks drift at different rates, mostly due to heat. Oven-controlled crystal oscillators [wikipedia.org] are employed in more expensive devices to provide more reliable clocking.
    Sources that are simply synchronized at the start will typically quickly drift and lose sync. Unfortuantely the drift is often not linear. So if you sync them at the start, and then stretch or shrink one source via re-sampling, they may sync at the start and end, but not in the middle.
    A good solution for the OP's problem would comp

Lack of skill dictates economy of style. - Joey Ramone

Working...