Forgot your password?
typodupeerror
Data Storage

Laptop/Server Data Synchronization? 305

Posted by kdawson
from the two-way-update dept.
gbr writes "I've been trying to automatically synchronize data between a laptop and a server. When the laptop is connected to the network, I want all writes to automatically propagate across to the server. When the laptop is disconnected I want the laptop user to continue working with the local data. When the laptop is reconnected, I want the data to automatically re-sync. The issue is, the data on the server may have changed as well, which needs to propagate back to the laptop. The data doesn't contain anything too special, no database tables etc. It does contain binary data such as executables and word processing documents. I've looked at ChironFS, Unison file sync, and drbd. ChironFS needs a manual rebuild if a connection fails, and the user needs to know which machine contains the correct data. Unison requires the user to initiate the synchronization process manually every time, and drbd is just not meant for the job at hand. How do you automatically, and invisibly to the user (except in the case of conflicts), synchronize between a laptop and a server?"
This discussion has been archived. No new comments can be posted.

Laptop/Server Data Synchronization?

Comments Filter:
  • by wpanderson (67273) on Tuesday August 28, 2007 @12:07AM (#20380203)

    My Briefcase in Windows 95. It even has a cute ickle briefcase icon.

    Somewhat seriously, Offline Files in Win2K/XP is something I've yet to see done well on any other OS.

  • Re:Subversion (Score:5, Interesting)

    by Iron Condor (964856) on Tuesday August 28, 2007 @12:08AM (#20380219)

    At the risk of saying something stupid or blasphemous: why offer something that requires "writing some scripts"?

    If the OP wanted to "write some scripts" s/h/it could have done all the neccessary work with a couple foreach...cp...end. Or, hey, rsync.

    I am suspecting the OP is wondering whether there isn't something out there that "just kinda works" and only needs intervvention in case of a conflict.

    Knowing well that this will definitely be considered blasphemy: I've been using Window's "briefcase" system since Win98. It does "kinda work". Most of the time. And requires work when there's a conflict. Which appears to be what the OP is looking for. Given that the OP doesn't seem to want to just go that route, the question appears pertinent what s/h/it is looking for that Mr. Gates briefcases can't/won't do...

  • Groove (Score:1, Interesting)

    by Anonymous Coward on Tuesday August 28, 2007 @12:26AM (#20380365)
    Well their once was a really great program called Groove, but now MS ownes it, but it does exactly what you are looking for... It even has a nice interface. Obviously does not work with Linux without some tweaking but I've had success with CrossOver Office and Groove on FC7.
  • Try coda (Score:2, Interesting)

    by blymn (621998) on Tuesday August 28, 2007 @12:28AM (#20380377)
    Have a look at http://coda.cs.cmu.edu/ [cmu.edu] This is a disconnectable file system. It could be what you are looking for. Certainly, that is what I use for doing the same thing.
  • commercial products. (Score:4, Interesting)

    by deviator (92787) <bdp@PARISamnesia.org minus city> on Tuesday August 28, 2007 @12:44AM (#20380465) Homepage
    Offline Folders on a Windows client connected to a Windows server work reasonably well but sometimes get screwed up.

    Novell's iFolder is a very interesting alternative... runs on Linux/Apache/Java stack & only transmits changed blocks over an SSL connection.

    Other things worth looking into include Microsoft Groove--let's you synchronize an entire workspace with yourself on other computers or other people - and is relatively network & environment-independent (though Windows only)

  • by Anonymous Coward on Tuesday August 28, 2007 @12:53AM (#20380539)
    Yeah, if it actually worked.

    When I tried using it I found it made the system intolerably slow when resuming from standby or placing the laptop in a dock or anything that made the system try to resync the files.

    And then it was just plain wrong about determining which files needed syncing.

    At the time I did this, about a year after XP came out, it was clear from usenet groups that the windows solution was woefully tested and just not ready for prime time.

    It was then that I started using Unison, which when combined with ssh tunneling makes for a great solution.
  • Re:iFolder? (Score:5, Interesting)

    by killjoe (766577) on Tuesday August 28, 2007 @12:59AM (#20380577)
    >What do you mean if iFolder was mature?

    I don't know how mature the novell version is but the open source version is very far from being mature. In fact there hasn't been a stable release in more then two years and nobody knows if or when there will ever be a stable release.

    A while ago all the developers on ifolder either quit or were fired and the development was moved to india. Since then the pace of development has slowed down to a crawl and the new developers try to understand the code base and fix bug reports.

    Right now you can download something that is beta-ish but I certainly would not trust my mission critical data to it. If you want something that works you are going to have to pull off the trunk and compile it.

  • Synchronization Woes (Score:5, Interesting)

    by JWSmythe (446288) * <jwsmythe.jwsmythe@com> on Tuesday August 28, 2007 @01:01AM (#20380593) Homepage Journal
    A few people hit this one pretty well. rsync (and probably rsyncd).

        The more complex problem has been thrown at me a few times. What if it's not just one person?

        Say you have a repository of data that a dozen people may be working in. When they're all network connected, they're all dealing with the same file pool. When they take their off-line copies with them (unplugged laptops on vacation), they all make changes to the same files. Maybe mine is a one line change. Maybe one guy copy&pasted the first 3 chapters from War And Peace into a comment somewhere in the middle. Maybe another developer did some very intellectual looking changes but hosed some major functionality.

        When you start putting machines back on the network, who is right? The 6 guys who did real work are obviously right(ish), but they all made different changes. The very last change will end up being someone's 3 year old kid who was pounding on the keyboard right before daddy shut down the laptop, saving the new changes. Probably the last is the most recent, and right by most methods.

        It's not a pretty picture, and requires some intelligence to sort out the mess.

        The only "good" resolution I've found is to give logical authority to the changes. Bob is in charge of development. Any changes going into the development or production tree must clear him. He should be able to recognize that the 6 guys made changes, and diff them to come up with the common changes. The 3 chapters of war and peace go by the way side. And the guy with the 3 year old "developer" gets reprimanded.

        In the end, a good revision system and good backups are needed too. Something will slip through the cracks, and you'll need to roll back to something you hope is good.

        I take control over whatever I'm working on, so if I know I'll be working offline, I'll scp the data to my laptop, work on it on the road, and scp my changes up to the server when I'm done. Anyone else who may have worked in my project space in the duration should have known better. :)

  • Re:Coda (Score:3, Interesting)

    by gouldtj (21635) on Tuesday August 28, 2007 @01:04AM (#20380611) Homepage Journal

    I've always wanted Coda to work for this, but I haven't ever gotten it working. My current thinking is that I'd need to set up a server on my laptop, and then have the client talk to the local server. Then the two servers could sync.

    Does anyone have any information or case studies on how to make this work for a small network? Easy conversion tools? It seems like the ideal solution to me, but getting it to work seems difficult.

    --Ted

  • by knitterb (103829) on Tuesday August 28, 2007 @01:05AM (#20380623) Homepage
    Works well for me too. Sure, every now and then you get a mishap or a mistaken overwrite, but that's what [client and server] backups are for, right?

    I use it with my wife for our financial data. She syncs, makes changes, then next time I connect I can choose to use the server version or my version. Since there is no version control, you have to communicate. Then again, with the way version control works, if you end up merging a lot, perhaps too many people are working on the same problem anyhow.

    In the end it works well.
  • Re:Coda (Score:2, Interesting)

    by EvanED (569694) <evaned@nOspAm.gmail.com> on Tuesday August 28, 2007 @01:55AM (#20380859)
    My current thinking is that I'd need to set up a server on my laptop, and then have the client talk to the local server.

    I haven't actually used Coda, though I'm planning on it for a small network myself, because this is exactly what it was designed to do. But why would you need the server on the laptop? All you should need is a client. Have you tried it that way and it didn't work or something?

    I do hear it's a pain to set up though.
  • Re:rsync (Score:3, Interesting)

    by arivanov (12034) on Tuesday August 28, 2007 @02:17AM (#20380937) Homepage
    There is no point implementing laptop backups before implementing a no-quota IMAP mail server. Exchange in its native mode does not count due to a number of corruption bugs which hit you once your inbox exceeds 2G (it should be OK as an IMAP server, bugs are mostly in Outlook).
    As far as the user is concerned his primary concern for laptop data loss is email. So you have to back it up as a part of any backup solution. If you are storing email locally on the laptop and backing it up the backup will nearly always be corrupt due to file locking. Further to this, the financial, network and storage resource required to implement a laptop backup solution exceeds by far the cost of extra storage on the mail server. In addition to that if the email is stored centrally the backup size per laptop decreases straight away to become on the order of MBs instead of GBs for most users.
  • Re:rsync (Score:1, Interesting)

    by Anonymous Coward on Tuesday August 28, 2007 @03:22AM (#20381251)
    I am using Subversion. I keep the whole $HOME and app configs in it. For both Windows and Linux (altogether for 5 different hosts). Sharing even config of Firefox, Skype, localtexmf dir, etc between the two. Well, it's not precisely what you want (automagic switching between the two data storages), but most probably it is the cheapest solution for this anyway. Otherwise you might end up with a nasty inconsistent udpates from server thus loosing your temporary work (there is always a small piece which you have to merge manually if you have a multi-user environment!).
  • Re:rsync (Score:5, Interesting)

    by frenetic3 (166950) <houston.alum@mit@edu> on Tuesday August 28, 2007 @03:27AM (#20381279) Homepage Journal
    Apologies if it's in bad taste to reply to my own post, especially because it's about the product I'm working on, but here are some of Dropbox's differences/improvements over what people typically hack together themselves:

    - syncs continuously/watches the FS for file changes (no cron jobs needed -- things usually sync as quickly as they can be sent)
    - does binary diffing and only sends deltas (compressed & over SSL)
    - transparently archives past versions of all files (i.e. undelete/infinite undo)
    - syncs across any number of machines
    - lets you get to your files from the web
    - some more info @ http://venturebeat.com/2007/08/16/the-y-combinator -list/ [venturebeat.com]

    We made it after hacking together our own rsync-based abominations and getting really annoyed that no one had solved this genre of problems in a way that normal people could use.

    Okay, I can stop shilling now. I was just excited that other people run into these problems.
  • Re:rsync (Score:3, Interesting)

    by Jimithing DMB (29796) <dfe@noSpAM.tgwbd.org> on Tuesday August 28, 2007 @04:07AM (#20381449) Homepage

    This looks quite similar to Novell's iFolder but with you running the server yourselves instead of having your users set up an iFolder server. Last I used iFolder was in the 1.x or 2.x days and it frankly wasn't anywhere near the polished product you have here. Now it seems that iFolder 3.x [ifolder.com] is open source and looks a lot more polished.

    Still, I think you surely have a great service market here even though the polished front-end app seems to now be done open source. Best of luck to you on your new venture!

  • by ajs318 (655362) <sd_resp2NO@SPAMearthshod.co.uk> on Tuesday August 28, 2007 @07:07AM (#20382003)
    DEC had all this sorted out back in the days of the PDP-11! Filenames would get a version number appended (such as LOGIN.COM;12) and you could specify how many versions to keep hanging around. Writing a file without specifying a version number would create a new version, reading a file without specifying a version number would use the latest version.
  • by STFS (671004) on Tuesday August 28, 2007 @10:11AM (#20383527) Homepage
    subject says it all.

Take an astronaut to launch.

Working...