Ask Slashdot: Syncing Files With Remote Server While On the Road? 239
An anonymous reader writes "Here's a scenario: you are on a vacation trip for a couple of weeks — on the road. Lots of pictures — 2-300 per day. Maybe some text files with short notes etc. You have a camera with Eye-Fi, a PC, and a phone with WiFi and 3G. Files ends up on the PC (mobile storage), phone provides Internet connectivity. Now, if you wanted to upload all files pretty much as you go — given spotty access to Internet over G3 and WiFi — what would be the best way to do that automatically; set-it-and-forget-it style? I would like them to end up on my own server. rsync script? ownCloud? Some BitTorrent setup? Other? I'm thinking of interrupted file transfers due to no network, re-starts etc. And I would not want to lose any files; including scenarios where files gets deleted locally — that should not result in files getting automatically deleted on the server as well. Sure; I could perhaps use something like Dropbox but that would take the fun out of it."
Fun? (Score:5, Insightful)
You say you don't want to take all the fun out of it, but you're trying to foist this idiocy off on a public forum? Save the fun for yourself, and make a blog post about your solution.
Re:Fun? (Score:5, Insightful)
I'm thinking this guy is a senior manager who would normally ask his IT drones to attempt to solve this problem. Like the one I had in my youth that wanted backup of his laptop to happen automatically for the random 4 hours a day it was connected to the corporate network, without impacting the performance of the laptop by doing anything too heavy like, you know, syncing files across the network.
I'm at a loss as to why people answer these kinda questions, if it was your own family you'd tell them to stop being such a lazy ass and remember to hit dropbox or whatever whenever they have a link.
Re:Fun? (Score:5, Informative)
You say you don't want to take all the fun out of it, but you're trying to foist this idiocy off on a public forum? Save the fun for yourself, and make a blog post about your solution.
Allow me....
Hello from my vacation in Indonesia. There is no automated solution. What you're ignoring is that networks in the developing world are not only patchy, they're flaky too. So, whatever worked for you yesterday might not work tomorrow or even in half an hour's time. Counting on 3G is a bad idea, because of its unreliability, but also because of its cost. Use wi-fi wherever you can. Most hotels these days provide it free of charge. Use rsync (with the zip option if you like), and keep it simple. My update script looks like this:
rsync -av ${SRC}/* ${USERNAME}@${DEST}:${PATH}
Yep, just a single folder in which I dump everything of value and a corresponding folder on my home machine. I just pop open a command line whenever ity's convenient (and possible) and run it. It doesn't always complete in time (the one I'm running as we speak won't be finished before I leave to go scuba diving), but I can always complete the sync later in the day.
Also, bring one or more external disks. Use them for quick and dirty backup while you're on the move. It only takes one rain storm (or fall in a river) to be glad you did. And don't count on buying new SD cards when you're on the road. Most of the ones for sale in the developing world are convincing knock-offs that last about two weeks. That's my experience anyway.
Anyway, simplicity is the single most important step for you when you're backing up data in the developing world. You can't rely on any other factor, so you should at least be able to rely on your own scripts. Which leads to my maxim: "In the absence of robustness, choose simplicity"
Re:Fun? (Score:5, Informative)
So having us piece something together for you (Score:5, Informative)
Leaves the fun in it? Either take the suggestions that work, ie Dropbox, or figure it out yourself.
Re: (Score:3, Funny)
Dropbox probably isn't going to work.
Just buy MORE INTERNAL STORAGE. Beef up your devices so that they can store as much stuff as you are going to generate while you are traveling.
That way you avoid all of the Cloud nonsense and it's limitations.
The simplest solution is to avoid the problem. I did the same thing with my own devices recently to avoid this very sort of problem and it worked out very well.
Re: (Score:3, Insightful)
Ehm, no, that is not backup. As a matter of fact, you might get robbed and then all your precious storage is gone. What about accidental damage (water damage: bag falls in water). No, the the best way is network backup, and I'd do it with rsync. What you suggest is not avoiding the problem: it's thinking that you avoid the problem.
For me, when I'm on travels: my devices need to be completely destroyable, losable and robable.... without losing much (max one day).
Re:So having us piece something together for you (Score:4, Insightful)
Multiple copies. One on your person and one with your luggage at the hotel/hostel.
Re: (Score:3)
"Unless the extra storage is flash drives that you mail home to yourself periodically"
Exactly.
IT seems the engineering with the lowest ability to "stay over the shoulders of giants". It seems that not only is there the need to constantly reinvent the wheel, but that it must be reinvented in a vacuum, forgetting all the lessons from the past.
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." â"Tanenbaum, Andrew S. (1996)
Re:So having us piece something together for you (Score:5, Informative)
Dropbox probably isn't going to work.
Yes it will. Perfectly, actually, in my experience.
And it won't delete photos when you delete them from the camera upload directory.
Check it out here: https://www.dropbox.com/help/288 [dropbox.com]
Further, it will chew on each image file till it gets a successful upload.
Re: (Score:2)
Re: (Score:2, Interesting)
I haven't tried it lately as I got scared off - does dropbox actually handle a file that you're using for a truecrypt container rationally yet?
Last I tried using dropbox it claimed to be able to handle partial files just fine - but it never worked in practice. My latest fly by night tests seem to indicate that this has been resolved - but I'm still leery of it since it's spectacularly failed so many times in the past.
People still recommend these clowns? (Score:3)
Re: (Score:2)
That's a terrible idea, and frankly I'm a little surprised someone with a 4-digit UID would say this. As the other posters have already probably said, there's good reasons for uploading your data to "the cloud" (but your own cloud, not just any cloud) when you're mobile and have valuable data: your mobile devices can easily be lost, stolen, or damaged. Making redundant copies and then keeping them with you doesn't solve that problem at all.
The solution seems fairly simple to me, but it would take a little
Re:So having us piece something together for you (Score:5, Informative)
rsync makes no sense to me, because of the requirement that file deletions on the mobile device should not be repeated remotely, which is exactly what rsync normally does.
It doesn't delete files unless you specify --delete
Re: (Score:3)
> That's a terrible idea, and frankly I'm a little surprised someone with a 4-digit UID would say this.
I simply bothered to read the details of the use case.
Been there. Done that. Been there again.
Your hysterical ranting about "valuable data on mobile devices" is really funny.
Re:So having us piece something together for you (Score:5, Insightful)
Calling "the cloud" nonsense is exactly the same as calling the Internet nonsense. "The cloud" is just a stupid marketing term for it, but it's still the Internet. I saw pictures depicting the Internet as a cloud long before "the cloud" came into popular usage.
And calling the Internet nonsense on an Internet forum is the height of hypocrisy.
Re: (Score:3)
Leaves the fun in it? Either take the suggestions that work, ie Dropbox, or figure it out yourself.
No, don't use Dropbox. Not for large amounts of data, anyway. It's shit on shitty networks (surprise surprise). Not their fault, of course. I live and work in the developing world, so I say this from experience.
Let me be the first one to say (Score:5, Insightful)
Re:Let me be the first one to say (Score:4, Informative)
Why use cloud service like dropbox when you can do what real men do and build your own using unison -
http://www.cis.upenn.edu/~bcpierce/unison/ [upenn.edu]
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
Unison IS really cool if you need to synchronize two directories on two different machines and may make changes on either one. However, I don't see how it'd be useful in this case; unless I'm missing something, the asker doesn't need to keep copies locally once they're safely uploaded to the remote server (it'll take too much space), plus he doesn't want the remote copy deleted if the local copy is, and unison and rsync both do that by default. It's not "synchronizing" if you don't mirror ALL changes to t
Re: (Score:2)
Good points, unison is probably not the way to go. I just wanted to sneak in a semi-on-topic plug, because I use it all the time (seems there was no need, so many people are suggesting it).
It seems stupid to delete the local copies though. Unless the laptop has an SSD, there will be enough space for all the pictures, and that's the only backup the submitter will have. Sure, he/she can set up timestamped backups at the home server, but there's no way to know for sure that a bug didn't sneak in to the scripts
Re: (Score:2)
It really depends how many photos he's taking, and what kind of storage space he has on the laptop. If there's enough for the whole vacation, then great, might as well leave them all on there. If not, you'll have to come up with a scheme for deleting at least some of them. With typical DSLR photos these days being at least 5-10MB each (more if you save them in RAW format), it doesn't take that many to fill up a ~300GB laptop drive. Sure, some 2TB drive can store a lot more, but laptops don't have drives
Re: (Score:2)
plus he doesn't want the remote copy deleted if the local copy is, and unison and rsync both do that by default.
Remote deletion of files is NOT default in rsync. You have to use --delete if you want that.
Re: (Score:2)
No, rsync does not do that by default.
Rsync is exactly what the poster wants, but he'll have to put it inside an script that delete the files after they are uploaded. The script can be something like this:
find -iname "*.jpeg" -exec rsync '{}' machine:remote/dir ';' -exec rm '{}' ';'
Just put that (at second plane, with output redirect to some file) at the on-connect script at his machine, and he's done.
Re:Let me be the first one to say (Score:5, Insightful)
use something like Dropbox. It works fine, does exactly what you want, what's the point in reinventing the wheel?
I think, "that would take the fun out of it" pretty much covers it. Sounds like he's interested in the process of rolling out his own solution and putting it to the text. There's a lot of pride that goes with using something you've built yourself.
Your answer, given that he already says he's aware such solutions exist, is a bit like telling a guy rebuilding a car in his garage to just buy one new, because it'd be simpler and cheaper. Yes, it would, but that's not the point.
Re:Let me be the first one to say (Score:5, Interesting)
Re: (Score:2)
The alternative is your much more sophisticated networking solutions that would require enterprise level remote syncing.
If you're talking about 20 or 30 GB of data per day that might need to be synced in whole or in part, rather than 2-300 MB of files, then you would want something like a VPN solution or enterprise cloud services. Which is basically dropbox that you pay a lot of money for.
If this is windows I would think synctoy could be configured to do what the OP wants, but... why?
Re: (Score:2)
If you're doing 20-30GB of pictures per day on vacation...that's not a vacation, that's work.
Re: (Score:2)
Re: (Score:2)
Hear hear. Some of these "ask Slashdot" posts are getting ludicrously complex for such simple problems. To he extent that I, like you, think it's just trolling.
If you want a complex solution, don't ask Slashdot, figure it out yourself. If you want a simple elegant solution, then it has been suggested already.
Re: (Score:2)
It won't be much "their own solution" if (s)he's asking /. for one, now will it?
I'd vote for Tarsnap. Cheap, secure (encrypted in the client), integrates well with standard Unix tools, etc.
Re: (Score:2)
Re: (Score:2)
All the internet is one giant Troll - the secret is in extracting value from it. Some people succeed.
Re: (Score:2)
So sync into a truecrypt file locally. Or use SpiderOak instead.
Re:Let me be the first one to say (Score:5, Insightful)
What a useless comment. The original poster is already aware of DropBox, and discarded it from consideration
He discarded it for no valid reason. That is what the GP is pointing out.
It is by far the cheapest most reliable solution to this problem, and it even allows leaving the laptop at home and simply uploading camera pictures to your smartphone where they can automatically be loaded to dropbox.
If the OP is unwilling to consider dropbox, where the solution is handed to him on a silver platter, then why should we waste our time to spec out his system for him?
Re: (Score:2)
Re: (Score:2)
You lost me. What in the name of the mighty Zeus does copyright and patent law have to do with this? I mean I know this is slashdot and all, but please, spare us the drama.
Yeah, it's dramatic to be wasting billions in labor to invent devices, methods, and software that's only different enough to get past some patent or copyright restriction. But as you said, this is slashdot, and big picture thinking is a rarity here... you know, seems too... dramatic.
git-annex - "The Nomad" use case (Score:5, Informative)
http://git-annex.branchable.com/
use case: The Nomad
Alice is always on the move, often with her trusty netbook and a small handheld terabyte USB drive, or a smaller USB keydrive. She has a server out there on the net. She stores data, encrypted in the Cloud.
All these things can have different files on them, but Alice no longer has to deal with the tedious process of keeping them manually in sync, or remembering where she put a file. git-annex manages all these data sources as if they were git remotes.
When she has 1 bar on her cell, Alice queues up interesting files on her server for later. At a coffee shop, she has git-annex download them to her USB drive. High in the sky or in a remote cabin, she catches up on podcasts, videos, and games, first letting git-annex copy them from her USB drive to the netbook (this saves battery power).
When she's done, she tells git-annex which to keep and which to remove. They're all removed from her netbook to save space, and Alice knows that next time she syncs up to the net, her changes will be synced back to her server.
Re:git-annex - "The Nomad" use case (Score:5, Informative)
I've been using git-annex for this kind of synchronization for a while and have found it works quite well. It supports interrupted file transfers (via rsync), automatically keeps hashes of all your content, and lets you work offline effecively. git-annex is particularly good at synchronizing file renames between two machines: it actually tracks the renames so you won't have to re-upload anything like you would with rsync. It also supports more than one machine, if you want to make backups.
I would definately recommend trying it.
Re: (Score:2)
When she's done, she tells git-annex which to keep and which to remove. They're all removed from her netbook to save space, and Alice knows that next time she syncs up to the net, her changes will be synced back to her server.
Isn't this exactly what the OP didn't want to happen?
scenarios where files gets deleted locally — that should not result in files getting automatically deleted on the server as well
I'm sure there will be a flag to disable server-side deletion, but is this a "supported" use case or a dirty hack? What happens if the OP deletes all the old photos from the synchronized folder on his/her laptop, fills the folder with new photos and tries to sync again? Will git require for the old photos to be downloaded before the new ones can be sync'ed?
Re: (Score:3)
You misunderstood, 'though it's not really your fault, it's confusing.
When Alice removes the files, it's not system-wide deletion, just a local one. In fact, she can remove them safely because Git Annex tracks their location and knows there is at least one more copy remotely, so it's safe to delete the local one.
You can even configure it to require three or more copies of each file.
Take fewer pictures (Score:5, Insightful)
Re:Take fewer pictures (Score:4, Informative)
Oh, any unreasonable photographer will do that in a nice place.
That said, uploading to the Internet is kinda dumb. Just take a pair of cheap hard drives, download it to both, keep a copy on the laptop. If you're going through customs, give one to someone else in the unlikely event you're mistaken for a terrorist. You can also mail a drive back.
Forget the Internet for a while. You'll be healthier and happier.
Re: (Score:3)
300 photos, ~10meg each = 3GB/day... 32GB flash card, problem solved?
Re: (Score:2)
If you are someplace where there's something worth looking at, 300 pictures is nothing. Also, you take more than you are going to keep to better ensure that you end up with results that are worth keeping.
Current tech makes it easy. Makes it easy to take them, to accumulate them, and to cull them later.
Also, there is nothing "burdensome" about taking in a place well enough to take some good pictures of it.
Re: (Score:2)
You don't have to be taking pictures every couple of minutes to accumulate that many over the course of the day. When I take pictures on vacation I take a few of each shot I'm trying to get. Maybe I don't see that part of it is out of focus on that tiny 3" screen on the back of the camera. Or maybe I want some shots from different angles. I decide once I am home which are worth saving and which are not.
Re: (Score:2, Interesting)
These people are from a different generation. People my age that were into photography learned to compose our shots and consider proper exposure prior to taking the shot because each roll of film only had so many exposures. When I traveled, I MIGHT shoot 3 rolls a day.
Kids these days just spam the button on their DSLRs, relying on quantity to provide them with a good shot after the fact.
Re: (Score:2)
Re:Take fewer pictures (Score:5, Informative)
Not true. This from an interview with Adams' son:
Ansel Adams frequently made duplicate photographs of his images when taking them. One thing that I tell people constantly is that it is always a good idea to take more than one shot of an image if you can in the camera. According to Michael, Ansel frequently took multiple exposures of the same shots. Many of his negatives are duplicate images of which he'd select the best image to use for printing.
10 Interesting Things I Learned About Ansel Adams [flickr.com]
Re: (Score:3)
Yes, and that shows that you weren't really that much into photography. A serious photographer back then (like the guys who worked for National Geographic) would go through dozens of rolls of film a day. Sure, it's expensive, but if you're not willing to develop hundreds of shots a day and throw most of them away, and pay the price that entailed, then you're not a serious photographer.
These days, you don't have to have a big budget to shoot the way a serious photographer did in the film days; you can be j
Re: (Score:3)
Kids these days just spam the button on their DSLRs, relying on quantity to provide them with a good shot after the fact.
What a bunch of elitist bullshit. You know damn good and well that you missed lots of opportunities to shoot because of film rationing.
Re: (Score:2)
When I was 20, I shot 30 rolls of 36 exposures in approximately 30 days on a Eurail excursion. It's easier to point and shoot 3 fast compositions, then throw 2 away, rather than agonizing about how to carry a little less film. Digital just does this one better.
Re: (Score:2)
The important thing is that he not share ALL of the pictures he takes. Unless you're an exceptionally good photographer, if you take 300 pictures, you'll probably end up with at least 290 bad ones, but if you're lucky, you'll have 10 that are great. Only share the good ones, and your friends will think you're a talented photographer.
So I hope you're uploading those for archive purposes, to be sorted later. If you're sharing all of them, your audience will likely get bored fast.
Re: (Score:2)
So I hope you're uploading those for archive purposes, to be sorted later. If you're sharing all of them, your audience will likely get bored fast.
When I was a kid, every year my dad would take us on a two week trip where we'd drive across a corner of America. In one trip we hit the Rocky Mountains, Las Vegas, and lotsa neat places in Utah whose names I cannot remember right this second. If my dad called me up and said "We wanna do this trip again!" not only would I be interested in doing what the star of this Ask Slashdot is doing, but I'd easily be shooting 300+ photos a day... only I wouldn't be wasting any of the photos. Unlike him, though, I w
Re: (Score:2)
He could be shooting time-lapses.
BitTorrent (Score:3)
One drawback of BitTorrent is that it is meant for static, large files. RSS integration into bittorrent clients can help, but it's still not a good publishing mechanism. Plus you need quite a few people (or a dedicated seed box) to get it going. For your case it wouldn't be better than pushing to a web server.
The answer also depends on who you want it to view it, and how the access should be. rsync script is probably easiest.
Synology Diskstation (or other NAS) (Score:2)
However I suggest getting a good NAS and my suggestion is a Syn
Fish-Sync (Score:3, Informative)
There is an open source tool for just this kind of situation. It is called Fish-Sync and can be found here: http://fishsync.sourceforge.net/
It basically works like Dropbox or other sync services, but it syncs files between computers you have access to, rather than a third-party server. Fish-Sync is basically a combination of rsync + OpenSSH + Dropbox's LAN sync with an optional pretty graphical interface. (It also works on headless serves if you don't want the GUI.
Unison (Score:5, Informative)
I am a recent convert to Unison. I discovered this because I was trying out Google Drive, which I found worked well, but I don't want to keep more of my data in someone else's cloud if I have to. So far I have been using the Mac OS UI. With a solution with Unison you will need your own server with ssh access to the Internet. The downside is that you have to worry about backups or uptime, though you don't have to worry about some government taking the service offline permanently (or while they spends years trying to establish possible guilt).
The advantages with solutions like Drop Box or Google Drive is that you not have to worry about the server side. Depending on the amount of data you want to store you will have to choose between the basic paid access or the paid access, which gives you more storage.
BitTorrent is probably the worst solution here, since it only works well when the data is massively distributed. If you only ever have one peer, then you are better off with one of the solutions mentioned above.
Unison ! (Score:2)
Is this an Ad for eye.fi Premium? (Score:2)
You mentioned you have a eye-fi card already. You're describing the eye-fi premium upgrade:
http://www.eye.fi/how-it-works/eyefiview#premium [www.eye.fi]
"Eye-Fi Premium: no limits.
With Eye-Fi Premium, photos & videos sent directly from your camera to your Eye-Fi View are available for as long as you like. Enjoy unlimited storage and the flexibility to access, share and download your media in full resolution anytime. Whether you’re at home on a second computer, on your iPhone on the go or on an iPad on vacation
The correct question should have been... (Score:2)
... to ask yourself "Am I a journalist, or a blogger where I need to report every day and send my photos to the mothership???" If the answer is 'no' why go through this elaborate setup? Have some consideration for the people whose open wifi hotspots you'll be leeching off of in order to send hundreds of megabytes over; think of the outrageous charges you'll be incurring for sending that much volume over 3G (and overloading the system for all the other users while you're at it). What's your big rush to se
Three hundred?! (Score:5, Funny)
Re: (Score:2)
300? I think my sister shoots that while they're eating breakfast. A typical day for her is probably north of 1000. Kids these days.
*geezer mode* Why I remember when I went to Europe for three weeks as a teen, and shot nearly a dozen rolls of film - nearly 300 pictures in all. I though I'd go broke developing all the film. */geezer mode*
Re: (Score:2, Funny)
*geezer mode* Why I remember when I went to Europe for three weeks as a teen, and shot nearly a dozen rolls of film - nearly 300 pictures in all. I though I'd go broke developing all the film. */geezer mode*
<older-geezer>When I went to Europe for 3 months as a teen, we were too busy shooting the enemy.</older-geezer>
Why? (Score:2)
User your phone as file server (Score:2)
1. Call up your phone provider, and shell out $500 or so for a static IP address
2. Hack your phone to run an FTP server.
There,it's done. Your files are not on your own file server, available from anywhere.
or just go use dropbox and stop looking for convoluted solutions.
Ugh. Why worry about it? (Score:2)
You're on vacation. Just get a couple 16 gig SD cards (or whatever your camera takes). Unless you are shooting in RAW you are not going to fill those cards. Then just dump them to the laptop you have with you. Worry about transferring everything when you get back. You are on vacation, enjoy it.
Re: (Score:2)
that's what i was thinking. then for a 'backup', copy the pictures to extra blank cards or usb stick and ship home (or to office, parents, whatever).
there is no need to upload up to 20 gigs (or more.. an estimate: our own camera would be about 20 gigs for 4000 photos) of photos over the two weeks over the internet... none at all: would take forever and a day on mobile (if your plan could even survive that volume without insane surcharges); you'd be lucky to get necessary bandwidth at hotels (every one we
Why Sync at all? (Score:2)
For $150 bucks you can easily get an external 2-3 terabyte USB hard drive. Dump everything to that and deal with it when you get back home. Unless there's some reason why you want/need those photos and files at home before you get there...
Re: (Score:2)
And what happens when you drop your USB hard drive in the hotel room and it won't work again, losing ALL the photos you took that vacation? Then you'll be wishing you had uploaded them to the much more reliable drives on your home server.
The ovbious (Score:2)
Don't come here and dismiss the obvious solution, (Dropbox) for which which many newer phones support automatic uploading of both phone pictures and photos transferred to the phone from an external camera. (And deletes are not mirrored). You can transfer picture so the phone via Samba server on your phone (at least with Android you can) or bluetooth or cable. From then on you forget it. It will take care of it. It will re-try till it succeeds without you having to do anything.
And when you get home, Dropbo
Re: (Score:2)
"There is little point in building your own these days."
What about privacy? Maybe OP is a spy. You would not want trade secrets on dropbox's servers, now do you?
Re: (Score:2)
Yup, spys always post for solutions on Slashdot.
Sorry, I forgot about that.
Presumably all that gear, big camera, laptop, cell phone, all fits in the heel of his shoe and nobody would suspect a thing.
Re: (Score:2)
I prefer Slashdot to be the kind of site where its users research and try to develop their own stuff instead of asking others to do it for them.
If he wants someone else to design him/her a syncing solution, then Dropbox is a valid answer.
If (s)he wants to build it him/herself, then (s)he should be reading the man page for rsync, not this thread.
Re: (Score:2)
Sparkleshare? (Score:3)
Can anyone who has used Sparkleshare say if it would work in this situation? I'm looking at building a Sparkleshare server, which is described as an open source version of dropbox, but where you control the server. On some level, it doesn't look that hard to set up, but there are parts of it that still aren't explained well at the website.
But if you search for open source dropbox alternative, Sparkleshare shows up on a lot of lists.
http://sparkleshare.org/ [sparkleshare.org]
some hotel / free wifi block ports 3g nated at tim (Score:2)
some hotel / free wifi block ports some times the 3g is nated.
Don't count on having all the ports that you do have at home.
A few questions (Score:2)
The primary question is: why are you trying to do this? Is it to make sure you have an off-site backup in case all of your electronics gear gets stolen? Redundancy can be best covered with extra hard drives.
Another consideration is what kind of photos you are taking. If you're shooting RAW with a modern DSLR, you're going to have images of 20-30MB each. At 300 pictures per day, you could be looking at a data footprint of up to 9GB per day. I don't know what kind of coverage or data plan you have, but in my
Try CrashPlan (Score:5, Informative)
I've been running CrashPlan as an online backup solution for my Linux system to back up all of my photos. It also has a feature allowing you to back up to another PC over the Internet. It's easy to set up so you can back up to your home PC and it's free (unless you buy the cloud backup service). See http://www.crashplan.com/ [crashplan.com]
-Aaron
Rsync; better yet: datamover (Score:4, Informative)
I think rsync pretty much provides all you need in one tiny command-line to get data from A to B.
But if you want to increase your resilience against failing network connectivity, and make sure you don't delete anything that hasn't been properly copied to your server, I suggest you take a look at datamover: http://www.cisd.ethz.ch/software/Data_Mover [cisd.ethz.ch]
Essentially, it's a daemon written in Java that monitors an outgoing directory. Everythings that is written in there gets safely copied over to a central storage drive. Behind the scenes, they use rsync to do the copying, but it's wrapped in tons of features that improve the reliability of the moving process, like a quiet period before a file gets moved (good for applications that write their output incrementally and sporadically into files), multiple retries on network time-outs, high-water marks, data transformation (e.g. compression) during the move process, etc. It also is very anal about sending you emails for anything that could possibly be a data integrity problem.
We rely on it to store the raw data from scientific experiments. With the proper configuration, your holiday pictures should be just fine.
Wow ... missing vacation by saving it! (Score:2, Insightful)
Sigh ... this is really, really sad. The original poster is willing to "miss out" on the true vacation by trying to save a digital copy! Am I the only one here thinking ... relax! ... enjoy your loved ones! ... live in the now! What, exactly, is the point of "saving" something that you were never really with 100% in reality?
On the road? (Score:2)
Is there a circle of hell reserved for distracted drivers who kill/injure others?
(Maybe in Nivens Inferno rather than Dante's
Seriously (Score:2)
Mail yourself CDs. (Score:2)
Why not leave the junk at home and have a holiday? (Score:2)
I'm sure that whoever you're going with would appreciate you paying them a little more attention and not spending quite *all* of your time fiddling with gadgets.
Or is it not allowed to let the real world intrude into Planet Slashdot?
Kids these days and their 'problems' (Score:2)
Get off my lawn.
When i was a kid we put the spent film in our pocket ( and then suitcase ) and waited until we got back to see the pictures.
USB Drive in backpack; netbook in hotel/car (Score:4, Insightful)
Synkron (Score:2)
I used to use cloud sync, but I found that none of them were robust enough for my abusive ways. Most required periodic manual merges.
I switched to Synkron syncing to a network drive (VPNed when I'm on the road) and I've been happy ever since.
iCloud (Score:2)
Err.. wouldn't Apple's iCloud be the best, since that's what it was designed to do? Sync photos, set and forget style?
Re: (Score:2)
I was checking to see if someone posted this. Click the iPhone/iPad/iPod shutter button. The picture automatically goes to the cloud and is automatically downloaded to a PC/Mac at home. It also automatically appears on any other authorized iDevice, and if you have Apple TV it is also immedately part of the screen saver. Then the PC can farm it out to friends and family. Travelling with a laptop, command lines, extra hard drives... Quaint.
my experience (Score:3)
I am on holiday now and am currently uploading the day's photos with SugarSync. Its cheaper per byte than DropBox and has more options. I think some people might find the many options confusing but not Slashdotters. eg you can sync multiple folders. When I was in a remote place with a slow connection (Laos) .... the smallest transaction was a file. So it might labor away for an 30 minutes to almost send a 3M file then the connection would drop and resume and it would start again with the same file - ouch.
I found a problem
Fun - Competent - Vacation (Score:2)
Fun - Competent - Vacation
Choose two.
Dropbox / lsyncd / rsync (Score:2)
I'm on the road since more than a year in a round-the-world trip and I think my setup is fit.
Two notes,
- when travelling, internet connection level changes a lot and sometimes you can be not connected for days, so an external disk/memory card rotation should stay your primary backup, especially for photos
- for pictures, I'm taking both JPG+RAW and doing Internet backup only for JPG. If I have a full internet connection at my accomodation and can let transfer during night, if not day, it's working pretty wel
Rsync script (Score:3)
Your first idea will do exactly what you want. It'll resume interrupted uploads, verify upload integrity, be secure in whatever context you're in... it's perfect. As long as you don't specify the --delete option, it won't delete remote files just because local files are gone, either.
A crontab line with: "killall rsync && rsync -a ~/local/photo/dir hostname.foo.net:remote/photo/dir" will do the job admirably. Set it to run every 10 minutes or so. You could obviously polish this solution in various ways, but it's quick, it's easy and it will work.
Re: (Score:3)
Er... replace the "&&" with a semi-colon. I type too many chained commands with && and my fingers get ahead of my brain...
Re: (Score:2)
I'm trying out CrashPlan now. Prices are reasonable, seems to work well (if slowly) and has little overhead.