Ask Slashdot: Distributed Online Storage For Families?

StonyCreekBare writes "What options are available for distributed storage for families? My two brothers, my daughter and her husband, and his mother all have homes in various parts of the country. We use various cloud storage providers to keep our shared data. This has numerous limitations and we are starting to think maybe we can do it better ourselves. We all have decent Internet connections, are all somewhat tech savvy, and think that by leveraging the Internet we can maybe provide for our needs better and at lower cost by buying some hardware and doing it ourselves. How would you go about implementing such a family-oriented, distributed cloud platform? What hardware? What applications, beyond simply the preservation and sharing of family data, (grandkids' photos, home videos, and more) would be good to leverage such a platform? Security Cameras? HTPC? VoIP? Home Automation? Primary requirements are Cheap, Secure, Reliable."

s3

[Anonymous Coward]

Amazon S3 with Expandrive

Re:s3

[Jeremiah Cornelius]

Amazon with OwnCloud [owncloud.org]

rsync

[khasim]

What applications, beyond simply the preservation and sharing of family data, (grandkids' photos, home videos, and more) would be good to leverage such a platform? Security Cameras? HTPC? VoIP? Home Automation?

FIRST, you decide on what functionality you want.

THEN you look at how to achieve that functionality within your budget.

I'd use rsync as the cheapest means of replicating data between multiple sites. But once you start adding additional functionality requirements that might change.

Re:s3

[Jeremiah Cornelius]

Commercial, propietary and expensive. Stand up a linux box on EC3, with your storage portal of choice.

ownCloud is open source. If you are a Slashdotter, the time investment should be trivial and the geek/maker factor somewhat exhilarating. 20 bucks a month will blow the doors off of Dropbox pricing for terabyte in the sky. Plus you have a migration/passthrough to Drop, etc.

Many solutions

[Tynin]

I think I would have each family member that wants to be part of this shared drive do something like:
- each of you buy the biggest HDD available
- setup a ssh tunnel in the form of a circle between each family member, where Alice connects to Bob, Bob connects to Charlie, and Charlie connects to Alice.
- each family member then rsync's to the next family member over, where they would do a full rsync of the shared disk, but do an rsync --delete on directories that belong to themselves, so if they delete / move files around, it makes the needed corrections on other family members shared disks without wasting space.

If you are running Windows, you can setup a scheduled task to at a time in the middle of the night to launch cygwin, open the ssh tunnel, and rsync away. If it is linux, setup a crontab. Initial coordination would need to be done to get everything right, but then it would be very automatable.

I do not suggest trying to setup a distributed filesystem across the internet. There are many pitfalls. Whereas this solution, your only concern is, 1) is ssh up? 2) did rsync run? 3) is the disk full?

ownCloud Community edition will do nicely

[passionplay]

Clients for every platform. Server distributions for every platform. Mobile clients too. Runs on HTTPS.

1. 1. Download the community edition from any of the repositories found on https://owncloud.org/ [owncloud.org]
2. 2. Install using wizard - if you pick SQLite as the database, there is nothing to install for the database - configure to force SSL connections
3. 3. Setup your router to forward 443 to the box you've set up
4. 4. Setup a dyndns or similar IP address (or your own domain name) to said IP address.
5. 5. Install client (desktop or mobile) and start accessing using https://yourserver-or-ip-addre... [yourserver-or-ip-address] as the URL

I've set up something similar for my family - love it. I've also set up something simliar for our enterprise. No complaints about the regular feature set. Just some of the enterprise level things could do with a little more work.

Simple NAS boxes

[Anonymous Coward]

Our family uses simple NAS boxes (Dlink DNS-323, etc.). We put Debian on them and all the boxes use rsync over ssh in the middle of the night to synchronize the data. Pretty much every "family site" has one. They are also useful for private local storage, shared folders, etc. Everyone knows that any file they put in the "backup" folder will be looked after, everything else is just local. Been working okay for 2 years now. Note - this is not RAID, just distributed backups. Way cheaper than commercial offerings.

Re:s3

[Immerman]

But that's not distributed. Power outage at Uncle Jeremiah's while he's on vacation means everybody loses access to all their files until he gets home and fixes it. A fire means everything is lost permanently.

Now if ownCloud allows transparent mirroring between servers at different locations then you're on to something.

Re:rsync -- look at Unison!

[drstevep]

Rsync is a one-way synchronization. Check out Unison; it readily performs a bidirectional merge. You might have to do a little compiling, but hey, isn't that what the Family Geek is for?

I've been using Unison to sync a pair of Synology boxes that act as my cloud. (One in my office, one at home, each with a RAID-1 array.) I've also gotten it running on a pair of DLink DNS-323 boxes (yes, also RAID-1'ed). The Synology has cloud software; might be a good choice if you want to invest in a cheap small light unobtrusive (Linux) NFS/cloud/music server/etc box.

Re:Many solutions

[Compunexus]

Well that works if each house has a Geek. I have reduced the number of "please help" calls I get since I started getting them to use Bittorrent Sync. It works on almost any platform, does multi-way sync, and is as private as you want it. The caveat I found is that if one person deletes something it gets deleted on all. However, there is a settings file for each sync share that can alter that in a variety of ways. The only limit is the amount of storage that is dedicated to the share. Since everyone has a local copy of the files, even connection speed is not a factor.