Ask Slashdot: Building a Personal FOSS Cloud? 189
An anonymous reader writes "Cloud-based personal data management is pretty cool... if you don't mind entrusting the entirety of your personal data to a gigantic corporation. Apart from the risks of their doing unseemly things with your data, also the security of your data is entirely in their unreliable hands. So, is it possible to build my own personal data repository, where for example, I can store my contacts and calendars to sync to multiple devices? This could be hosted on any third party hosting service assuming also that all of my data was encrypted at the data level. So even if the host wanted to look at my data, all they'd see is 1s and 0s. What are the options for the tinfoil hat wearing FOSS folks that want to participate in the cloud age?"
Thanks for sharing (Score:5, Insightful)
So even if the host wanted to look at my data, all they'd see is 1s and 0s.
That was the dumbest thing I read all day.
Re:Thanks for sharing (Score:5, Funny)
Re: (Score:3)
Pfft! When I open a book I see blonde, brunette, redhead....
Re: (Score:2)
Re:Thanks for sharing (Score:4, Funny)
Re: (Score:2)
Re: (Score:2)
SQUIRREL !
Re: (Score:3)
Found it when googling for dropbox alternatives (Score:5, Informative)
http://owncloud.org/ [owncloud.org]
- Calendar
- Contacts
- dropbox like storage
Re:Found it when googling for dropbox alternatives (Score:5, Interesting)
Or use any of the usual storage services that provide a client to maintain a sync'd mount point, and just secure the contents. Jungledisk will do this for you for Amazon or Rackspace backed storage. Google Drive, Dropbox, etc. can be used with your own encryption mechanism.
For bonus redundancy, sync the local cache to an external USB drive so you don't get caught with your pants down if one of those services botches your remote store.
Re: (Score:3)
Re: (Score:2, Informative)
http://owncloud.org/
It's pretty cool, but right on the first page it pulls code from googleapis.com. Hit the front page and you send a request with the referrer URL to one of the biggest stalkers. Maybe it's still good, maybe it's not hard to redirect that js link to your own machine, but it just seems like they've missed the fundamental point of not giving your data away.
Re:Found it when googling for dropbox alternatives (Score:4, Insightful)
Re:Found it when googling for dropbox alternatives (Score:5, Informative)
The actual ownCloud application that you setup on your server doesn't have a reference to googleapis. I just checked on my installation.
For those wondering, the project website links to the jQuery library hosted on Google's server so they don't have to host it themselves.
Re:Found it when googling for dropbox alternatives (Score:5, Informative)
For those wondering, the project website links to the jQuery library hosted on Google's server so they don't have to host it themselves.
And more importantly, so that we don't have to be constantly re-download the same file, since we probably already have Google's copy cached.
Re: (Score:3)
That's on their site. The one where you download the software from. The point of his question was how to store data on your own site.
Download and install owncloud, and there's no sign of googleapis.
Re: (Score:3)
And the sysadmin in me feels that constantly polling off to get the latest ver
Re: (Score:2, Insightful)
A solution to this problem is to use a proxy that removes the Referrer header.
Personally, I use a proxy that removes all headers except 'Host', unless a site absolutely needs it. For a few sites I need to enable 'Cookie' for example.
In addition, you could configure the proxy to only get files from googleapis once and cache it in the proxy.
Posting anonymously since I was never here.
Re: (Score:2)
If you believe that your local department store is not using your address and your purchase information for marketing purposes, I have this shiny bridge you might be interested in.
Re: (Score:2)
You're right! They only have what you give them, so don't give them anything... I'm not sure what your problem is with that?
"Besides, who gives a rat's ass what data they have."
You seem to be mighty excited, so.. you, for one. Certainly enough to make claims such as "in fact they keep little relevant personal data" with nothing to back it up :P
cloud vs server (Score:5, Interesting)
Also, building a server or buying one secondhand is cheap, if you want to DIY.
Re:cloud vs server (Score:5, Interesting)
redundancy for family photos for instance on all family computers for instance. obviously private storage as well. the odds of all computers going down at once in multiple locations is highly unlikely. p
Re: (Score:3)
You might want to try tahoe-lafs [tahoe-lafs.org] if you want to share stuff with a fair number of people without giving them default access to the content.
Re: (Score:2)
I've actually been confused for a while now how the "cloud" is different from just having servers on the internet.
Is it just the synchronization that makes it the "cloud"?
Re:cloud vs server (Score:5, Informative)
Servers are web 1.0. Cloud is web 3.0. Much buzzier and hipper.
Re: (Score:3)
Re: (Score:2)
I'm still not really seeing the difference. I rent a VPS that I rsync stuff to. I don't care where it is physically, it has a domain name and I can reach it wherever it may be, even if it gets relocated somewhere.
Re: (Score:2)
For Christ's Sake, Just Get A Big USB Drive (Score:5, Funny)
You can write "The Cloud" on it with a Sharpie if you absolutely must.
Re:For Christ's Sake, Just Get A Big USB Drive (Score:5, Funny)
The difference is, if there's a fire in the house, your cloud will go up in smoke.
Thank you, thank you, I'll be here all week.
Re: (Score:2)
Re: (Score:2)
... and plugged into a small little audrino powered by induction transfer from outside the safe!
(that is, after somehow solving the 'faraday cage' issue with metal safes :D ... we need to get power out, and exchange a Wifi signal with the outside world)
http://commons.wikimedia.org/wiki/File:Arduino_Pro_Mini_powerd_by_inductive_power_transfer.jpg [wikimedia.org]
Re: (Score:2)
Then store it in a small fireproof safe.
A cloudsafe.
Re: (Score:2)
We're working on it (Score:5, Informative)
https://github.com/wurp/Friendly-Backup [github.com]
It works now, with some bugs. The first targeted usecase is distributed backup.
However, it can store arbitrary read-only content-addressed data as well as signed labels that point point to a particular piece of CBA data to emulate mutable data.
I have a whole slew of plans beyond backup for it, but backup seemed like the thing everyone needs and would most like to have for free on a federated data store.
Re: (Score:2)
Java? *facepalm* (Score:2)
:(
I wish Linus would take a few weeks off to write a distributed backup system, but he just uses public FTP servers...
Of course, there're several projects that use git as a backend, like http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own [kickstarter.com] (already funded; he's also a Debian Developer).
Since git isn't a backup system, using it as one isn't as efficient as it could be, but it is powerful. Joey's project is an exciting potential Dropbox replacement. He knows what he's
I don't get it. (Score:5, Insightful)
OMFG, the cloud. I got to have or do the cloud. Magic Ponies in the cloud!!!!
Seriously, wtf do you really need the cloud for? Is it going to magically sync all your different data together so you can access it all the time?
No, seriously, do you think it's going to sync all your data so you can use it and access it anywhere?
No, it's not. Sure, you can access you data anywhere, but duder, we've been doing that for a couple of decades now, way to join the late train.
Unfortunately, the various corporations don't want to agree to standards, so having docs/apps/whatever working with everything isn't in the "rape as much money as we can" business plan. so nothing is going to change.
Now let's look at the Megaupload thingy. That was cloud storage, file lockers. It's not around now, is it? That is what happens to clouds, the winds blow them away. The wind? Oh ya, in this case, that's the good old USA Government, working for their Pimps, the Music/Movie Industry. You think that can't happen to any "cloud" servers? Think again. OMG, Terrorist used that server, Child porn was on that server, boom! You're data, which has nothing to do with those 2 things, is gone also. Hope you make a backup. Oh, wait, the cloud was magically supposed to back it up for you?
Cloud has been around for awhile, but we called it what it was, the internet.
I have the above, and it's not a cloud (Score:5, Interesting)
You want the above? That's easy. Access to email from anywhere, access to my contacts and my calendar, how about access to all my files? Yep got that. Though it doesn't have a fancy name like "cloud". If I were into marketing I'd call it a cloud, but right now I'll stick to calling it an "internet facing linux machine"
Yeah it's not as exciting, but it does everything the so called cloud has done and it has done it for many years before this mythical cloud has existed. My phone sees the same set of files and emails as my home desktop PC, and there's a web interface to access all the above too.
Seriously just google "Linux Groupware" and maybe "Linux Web Fileserver" and you'll have everything that the cloud has.
Re: (Score:2)
It is a fun reaction when I show folks how my netbook storage expands from a measly 250GB to 6.2TB when I'm connected to the Internet.
SSHFS has been around for years!
Re: (Score:2)
While I agree on the overuse of the cloud meme, things like sshfs still don't offer encryption of the data at the server, so that someone with physical access can view all the data; it only encrypts the transfer.
Is there anything that can hold an encrypted data store that is only decrypted by authorized clients on a local basis? Ideally it would be something that gives different layers of access so you can navigate a directory tree only with authorization, and only download files you need.
The only thing I
Re: (Score:2)
This is something I have been pondering... i havent found it out in the wild so am tempted to build it myself. Perhaps we are looking for the same thing.
What i essentially need is an encrypted disk image (crypto-loop based?) that can be read and written to without ever "mounting". When ssh authentication has passed, the SSH key is used as a "pass thru" into the disk image. But it is never mounted or exposed to the host OS, only mounted "virtually" to the userspace process that handles the ssh connection.
I u
Re: (Score:2)
Isn't that basically FUSE? Not mounting is one thing, but FUSE mounts per-user, anyway.
But if you don't trust the kernel to restrict permissions, how can you trust it for anything? You can't not expose something to the kernel--it's the kernel. So encfs may suit this use case. If you need it as a single file, you could archive the directory when it's not in use.
Perhaps something like a GPG-encrypted tar.bz2 file that's decrypted to a tmpfs would work. But for a stream-like format, rather than requiring
Re: (Score:2)
Thank you for your insight. I will keep it in mind.
My problem is that I do not trust root. If your data is on a KVM/XEN VPS somewhere (even with whole disk encryption), root on the host machine still has access if your VM is running. You simply don't know how well that host machine is secured. Someone gains root access to the host, everyone on there is screwed.
Upon further thinking this through, it would seem (on the surface) I would have to build an SSH server and emulate the FS calls through to an archiva
Re:I don't get it. (Score:5, Insightful)
Oh give the guy a break. This is exactly the situation the "the cloud" buzzword was created for: people who are scared of the phrase "file server". There is absolutely nothing new about "the cloud" in any way but it's a nice fluffy word that people are comfortable with and it's acceptable to not have any idea what it actually is. I'd change the hostname of my home server to thecloud just for wiseassery's sake if it wouldn't hose my Trek shipname naming scheme.
Re: (Score:2)
No, they're two completely separate concepts, but you're right in that it seems to be acceptable for people to talk "the cloud" about it without understanding what it actually refers to.
Re: (Score:2)
Is it going to magically sync all your different data together so you can access it all the time?
Mine does. But then again, i run my own services at home, and do a regular sync to an off-site data store for backups, that i also own. ( in another state, just in case )
None are reliant on a 'free' storage provider like megaupload or some other such unpredictable system.
Re: (Score:2)
I'm seriously tempted to post this on the wall at work.
SSH not good enough for you? (Score:4, Insightful)
Re: (Score:3)
A pretty n00b friendly web interface.
Re: (Score:2)
Re: (Score:2, Insightful)
Re: (Score:2)
Freedombox (Score:5, Informative)
slashdot ate my last comment, so just check out the link [debian.org]
Re: (Score:2, Informative)
For more context: http://archive.org/details/EbenMoglen-FreedomInTheCloud2010
Re: (Score:2)
don't trust others... (Score:4, Informative)
the safest storage is your own high speed server quality RAID 7 write-only drive
Re: (Score:2)
and
http://yro.slashdot.org/story/12/07/13/1247257/facebook-scans-chats-and-posts-for-criminal-activity [slashdot.org]
and
http://www.zdnet.com/aussie-govt-aloof-on-cloud-csc-1339320492/ [zdnet.com]
You have the US gov telling you it wants in on your data, a US
The cloud is not looking great from any perspective, a
Re:don't trust others... (Score:5, Funny)
the safest storage is your own high speed server quality RAID 7 write-only drive
There's a readily available device for this that emulates a RAID 7 write-only drive but with better performance. It's called /dev/null.
Re:don't trust others... (Score:4, Funny)
Re: (Score:2)
Yes, but does it do dedup?
Fear not my lad! /dev/null/ is the pinnacle in write only technology. The leading edge, no less.
I can assure you in unambiguous wording that dedup is implicit in this wonderful technique. Indeed, duplicates vanish automatically.
SparkleShare (Score:5, Informative)
Try the free open-source SparkleShare software and roll your your own cloud 100%. That would trump any cloud provider option if this is your concern, since all the disks and PCs are under your ownership and control.
SparkleShare is essentially a DropBox clone in terms of a GUI, which extends to recovering older versions with a right-click. It looks like DropBox, and it works like DropBox too. But it is just a scripted GIT environment. In fact if you already have a GIT Repo hosted on a server (or service) somewhere, SparkleShare is easily configured to wrk with it. Here's how you start from scratch, assuming you already have PGP keys shared with the server:
At the server, create a new, empty GIT repository:
git init --bare NEWREPOSITORY.git
At the workstation:
Normally, you might use something like the following commands to work with GIT. (these are not necessary if you use SparkleShare)
git clone ssh://user@example.com:port/home/user/NEWREPOSITORY.git
cd NEWREPOSITORY.git
git clone ssh://user@example.com:port/home/user/NEWREPOSITORY.git
The SparkleShare config:
Add Hosted Project...
Address:
ssh://user@example.com:port
Remote Path: /home/user/NEWREPOSITORY.git
This document explains how to add a layer of encryption, (which also works to secure services like DropBox btw: https://github.com/hbons/SparkleShare/wiki/Encrypting-your-files-before-transfer [github.com]
Re: (Score:2)
Sparkleshare is nice but not ready for production. The second bug report in the issue tracker has the developers/users sharing thier code by dropbox of all things ironic.
Re: (Score:3)
Git is not designed to handle big binary data. Since Git is creating SHA hashs for each file, with a file 500MB and more it will take more time, also it will use up all the RAM to calculate the hash. In addition the size of the repository will skyrocket if you put revisions of big binary files, since you can't easily delete old files in a git repository.
Git is good for text documents and source code. But since even the Odt documents are binary blobs (the xml data is compressed with zip), you can't use git e
Re: (Score:2)
There is a variant of ODT which is a flat uncompressed XML file... That works well with git, also there is a plugin for libreoffice which saves your documents directly into a git (or subversion, cvs etc) repository (which i believe stores the data as dirs rather than zipfiles)...
Re: (Score:2)
What about using git-annexe to store large binaries?
Re: (Score:2)
Git is not designed to handle big binary data. Since Git is creating SHA hashs for each file, with a file 500MB and more it will take more time, also it will use up all the RAM to calculate the hash.
Git uses way too much RAM for some reason, but this is not it. Correctly done, it only takes kilobytes to calculate any hash, regardless of the size of the hashed data.
[...] Git is good for text documents and source code. But since even the Odt documents are binary blobs (the xml data is compressed with zip), you can't use git efficient with open document text files or other documents like Excel, Spreadsheets, etc.
My solution to that is to avoid file formats and tools which are hostile to version control. We knew decades ago that MS Office was a mistake; why repeat it?
git-annex assistant (Score:4, Informative)
This is what we are all waiting for, and it's already been funded! Just a matter of time until Joey finishes it: http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own [kickstarter.com]
Re: (Score:2)
I was attempting to install this, when I went with ownCloud instead. The reason? SparkleShare doesn't have Windows-sync clients that work on XP. That's a deal killer. Many small businesses have only XP machines. Yeah, it may be time to let it die, but what good is a syn-client that only works on half of your PCs. It doesn't matter if it is time to move on from XP, what does matter is that lots of people haven't.
ownCloud, however, was smart enough to make a Windows sync-client that works with XP.
So far, ownC
Real Cloud (Score:4, Informative)
If anyone actually thinks this question is in any way relevant, please let me know if there are other resources.
What CS definition? (Score:2)
Re: (Score:2)
Re: (Score:2)
Servers used to have names, now, in the cloud they are scalable service template instances
That is only one of literally dozens of uses of the word "cloud." Deploying virtual machines on clusters or grids is great, I agree -- but calling it "cloud" is about as useful as calling it "a thing."
It's not a buzzword, it's quite a complicated thing that has arisen from the abundance of hardware as a unit and the requirement that none of it be solely relied upon just to provide services
So now it is redundancy? Virtualization?
People who work with cloud infrastructure
Everyone works with "cloud" infrastructure today, because it is trendy...
This is why it's a cloud. It's not a buzzword after all.
In other words, because you think "cloud" should mean the thing you are using it to mean, it must not be a buzzword.
Re: (Score:2)
Sd card? (Score:2)
pull it out fo your phone and plop it into another device to import? If you're gonna pull all this retarded effort into the "cloud" why not just set up VNC and log into your computer at home and grab the contacts? You know something thats been available for over a decade.
My experience: possibly eGroupWare or SOGo? (Score:2, Interesting)
*snort* 27 posts so far and no one seems to really have addressed the poster's real question. (Instead, all I've read is basic suggestions like a file share, VNC/SSH, or OpenStack; all of which seem to ignore the main point: "is it possible to build my own personal data repository, where for example, I can store my contacts and calendars to sync to multiple devices?")
I've been looking for something like this for a while now, actually. From my research, I think the best way to solve this problem is to set
Re:My experience: possibly eGroupWare or SOGo? (Score:4, Informative)
I tried something like this last year using Linuxy solutions. For a midsized setup (30k users in groups ranging from about 30-500). For personal though I'm not sure it doesn't make more sense to just treat calendar and disk storage as two totally distinct problems and thus simplify the solution. Pick any of a dozen different internet calendar / scheduling services and do storage by itself.
But if you want to know the lay of the land as far as groupware:
1) I didn't go with Zimbra because at the time they were focused heavily on the rack server space and their longer term direction scared me. The cost per user was high for the commercial version and I did want commercial version features.
2) Scalix was really good 4-5 years ago. But is essentially now unmaintained. If you can live with broken compatibility and FireFox 3 for less than 10 users it is free. It has a very advanced calendar and an easy to use but powerful administration system. Really nice but I'd have a hard time going with a product that is now essentially dead.
3) OX (http://www.open-xchange.com/home.html) has what you are looking for. But understand that for whatever reason the app is not written MVC gui code is completely intermixed with functionality. It is effectively not much more changeable than a closed source program. They were working on this and by 2014 or so that likely will be fixed.
There were some others I experimented with if this is the sort of information you are looking for.
Re: (Score:2)
Re: (Score:2)
Well I played a round a little with Apple's open-source calendar server. That has webdav but really isn't designed for document sharing which seems to be something you are looking for. But in terms of a good quality calendar share only that's free it's excellent.
egroupware which you mentioned is in the process of going commercial. Stuff is being dropped form the community edition to make it more annoying. For me the killer was you couldn't save views. But it has been around.
These were not a fit for me co
Re: (Score:2)
Re: (Score:2)
I looked at funambol, but didn't like the idea of having to install a client on each device.
However, I do something similar with Zarafa...
Their old web ui was pretty ugly, but the new one is much improved...
It supports caldav (which many desktop clients and ios devices support by default).
It also supports activesync through the z-push plugin, which ios/android/webos/etc all support by default, and which will sync mail/contacts/calenders.
And there's another plugin i recently installed to get carddav support,
Re: (Score:2)
I'm able to view calendars shared by other users on the system, or are you referring to something else?
zarafa webaccess is the old interface, the new one is called zarafa webapp, although im pretty sure the old one let me share calendars too...
the comparison table does say "advanced calendar sharing", so perhaps there are some extra features i'm missing.
WD mybook live (Score:2)
My personal cloud. (Score:2)
My Personal Cloud. [tonidoplug.com]
Really not that hard (Score:2)
Grab an old box, stick some hard drives in it with some sort of RAID, encrypt the partitions and use rsync or similar for backing things up. Want extra redundancy? Use a USB drive or buy a cheap old tape drive off ebay.
Forward SSH to it and you have "Cloud Storage". This really isn't a new concept.
Cost (Score:2)
What is the cost of a roll-your-own cloud solution? Most discussions about the cloud miss out on the most important element, which is the cost. People use Google because it is essentially free, and gives you very decent reliability. I know you can make your own home server super reliable, but in aggregate, if 1 million people were running their own servers, compared to 1 million on google, I would bet that the 1 million on Google's cloud would do better on uptime in aggregate. The cost of trying to get to G
Use rsync, mysqldump and mysql replication (Score:2)
Run your own fail safe data repository. Companies have been doing this for ages and it isn't that hard nor expensive to implement it at a smaller scale for your own needs. No cloud needed;-)
Just use rsync, and something similar to mysqldump and mysql replication along with 2-4 linux nodes ideally hosted on different network/providers. You can host the nodes in VMs connected to regular consumer grade DSL or cable modem connections. You could make peering agreements with friends and relatives, I host your nod
personal cloud, like a pogo plug? (Score:2)
then there is this pogo plug thingy which does the same thing but for newbs who don't want to do the setup, and for cheap.
*yawn* (Score:2)
Doesn't this question get asked here like every other week now?
Personal Cloud Can't Exist, By Definition (Score:4, Insightful)
Using the wikipedia definition:
"Cloud computing is the delivery of computing and storage capacity [1] as a service [2] to a community of end-recipients.".
The whole point of a cloud is to abstract a massive underlying infrastructure to deliver some type of computing service (PaaS, IaaS, SaaS, etc ad naseum) to a large group of users and to be able to scale that infrastructure seamlessly. A "personal cloud" is an oxymoron.
Citadel (Score:2)
DIY? Go Remus (Score:2)
Re: (Score:2)
"Had you provided an insightful comment maybe then AC would have been a sensible choice."
Point out how it's not an insightful comment. Just you stating so doesn't cut it.
"Your post gave me that feeling I get when I see an old laptop that has covers missing and has been stripped of its hard-drive and RAM."
That's awesome. Now shit or get off the pot.
Re: (Score:2, Insightful)
Yea, those poverty-stricken, starving kids in Africa should keep their aircraft carriers, long-range bomber aircraft and unmanned drone fleets in their own fucking country!
Re: (Score:3)
I think he was talking about their population/resource imbalance.
Re: (Score:2)
So I wonder, "Why can't the U.S.Government send a billion dollars of aid to Texas in the form of Wind Turbines, and Solar Farms to power Desalination Plants?"
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
So do you propose a free cloud solution for starving kids in Africa?
Greetings from a least developed country [wikipedia.org];
Your words are so true! The absurdity of it all! After all, nobody ever used technology to improve their standard of living.
P.S. In case the sarcasm has escaped you so far: Fuck You.
Hugs,
The rest of the world.
Re: (Score:2)
How do you know a node isn't lying about how much storage it has, or just deleting files and saying it has them so it can have the privilege of uploading backup data?