Remote Backup of Windows Boxes w/o Samba?

reezle asks: "I'm looking for a good (free as in beer) method to have my Debian server back up some remote Windows machines. There is no Samba in the mix; this is supposed to be a strictly secure internet thing. I've been toying with OpenSSH on the windows computers as a good tunnel, thought of simple tools like ntbackup initiated from a script on the Linux box, but not all of the pieces have come together yet. I need to have the Linux box make the connection, back up data (full and incremental backups) and have that backup data get back to the Linux machine in an encrypted format (across the wild internet). Has anyone done something like this?"

A few solutions

[moosesocks]

As mentioned earlier here, there are numerous SSH/SCP implementations for Win32. Search around. They're somewhat hard to find, but there's quite a few (make sure you choose one in active development. There are quite a few abandoned projects with security holes and other bugs). Honestly, I don't remember what one I use on my windows machine :) As a word of advice, do not use a cygwin distribution. They're somewhat buggy and need to run inside cygwin. The 'native' servers which link against some cygwin libraries seem to be allright though.

There's also another easier option. But, it will cost you. Use a "real" backup program such as Retrospect [dantz.com] which will do compression and encryption (very strong encryption if you desire) client side. More often then not, this is what big businesses use. You can then safely use smb, ftp, scp, whatever you wish

netcat

[bergeron76]

Use netcat (nc) [atstake.com]

A version exists for Windows (it's what the kiddee's use), but it can be used for legit purposes if done properly.

rsync

[DDumitru]

You can push the data using rsync. Load the Windows / cygwin version of rsync/ssh on the windows box. Then sync the whole box up to a debian directory over ssh. If you get fancy you can build rotation sets, etc. rsync is very efficient at moving only what changes, handling include/exclude lists, etc. Great for backing up lots of data.

Do a google search on "rsync windows backup".

The only downside is that rync will "kill" your upstream internet connection (it is that efficient). Fortunately, there is a --bwlimit option so you can back off a little.

If you are looking for a commercial hoster with all of the scripts pre-built we are happy to help (sorry for the AD), but it is pretty easy to roll this yourself.

Re:Might give Unison a Try

[FrenZon]

I've had fairly good experiences with the Unison product.

I am a huge fan of Unison, and I use it to sync all files, bookmarks and settings between my windows laptops and desktops. But it is NOT a backup utility - if you have it set to run at set intervals, and one set of files becomes corrupted, deleted or otherwise changed, Unison will then do the same to all the other files. It does not have rollback.

It's handy for total machine or disk failures, but not so handy if, say, a virus trashes your files and you don't realise until after you've synced those files with everywhere else, or for work situations where someone deleted the wrong directory, renamed the files incorrectly etc etc.

Re:Cygwin + rsync

[cookd]

Cygwin is not secure for daemons -- if user A is running a Cygwin program, you should assume that any user on the system can execute code as user A. This is due to the global shared memory that Cygwin uses to keep track of things -- it is accessible to all users and is assumed to be safe.

If you're going to use rsync, use Interix/SFU. That's what I currently use, though I might switch to Unison. Works great, though sometimes the permissions are a pain (the Posix permissions don't always translate nicely to Windows ACLs)...

Re:Three letters: SCP.

[cookd]

If you are copying 400 GB, you'll use 400 GB of bandwidth (less with compression, perhaps 200GB) each time you back up, whether or not anything has actually changed. That's very bad.

The advantage of stuff like rsync is that it only sends the deltas over the network. I have a daily backup of my 400 GB over the net. If nothing changes, it only uses 100k of bandwidth.

Re:Might give Unison a Try

[cookd]

You're taking it out of context, or maybe misunderstanding the feature. Some backup systems require OS support, so you have to install something setuid, set up a service, or hack the kernel to even get it to work. The idea is that this runs as a user (as do other programs like rsync or scp) and not as a kernel component (like a filesystem driver).

Your interpretation: This program magically allows any user to read any file on the system without admin privileges.

Correct interpretation: This program does not require admin privileges to install or run, and can run as any user. (Of course, the files that the program can access are limited by the user it runs as.)

Re:A few solutions

[zatz]

Cygwin has it's shortcomings, but I use it for exactly this task and it works well. Some tips for new Cygwin users:

You will need to run ssh-host-config after you install the ssh package. Under Windows Server 2003, it apparently can't setuid properly unless an actual password is supplied, so change the service to run as the user, instead of local system.

Cygwin is sometimes a little slower than you might expect, especially for I/O, due to the extra layers of indirection. One huge performance problem is that stat() reads the first block of every file. Non-blocking network I/O is also a little iffy; you may have to fiddle with the options to rsync to get things working in both directions.

Don't expect it to preserve NTFS ACLs. It does a decent job of simulating Unix-style file permissions, though. I think it uses the Posix subsystem for some file operations, because you can do strange things like create a file named NUL (which you then won't be able to remove within Windows).

In very rare circumstances, the games Cygwin plays with the filesystem interface will break rsync entirely. For example, if you have a file named SETUP.EXE, rsync will copy it fine. If you later create a directory named SETUP in the same place, rsync will get confused, because it will stat SETUP on the destination and see the file.

The Cygwin terminal is hopelessly non-standard. The supplied terminfo mostly works, but not entirely. This might be fixable, but I haven't tried very hard.

Bacula

[DeathBunny]

Use Bacula [bacula.org]. It's a GPL'd client/server enterprise backup software. It includes clients for most versions of Unix, OSX, and Windows.

Although the clients do not have built in support for encryption, according to the manual you can run the clients through stunnel [stunnel.org] to encrypt the traffic between the clients and the backup server. Future versions are supposed to support encryption built into the client.

Re:Might give Unison a Try

[Vlad_Drak]

You would clearly want to have LVM (or some other) snapshotting enabled on the destination Linux host's volume. If you did want to export the snapshots for user restoration, you could then share out the point in time (12hr, yesterday, DOW, M,..) as well.

Rsync with cygwin would be another option, my experience with OpenSSH/scp on Windows has been poor when it's under pressure.

-Chris

cygwin, bleh

[kayen_telva]

you dont need cygwin. putty on the windows machine can create a tunnel to your debian machine. do what you like from there

A solution: Acronis TrueImage

[Futurepower(R)]

NTBackup that comes with Windows can NOT backup all the Windows system drive, only part of it. Windows XP and 2000 (not Windows 98) have crippled file systems, apparently to implement copy protection.

NTBackup presumes that you are a peon whose time is worth nothing, and you don't mind loading all of your programs again. Some people restore a backup over a working Windows XP installation, but this is said by Microsoft technical support people to be unstable.

The ONLY way to back up a Windows XP and 2000 system drive is to do a sector by sector copy. See my Slashdot story and a discussion of this: Experiences w/ Drive Imaging Software? [slashdot.org]. The story gives a method of copying under Linux.

However, most sector by sector copies require that you have complete control over the drive.

The only solution I've found is Acronis TrueImage [acronis.com]. It costs $50, and it works, although there are some small flakinesses. You can backup a Windows XP or 2000 system drive while Windows is running and being used. You can send the backup over a network to a local Linux machine.

To send the backups over the Internet, you would encrypt the Acronis files with GNU Privacy Guard [gnupg.org] first. There is a way to submit the password to GPG in a batch file. Since anyone who has access to the password has access to the backup files, this is not a security risk.

Google has more links [google.com] to GPG.

I'm interested in hearing about any experiences anyone has with this.

Microsoft enthusiasts: Please don't disagree with the facts here, they have been verified many times by Microsoft articles and technical support people. Specifically, the NTFS file system is crippled, and NTBackup cannot back up the working system partition.

Re:Three letters: SCP.

[Drakon]

WinRAR can create archives of files that were modified after a certain timestamp

Re:Might give Unison a Try

[Sarunas]

that's incorrect. you can have it keep multiple copies\revisions of your backups.

rsync over ssh (cygwin)

[dimss]

We use cygwin rsync+ssh as for backup purposes and data exchange between remote offices. This is the best solution we've found so far. Each office runs rsync every minute. Data is transferred via Linux server.

There are drawbacks too. For example, you cannot store SSH key on mapped network drive. And do not use it with Windows 98, bash script and rsync will hang after 1-2 hours.

Re:Three letters: SCP.

[Zocalo]

Nope, you didn't miss anything as SCP has no incremental option. You could work around that by using a data compression tool to create an archive of just the files that have been changed, then SCPing that and uncompressing over the data mirror on the remote server. That doesn't help much if part of your data set is a big database file of which only a couple of records have changed though.

A far more efficient method would be to look at using RSYNC with SSH as a few others have pointed out. There's a pretty good HOWTO for Windows here [unimelb.edu.au], which avoids the overhead of a full Cygwin install. You'll probably want to check out the link to cwRSYNC [itefix.no] at the top of that page too.

Yes, I've done this.

[millisa]

I've put something similar in place which utilized the built in NTBackup as well as a combination of SSH/FTP. Scripting out NTBackup is easy enough; you just create the BKS file it'll use, backup using that file, and then get the file over to your linux system.

To backup the systemstate (you need to do this on your DC's and Exchange boxes), you have to do it locally.

Instead of putting in a directory in your BKS, just put the word: SystemState
on its own line.

Your exchange directory store:
DS \\EXCHANGESERVERNAME

You exchange information store:
IS \\EXCHANGESERVERNAME

Directories and files:
c:\whereever\blah\blah\blah
c:\anotherdir \blah

To exclude:
add a /exclude to the dir/file line in the BKS.

Note: The BKS files are in a sort of unicode format (thanks Microsoft). I was using perl to create the files before every backup so I didn't have to depend on changing static files on each system being backed up. Its not quite unicode . . . open the file in a hex editor and check it out if you want to write them dynamically.

To kick off ntbackup at the commandline and use that BKS, something like this would work:

ntbackup backup "@\\server\backupdir\YOURSERVER_SystemState.bks" /j "Helpful Description SystemState" /f \\server\backupdir\name_of_your_backup.bkf /n "Helpful Description SystemState" /d "Helpful Description SystemState" /v:no /r:no /l:s /m normal /rs:no /hc:off

Or for an incremental:
ntbackup backup "@\\server\backupdir\YOURSERVER_bunchadirs.bks" /j "Helpful Description BunchaDirs" /f \\server\backupdir\name_of_your_backup.bkf /n "Helpful Description BunchaDirs" /d "Helpful Description BunchaDirs" /v:no /r:no /l:s /a /m incremental /rs:no /hc:off

I was planning on exclusively using SCP to transfer the files between the systems, but I ran into problems with the Win32 SSH client and server. Anytime I wanted to move a file bigger than 2G off the Win systems, I'd end up with a corrupt file. (So, I had to resort to using FTP in some cases).

Only semi related, If you are backing up systems at a datacenter and eventually sending them to a repository in the office (or vice versa) and sending them over a limited data line (we only had a couple T1's), you might want to look at the shaper app. I was able to limit the traffic heading over that T1 line after it was conglomerated on a datacenter server so that the users in the office could still do their work.

After adding a few hooks to the various scripts to have it spew its progress to a central server that I monitor for errors, I had a fairly scalable script based backup system using built in backups (NTBackup on 2k/Xp/2k3 and Tar). Oh, and Bzip2 is your friend on the windows systems. Getting around 2GB file size limitations was the biggest pain in the whole setup. Go through some good QA and check every backup that is created for the first week; it sucks finding out your files are corrupted when you need them . . .

Why not just...

[Gudlyf]

Why not just install the ssh server and rsync (via Cygwin [cygwin.com]), then:

rsync -avz -e ssh winbox:/cygdrive/c/ /path/to/backup

Or one thing I use, to keep incremental backups, is rdiff-backup [stanford.edu].