Offsite data backup

"Please leave a message at the beep, we will get back to you when your support contract expires."

Moderators: phlip, Moderators General, Prelates

User avatar
DavidH
Posts: 80
Joined: Mon Aug 03, 2009 3:53 am UTC

Offsite data backup

Postby DavidH » Tue Jan 10, 2012 1:08 am UTC

Hello! I'm back again and looking for a bit of advice. The engineering company I work for is full of brilliant engineers, but the last computer system they worked with used magnetic memory and data tapes. They're paying some college dropout obscene amounts of money to handle their computer infrastructure, and they found out today that his solution to backing up their important data was to simply mirror the server hard drive on an external USB hard drive in the exact same room.

There was a flood, data was lost, and everyone is unhappy.

The boss came to me, and since I'm rather ignorant about this too, I've come to you. What we basically decided was to have an onsite backup and an offsite backup.

For the onsite backup, I was thinking a level of RAID. I've never used RAID before, but I recall there being a RAID level that interleaved the data and thus could handle a hard drive crapping out without losing data. Can anyone point me to some documentation about this?

I have no idea what direction to head with the offsite backup. Every plan I look at seems quite expensive. Here's the situation: There is about 400 GB's of data, but only about 50 MB of changes are made each day. Is there an economical way to store the 400 GB's and then only the changes?

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

Re: Offsite data backup

Postby masher » Tue Jan 10, 2012 1:57 am UTC

No real experience here, but...

Don't think of RAID as a backup; it's just a disk system, however, your backup system can use RAID. Have a look at NAS (Network Attached Storage) devices and UPSs.

I've heard good things about http://rsync.net/ for offsite storage.

.

I was looking in to doing this where I work, and I was going with nightly backups, and to reduce storage, daily diffs for a week, weekly diffs for a month and monthly diffs for a year.

User avatar
phlip
Restorer of Worlds
Posts: 7572
Joined: Sat Sep 23, 2006 3:56 am UTC
Location: Australia
Contact:

Re: Offsite data backup

Postby phlip » Tue Jan 10, 2012 2:03 am UTC

RAID isn't really a backup tool... all RAID will do is protect you from specific kinds of hardware failure - ie a faulty hard drive. A good backup system, on the other hand, should also protect against other hardware failing, and software failing, and human error.

With just RAID, if a user accidentally deletes or overwrites an important file, or if some software crashes and corrupts the data, then RAID will happily mirror the fault and you won't be able to recover anything. Whereas with a backup you want to be able to get your stuff back when that happens.

Which in no way means that RAID is a bad thing, it's certainly a useful tool, but it should supplement your backup system, not replace it.

Code: Select all

enum ಠ_ಠ {°□°╰=1, °Д°╰, ಠ益ಠ╰};
void ┻━┻︵​╰(ಠ_ಠ ⚠) {exit((int)⚠);}
[he/him/his]

User avatar
Steax
SecondTalon's Goon Squad
Posts: 3038
Joined: Sat Jan 12, 2008 12:18 pm UTC

Re: Offsite data backup

Postby Steax » Tue Jan 10, 2012 2:43 am UTC

A few things to consider:
- How quickly will new updates to the data need to be pushed to the offsite backup location? (i.e. how much can you afford to lose at any given time) Anything under a day will require a direct network connection, so that's one thing.
- What kind of data are you storing? How compressible is it? How secure does it need to be?
- Could you do with just storing a portion of your data (the critical bits) in the cloud/offsite?
- Does your company have the bandwidth/reliable internet to ensure you can access offsite data when you need it?
In Minecraft, I use the username Rirez.

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Offsite data backup

Postby EvanED » Tue Jan 10, 2012 3:00 am UTC

phlip wrote:RAID isn't really a backup tool...

Everything phlip said is true... I just want to point out that the typical statement is that RAID is a tool for increasing your availability in the event of a failure (as well as potential performance improvements), not for decreasing your probability of data loss. In other words, if a drive craps out you'll be able to continue running (or at least just have a brief recovery time) by restoring from the array instead of from offsite, but for the reasons phlip said it's not a backup solution. It doesn't protect you if your PSU commits murder-suicide, for instance.

My impression is that if you've got the budget, you should go for something like RAID 5 or 6. Nice recovery and speed benefits and low wasted space. Good hardware controllers are fairly expensive though.

User avatar
DavidH
Posts: 80
Joined: Mon Aug 03, 2009 3:53 am UTC

Re: Offsite data backup

Postby DavidH » Wed Jan 11, 2012 4:03 pm UTC

Ah, can you explain to me how you would do an onsite backup, then? I thought the ideal part of RAID is it protected against physical damage to a drive (In RAID10, RAID5, and RAID6). I was hoping there was a separate way to protect against data corruption through software, but I didn't know how.

Steax wrote:A few things to consider:
- How quickly will new updates to the data need to be pushed to the offsite backup location? (i.e. how much can you afford to lose at any given time) Anything under a day will require a direct network connection, so that's one thing.
- What kind of data are you storing? How compressible is it? How secure does it need to be?
- Could you do with just storing a portion of your data (the critical bits) in the cloud/offsite?
- Does your company have the bandwidth/reliable internet to ensure you can access offsite data when you need it?


It only needs to be pushed to the offsite backup location at most once a day. The data is reasonably compressible, and is very valuable but only to the company (no credit card / personal information really), so light encryption would be fine. The bandwidth is fine, and the internet is reasonably reliable.

The issue with the suggest offsite backup is how much it charges per GB. I'd much rather pay more for data transmission (If it's possible to only upload the changes, then there really aren't very many), then so much for each GB. The amount needed to be stored hovers around 400 GB, with mere MB of changes a day.

User avatar
Steax
SecondTalon's Goon Squad
Posts: 3038
Joined: Sat Jan 12, 2008 12:18 pm UTC

Re: Offsite data backup

Postby Steax » Wed Jan 11, 2012 4:27 pm UTC

The simplest onsite data backup is simply having a script that copies everything to a separate data storage machine at a certain time every day. It doesn't get any simpler than that. If you don't have any need to access long-past versions of data, this works out pretty well.

The simplest offsite data backup is to rent a server (or collocate your own with another company) and upload there, using whatever method you like. Say, FTP.

There are full solutions for both, if you need to track file revisions and access and so forth.

RAID is ideal for protection against physical damage, but that's only half of the point of a backup. Backups don't let you, say, restore a deleted file, or un-save a revision.
In Minecraft, I use the username Rirez.

User avatar
Bhelliom
Posts: 275
Joined: Wed Jun 18, 2008 1:30 pm UTC

Re: Offsite data backup

Postby Bhelliom » Wed Jan 11, 2012 6:26 pm UTC

But also keep in mind that RAID does not protect against flooded computer rooms, or other disasters that take out the whole machine.
You should have:
-RAID on the main machine to protect against random failed drives.
-RAID on the backup storage machine to protect against random failed drives. This machine should be physically separated from the main machine. Not in the same room that could burn or flood.
-Offsite Storage. This could be a USB drive you bring in to do a backup, then keep in a safe place offsite, or cloud storage, etc. what you choose here depends on your bandwidth availability, size of the backups, etc.

EDIT:
Back of the envelope calculation for cost of offsite storage/backup with JungleDisk on a Rackspace server would be $62.50 per month for 400 gb of data. It is $4 per month for an account and 15 cents per gb, the first 10 gb are free. No cost for uploading or downloading and it has encryption.
"Eloquently Blunt"

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Offsite data backup

Postby EvanED » Wed Jan 11, 2012 6:53 pm UTC

Bhelliom wrote:But also keep in mind that RAID does not protect against flooded computer rooms, or other disasters that take out the whole machine.
You should have:
-RAID on the main machine to protect against random failed drives.
-RAID on the backup storage machine to protect against random failed drives. This machine should be physically separated from the main machine. Not in the same room that could burn or flood.

Eh, RAID on the backup machine is overkill IMO. That would help if you have something that takes out the primary machine and some, but not all, of the backup machine's drives. Yes, technically it could happen. Is it cost-effective? For most places, I'd guess no.

What I'd like to see is:

  • RAID on the primary machine (helps with speed and single-drive failures)
  • A local backup machine (or drive array) which stores multiple incremental backups going back at least a couple months
  • A remove mirror of the local backup machine

For point of comparison, my CS dept has the following backup regimen:
  • Full backups are made twice a month, alternately saved for 1 year and 1 month. An exception is one backup from the end of each semester, which is saved for 2 years.
  • Incremental backups are made each night, saved for 1 month.

Remember that you do really want something like this; if you trash something important and don't discover it for a couple weeks, and you just store one image, then your backup system will have dutifully mirrored the trashed data. (This is just an extension to the statement about why RAID isn't backup.)

Carnildo
Posts: 2023
Joined: Fri Jul 18, 2008 8:43 am UTC

Re: Offsite data backup

Postby Carnildo » Mon Jan 16, 2012 8:12 am UTC

Steax wrote:The simplest onsite data backup is simply having a script that copies everything to a separate data storage machine at a certain time every day. It doesn't get any simpler than that. If you don't have any need to access long-past versions of data, this works out pretty well.

The simplest offsite data backup is to rent a server (or collocate your own with another company) and upload there, using whatever method you like. Say, FTP.

The simplest offsite data backup is the one the company I work for uses: three sets of backup tapes.

One set of tapes is sitting in the tape drive and ready for use (the onsite backup), the second set is in the sysadmin's briefcase to be taken home that night, and the third set is at the sysadmin's home to be taken to work the next day (offsite backup). You can do a similar rotation with external hard drives.


Return to “The Help Desk”

Who is online

Users browsing this forum: Google [Bot] and 7 guests