How many copies of static data are you backing up?
by Jesse on May.13, 2007, under Backup, General
So I’m going through my MP3 library and the backups, and got to thinking.
How many times do you need to back up static content?f
I’ve got about 100G, but an MP3 library, like a lot of static storage, it doesn’t change much.
This means that every time i run a full backup of the drive, I’m backing up the same data every week. Since I’ve only got DLT-7000 drives, that means a full backup of my environment is taking 4-5 tapes.
The alternative is to do a full backup of the data once, and keep doing differentials going forward.
It makes single file or directory restores very easy, but makes it a nightmare if I ever have to restore the whole drive.
It also makes for an expensive backup plan, because you never recycle tapes.
10 Comments for this entry
1 Trackback or Pingback for this entry
-
Storagezilla
May 13th, 2007 on 5:00 amWhen not to bother with backups
If we could visualize backup recovery I think it would look something like the Kraken from Pirates of the Caribbean: Dead Man’s Chest. It that’s much of a monster. Seriously. SanGod is wrestling with how many copies of static (Fixed)
May 13th, 2007 on 3:52 am
Archive it. I keep static content on a RAID 1 set which isn’t backed up.
May 13th, 2007 on 4:04 am
*cough*vtl*cough*
May 15th, 2007 on 10:40 pm
This is why i don’t use tapes.
I just mirror to another drive on my LAN. And important things like my pictures exist on two systems at my house, untold copies on Flickr, and on my webserver, which is also mirrored hourly from OH to CA.
And anything that is easily replaceable, like most MP3s or DIVx movies, I don’t bother with. Some things, like the Complete Beatles catalog, I keep on CD as well. And I’ve put a lot fo my movies on DVD-R just to ise whilst flying.
May 16th, 2007 on 8:45 am
I actually got the VSS configured so that I get a snap at midnight every night, they are Raid-1 volumes so that’s probably the best way to do it.
I’ll put a couple of external 250G drives to periodically copy the information to either using Replistor or my favourite RoboCopy.
May 18th, 2007 on 7:43 pm
You need a system that can do deduplication
We just purchased a deduplication system for both our production and DR environments that really helps us with storing multiple full backups. When data is written to it, it locates identical small chunks of data and essentially stores a pointer to the identical chunk of data, and only stores the chunk once. This has the effect of compressing the amount of data by 25x or more.
Hopefully some day this type of technology will make it into the consumer market in the form of a file system that can do automatic deduplication or something like that….although not sure how technically possible that is to accomplish.
May 19th, 2007 on 10:19 am
That sounds almost exacly like the sales pitch I got recently for Avamar.
The problem is that when you’re trying to sell the idea to the brass, billing based on the “before” footprint of the data, it’s not cost effective. Now if they would start the fee schedule based on the “after” model I could probably see a use for it.
When it comes down to it, it’s still cheaper to throw more media servers, tapes and drives at the problem.
What seems to make the most sense to me is the comment above by both danisaacs and Storagezilla, archive it to a WORM system like Centerra and forget about it.
For my home use, I just bought a 250G SATA drive for $82, put it in an external case, and I do a nightly sync of the parts I need to keep. No more tape copies of stuff that never changes.
May 22nd, 2007 on 7:13 am
What about using something like Amazon S3 to do a offsite backup of your tunes? iirc the pricing wasn’t bad, it would probably end up being cheaper than buying tapes. I think the initial transfer of all of your data might cost a bit, but once its there the incrementals shouldn’t be bad. I found a list of backup tools that work with S3 here:
http://jeremy.zawodny.com/blog/archives/007641.html
May 22nd, 2007 on 7:18 am
Looks like as of June 1st, its $0.10 a GB month for storage, so if you only had 100GB you’re looking at $10, which isn’t too bad. I guess it depends on how much you value your mp3 collection and worry that Amazon may tell the RIAA what you’ve got
May 22nd, 2007 on 12:06 pm
Heh. Netapp has been doing de-dup for more than a decade. They just call it “snapshots”.
You might want to think about S3, too. A friend of mine (who actually owns the webserver my site runs on and manages the cross-country hourlies) switched over to Amazon’s S3, and figures it saved him a nice chunk of change. Smugmug also uses it.
My buddy’s reasoning is on his blog:
http://jeremy.zawodny.com/blog/archives/007624.html
May 22nd, 2007 on 1:13 pm
The RIAA can look all they want. I can now honestly say I have a CD to back up every song in my library. (Back-up? Only if I want to spend three weeks re-ripping everything)
After the massive delete I just went through I have more than enough free space to maintain a second disk copy on a USB removable as well as to let my iPod act as the third, since it now stores most of the music as well.