Do You Need More Space for Your Media Needs? 105
ewanrg asks: "I have about 1/2 Terabyte of storage on my couple of home systems, and it's filling up rapidly with captured Home Videos and shows recorded off
my TiVO. I'm thinking that if I want to get through the next season of TV and the Holiday season at home I need to add at least a Terabyte of storage. My first thought was to use DVD-R (since I have a burner). However, if you assume that you use about 4.4 Gigs (in real terms) per DVD-R, then
you'd need 230 DV-Rs to hold about a terabyte of data. Inconvenient if you're trying to find which of 10 DVDs you put that episode of Futurama on - particularly if you recorded them as they came (over a few years) rather than wait until you could get them every night on Cartoon Network. I've also looked at the various NAS devices out there, but $8-$20K seems a bit much. What I'd really like would be an inexpensive drive or array I could hook up to my PC which has a S-Video out port. I could then use all sorts of
Media Library programs to find a file and play it. Can folks suggest something big and reasonably fast with an affordable prosumer price tag?"
Google (Score:2)
ATA RAID-5 and MythTV (Score:2, Informative)
Re:ATA RAID-5 and MythTV (Score:2)
Well it all depends. (Score:4, Insightful)
A dual P3 (second hand) fitted with cheap promise ata cards. Let linux combine them into raids and you got pretty cheap storage for home use. Sure the speed is not going to win any benchmarks but for home use who cares?
Only problem is that you can have a max of 3 promise cards. So that limits you to 16 discs.
Of course if you are an american you can now get pretty cheap 200gig drives. So that gives you a lot of storage even with raid5.
I do something similar (Score:1)
Re:I do something similar (Score:2)
hehe (Score:4, Funny)
Maybe the problem isn't storage space.... (Score:2, Insightful)
If you don't have time to watch it within the first week, are you ever really going to watch it? I think you're trying to create the modern equivalent of the "dusty box of old videotapes that I meant to watch one day".
Re:Maybe the problem isn't storage space.... (Score:4, Insightful)
Obviously you've never done the "oo I got a day off" M*A*S*H marathon.
I hate problems like this. The guy wants a problem solved, not a reason not to solve it. If he wants to build a library, let him do it. Frankly, I wish this technology had been around a few years ago. Shows come and go. It's damn near impossible to find the majority of Mystery Science Theater episodes that aired on Comedy Central. That's why the Digital Archive Project is up and running. They don't want that show to die just because Comedy Central wouldn't renew it.
Re:Maybe the problem isn't storage space.... (Score:1)
Yeah? Well what if I had an accident and suffered brain damage so unusual that the only way I could effectively communicate was through the use of unrealistic extremes? Huh? What then, Buddy?
Re:Maybe the problem isn't storage space.... (Score:1)
Re:Maybe the problem isn't storage space.... (Score:1)
Hehehehe!
Did you even try Amazon? (Score:2)
Most stuff some people intend to put in terabytes of HD will never be seen and nowadays you can find most stuff in a e-shop (or i-shop, he) somewhere.
Maybe you should get a life (Score:4, Insightful)
If we start a conversation based on "how do I organize all my crap", butting in with a lecture on the crappiness of crap is arrogant and offtopic.
I'm assuming, of course, that you don't have any little vices that you prefer to cope with rather than simply get rid of. Or am I mistaken?
Neither arrogant nor off-topic. (Score:2)
1.- Ignore most of it.
2.- Realize that you can buy most stuff in specialist shops or second hand.
The world is your library, the Internet is your index.
Re:Neither arrogant nor off-topic. (Score:2)
Re:Maybe the problem isn't storage space.... (Score:2)
This is 'informative'? Exactly how is this comment helpful? He's basically saying "you can solve the problem by losing interest in it." Glad he's not my doctor.
Re:Maybe the problem isn't storage space.... (Score:2)
I think that thr problem is that he should delete some stuff. I too am in the habit of saving masses of 'crap' and no deleting it, even when I never watch it again. It takes and hard drive to fail before I ever erase somthing (and that happens surprisingly often for some reason).
I have mountains of saved video, but I hardly ever watch it again, I just accumulate it (that's why I just bought an array of hard-drives - to store this s
The upper limit (Score:1, Flamebait)
And at the rate you're going, I envision you on your deathbed watching Sixteen Candles from its Oxygen broadcast in May of 2005. Your grandchildren will be asking you what a floppy disk is.
Build your own (Score:4, Informative)
Re:Build your own (Score:1)
That's about what I did: get some IDE disks (4 in my case), get a drive cage [proware.com.tw] capable of keeping those cool ) and hot-plugable, use a RAID-5 IDE controller [lsilogic.com] and I had, with 120GB disks, 360GB space. Not extremly fast, but convenient. That box is my main server keeping all home directories, including lots of (Divx re-encoded) recorded movies/shows.
Nowawadays I would skip the RAID controller as it's potentially the single-point-of-failure in my setup. And instead of 120GB disks, I would choose larger ones.
Re:Build your own (Score:1)
Soooo... Instead of a single point of failure, you would want to have four single point of failure ? Makes tons of sense, of course.
Re:Build your own (Score:3, Informative)
you would want to have four single point of failure
Hu? What I ment was: I've got 1 RAID controller doing some magic to create a RAID-5 out of 4 disks. If a disk fails, no issue. Gets replaced. No data loss. No downtime. Easy to understand, yes?
Now if the controller fails, I've got 4 disks full of data and no (simple) way of getting the data back off. (Yes it's possible, but dou you know how AMI/LSI store their RAID-5 data on 4 disks?) I'd have to buy another controller of the same brand (which is not a
Re:Build your own (Score:2)
Re:Build your own (Score:3, Informative)
You will also need a good RAID controller. 3ware makes the best IDE RAID controllers. An Escalade 7506-8 [3ware.com] would be good here.
Re:Build your own (Score:2)
I had a conversation in #debian on freenode about this very issue the other night.
I forgot who I was talking to, but he made a very good point about software raid controllers vs hardware raid controllers:
Before you shell out money on hardware RAID, try setting it up in software and running some tests/benchmarks. Figure out how much CPU usage that RAID 1, RAID 0+1, RAID 5, et cetera, array is using. Then think about if its worth buying a hardware RAID controller card.
You need to buy the drives
Re:Build your own (Score:2)
Hardware RAID is much more reliable. You can't boot off software RAID. If software RAID was that good, then no one would buy 3ware cards. If I'm setting up a 1 TB+ array, I'm going to spend a few hundred on a good RAID controller.
Re:Build your own (Score:2)
> software RAID was that good, then no one would buy 3ware cards. If I'm setting
> up a 1 TB+ array, I'm going to spend a few hundred on a good RAID controller.
Well, its always a tradeoff, but thats not a reason to dismiss software raid totally.
Hardware raid is easier. You can also boot off raid levels other than just mirroring.
With harware raid, you slap in many disks, and the card lets the system see one disk (the array
Re:Build your own (Score:2)
Just a suggestion, you might want to switch your home network to gigabit ethernet. That will make things run nicely and schmoodly.
Look at firewire direct (Score:4, Informative)
Re:Look at firewire direct (Score:2)
Since it uses software RAID I think it spoils it's primary app
Re:Look at firewire direct (Score:2)
Well, the RAID software that it comes with [raidtoolbox.com] sucks but you d
try these (should be obvious, but what the hey) (Score:3, Informative)
-Number your DVDs. Then keep a listing what's on each disk. If you're really 1337, create a small searchable database on your computer, complete with episode information.
-Dont record everything at top quality. Cartoony shows like the Simpsons or Futurama will take less space being recorded at medium quality, and that lesser quality is less noticeable. If loss-less compresion is available, use it. (Do any DVRs have loss-less available?)
Re:try these (should be obvious, but what the hey) (Score:2)
It does make it much easier to search through ar
Re:try these (should be obvious, but what the hey) (Score:1)
No, don't! Lossless compression on audio and video is unlikely to make a significant dent in the file size, and you're looking at ~1GB/min for uncompressed video. Even if you have half a terabyte of storage you're not likely to be able to keep much lying around at that rate.
A "database" is not really necessary for 1 person. (Score:1)
My file has grown to 12,000 or so lines (unique episodes/movies/clips).
I search it with grep.
I search it with a script (vchk.bat) that works on every computer in my house despite the fact that the text file is only on one computer. (Environment variables to hold harddrive letters helps one write scripts that work on all 3 computers by addressing the harddrive itself, ie %HD120G1,
Me personally? (Score:3, Insightful)
If I were out to archive TV, then I'd look at two approaches.
1.) Use a PC instead of a TiVO with a program like Snapstream to capture and encode the video using DivX in real time. You can get 1.5 hours per CD, and I think 9-10 hours per DVD. If you drop the resolution to 320 by 240, you'll do even better. There's a little suffering in quality, but trust me when I say you won't notice once the show starts. Now you only need a fraction of a terabyte.
2.) Similar to step one, only use the TiVO (or a Replay with a network out) to capture the shows and transcode it into MPEG 4. The quality will be better than the previous approach, but you'll encode the same video twice. Personally, I don't think it's that big of deal.
There are considerations here, though.
- Playback of DivX files to TV is *almost* there but not quite. (makes you ache for a cracked XBOX, doesn't it?) On the flip side, though, these shows will easily travel to your laptop and PCs. I've done this before, and it was DAMN COOL to have several episodes of Quantum Leap to watch when I went on a 5 day business trip.
- Video quality probably won't be as good as captured with the TiVO. It has superior capture nad playback equipment. I can't help you there, but I can tell you that you won't notice after a while. I have a bunch of QL eps recorded at a strained bitrate, and they all came out wonderful. At first glance it's blocky, but once you're immersed, it just isn't noticed anymore.
- I don't think this would be ideal for home movie capture. For that, I recommend a digital video camera with firewire.
- Step 2 involves automation and extra processing. You might feel that after a while.
Personally, I'd rather go this route at the sacrifice of some quality than to try to get a terabyte of storage going. With 250 gig drives floating around, it's not all that challenging or expensive to do, but that is a backup nightmare.
Re:Me personally? (Score:2)
Sure it is. Have a look here. [kiss-technology.com].
Plays DivX (3.11, 4 and 5), Xvid, mpeg, whatever else you can toss at it. Has ethernet and streams all formats over it directly from your computer. Also plays mp3s, wmas, flacs, wavs and oggs in the same way (or from a burnt cd/dvd ofcourse).
And as if this was all not enough, the thing is StrongArm-based and runs Linux (though you'd never know unless you poked at it), making it very hackable.
About $250.
Re:Me personally? (Score:2)
I have been looking for just such a device and $250 sounds like a good deal, if only i could see more details
Thanks
Re:Me personally? (Score:2)
Kiss Technology ("Keep It Simple Stupid", nerdy enough connections rigth there) are selling the DP-series of divx-enabled dvd-players.
There's the DP-500 that I have, ethernet, no harddisk, no tv-tuner.
Then there's the DP-558, same as above, but with harddisk and tuner, so it's able to act like a PVR in addition to streaming media over the ethernet-link.
Recently they've started a new series, the DP-1000 and DP-1500, those are only available from medio october, allthough I saw a few prototyp
Silicon Mechanics (Score:4, Informative)
http://www.siliconmechanics.com/
Specifically:
http://www.siliconmechanics.com/c221/storage-se
You might even be able to order just the chassis, controller, and disks... but you'll have to figure that out on your own. We buy all of our stuff from them.
Promise External ATA RAID (Score:2, Informative)
Over a year ago it cost me about $5K, including a SCSI card. Today it would cost me a lot less and I could have more then a terabyte.
Both the Promise RAID box itself has been reliable, and I am quite happy with the WD hard drives.
-- H
8 ATA drives, raid 5, whitebox linux server (Score:2, Informative)
slashdot profit format ; man 9 style (Score:2)
2. s/Maybe I have myself a business plan here.../???/
3. Profit!
i believe you do!
DIY or Xserve (Score:2)
Realistically, if you're buying a terabyte (tibibyte?) or more of space, you have money to spare. Just buy an Xserve and Xserve raid unit, turn on NFS or SMB, and call it a day. They're brain-dead easy to set up, relatively cheap (not much more than building
Re:DIY or Xserve (Score:2)
Cheap? I don't think so. The Xserve RAID with 1TB of space (7 180GB drives RAID5) is $7500 alone. The fibre channel card you need to put in the Mac connected to the RAID unit is another $500. If you want more than a 1 year warranty, that's another $999. If you look at the configurator, Apple wants $500 for each 180GB PATA
Re:DIY or Xserve (Score:1)
Ok so I'm nitpicky, it's tebibyte [wolfram.com] not tibibyte.
TiVo - Transcode TySteam to MPEG2 - DiVX5 (Score:5, Informative)
Deinterlacing television is a pain, and I think that's why a lot of people go down the MPEG2/SVCD route (it handles interlacing natively.) I've found three solutions for AVISynth that are pretty decent:
1) Using SmoothDeinterlacer (visit www.100fps.com for more info on that)
2) Using DeComb - http://www.neuron2.net/decomb/decombnew.html
3) Using DGBob - http://www.neuron2.net/dgbob/dgbob.html
Anyhow, let me know if anyone needs help. I'm going to write a guide on this soon and put up a website detailing my steps.
Re:TiVo - Transcode TySteam to MPEG2 - DiVX5 (Score:3, Interesting)
And, once you've modified the video off the Tivo, how do you view it? The Tivo is not gonna handle your ultimate compression scheme. Do you just view it on a PC?
Re:TiVo - Transcode TySteam to MPEG2 - DiVX5 (Score:2)
Make your peace with God! (Score:1, Troll)
Re:Make your peace with God! (Score:2)
Get a media database (Score:4, Insightful)
What i've done... (Score:2)
Re:What i've done... (Score:2)
Sure do; Sony has one for about $799.
DVD-R actually works out pretty well... (Score:2, Informative)
The selector cases are actually cheaper than retail leather like music folders if you buy generic instead of discgear. As for the media, Ritek makes double sided DVD-Rs that are both cheap and reliable. I have over 100 burned with zero problems accessing later, although I
Re:DVD-R actually works out pretty well... (Score:1)
Anyway... I'm wondering... Do you have a URL for these selector cases? or a picture?
And also.. are the Ritek double-sided DVD-Rs more than twice the cost of single-sided DVD-rs? currently i pay about 60 to 90 cents for a blank DVD-R (buy 200 at a time).
Just buy it already (Score:1)
That way, everything is boxed and labelled.
Plus, if DVD-Rs are anything like CD-Rs, I don't know if I'd be willing to trust them for extended media storage (I've had several CD-Rs crap out on me after only 3 to 4 years).
When it comes out on DVD... (Score:2)
How about Bullshit (the Penn & Teller show that was on Showtime)... or even Penn & Teller's Sin City Spectacular?
Buying on DVD works great when it actually comes out on DVD, even if you might have to wait a few years, and you tend to get special features, commentary tracks, etc, but not all of 'em get released. How much of a market is there for Mad Jack the Pirate? Highwayman? A
Re:When it comes out on DVD... (Score:3, Funny)
October 28th for the first season in the US. Go check Amazon... hope that puts a smile on your face
Hard drives are cheap (Score:2)
If you don't need access to your data at all times, you could get away with one 4bay enclosure + removable trays for each drive, then pop in the drive you need.
I have put together a music system for my home stereo based on a computer and
Re:Hard drives are cheap (Score:2)
I think you hit on the solution for this guy. He doesn't need it all online all the time, he should use some kind of hot-swappable drive bay. The question is how cheap can you find the drive trays, sometimes those things cost as much as the drives themselves (well, not if they're 200GB drives).
Re:Hard drives are cheap (Score:2)
if you can build up your drives slowly, you can take advanage of rebate deals and pick up 200gb drives for $130 or so...
Backup? Wat meen dis wurd ... Backup? (Score:2)
Anyways, with RAID-1 you have insured through redundant hardware that a single drive failure will not destroy your data - but you haven't actually backed it up. RAID won't protect against software or wetware problems (del *.bak somehow becomes del *.* before you get your daily recommended allowance of caffeine, or what have you.)
As an extension to the discussion (because the
Dare I say it... (Score:1)
And since it includes redundant power supplies and [I believe] some surge protection, it shou
Say whut!? (Score:5, Funny)
I'm thinking that you've got a serious problem to deal with...
What when it breaks? (Score:3, Interesting)
Once you have all this storage, what are you going to do when it is all lost. Houses burn down, harddrives crash, CDs get scratched, kids take hammers to electronics, and other disasters that I can't even think of.
Answer that question first. If you just want the data, but don't worry too much about losing it, then 5 harddrives in a simple RAID without parity (I can never remember if that is level 0 or 1 - the other is mirror) will do just fine. If you care about losing data, then do you need offsite storage? If you need storage offsite, tape backup looks good. (perhaps cheaper than CD/DVD at the volumn you are looking at, and certinaly takes less space) DVD is nice in that you can write your videos in DVD format, and borrow a copy to anyone who wants to see your kids birthday party. However it is easy enough to burn a custom disk for anyone who wants it.
Have you looked at nearline robots? They are more expensive than harddrives, but the worst case in the case of breakage [that doesn't take the house with it] is you loose just a small fraction of your collection, and nothing gets scratched on handeling. If your dvd drive in the reader breaks you can still use the collection. Some allow you to hook several different drives to different comptuers, if IO bottlenecks are a problem for you this would allow more people to use your collection at a time. May or may not be useful, but you should consider it.
Xserve Raid (Score:2)
That said, you might want to reconsider saving all of those old TV shows. How often do you actually go back and watch them again? I was amassing a nice DVD collection until I realized that 90% of them I have never watched more than once.
Re:Xserve Raid (Score:2)
Other shows, though, like MST3K and the like I archive because they'll most likely never release all of the episodes to DVD.
Raid 5 Solution (Score:1)
With eight 160GB drives on a HighPoint 454 RAID 5 controller you would have access to just over 1TB (1024^4 Bytes) after Formatting.
I've ssen Maxtor 7200 RPM, 8MB buffer, ATA133 drives going for less than $90 after mail in rebates. With the controller your looking at under a grand, and there's nothing stopping you from buying drives bigger than 160GB.
And of course, since it's RA
Go Outside (Score:1, Offtopic)
some base hardware (Score:3, Informative)
Western Digital 250GB SATA 8 MB Cache 7200 RPM $325.00 QTY 5 [Using RAID5 gets you close to 1TB]
Sub-total $1625
3Ware Escalade 8506-8 Serial ATA RAID
$490.00 QTY 1
SuperMicro SATA Mobile RackCSE-M35T1 [supermicro.com]
$140 QTY:1
Total $2255+tax
The SuperMicro "RAID cage" holds 5 1" SATA drives in the space of 3 5.25" bays. I haven't found anything else that packs this many drives in such a small space. I'd be very interested to hear of people's experiences with this or other RAID cages.
If you have a big enough case, you could add this to your existing computer and be good to go. If the case isn't big enough, just get a bigger case and move the guts of the computer into it, like a hermit crab [xs4all.nl]
Alternately, you could buy/build a cheap computer with 4 5.25" bays (need one for the optical drive) and use it as a file server. Budget about $500 for it if it's really dedicated to just serving files, you can skimp on the processor, video card and the little extras. I would choose Linux for the file server but Windows would probably be okay if your main OS is Windows (but then you have to buy a Windows license which skews the cost of the file server). You would probably want to spend a little extra and get a extra pair of gigabit Ethernet NICs, one for the server, one for your desktop PC.
The whole thing should be around $3000 which is not too shabby. It could be even cheaper if you used smaller drives but more of them.
5 250GB @$325 = $1625
6 200GB @$260 =$1560
8 160GB @$156 = $1248
The 8 drive option would probably require bigger (more expensive) case than the other two.
For my project I'm planning on getting a 7 bay case and the 3Ware Escalade 8506-12 so I can just buy 5 more drives and another RAID cage to move up to 2TB. Woo!
Re:some base hardware (Score:2)
1. SATA is overpriced.
2. Other companies make cooler 5400rpm drives that are just dandy for media storage. I personally recommend Samsung drives. Someone will undoubtedly chime in that Samsung sux0rs, but I've had amazing luck with them, they're dirt cheap, cool, quiet and can be found in both 5400 and 7200rpm models, some with 8MB cache (though those don't match the performance of WD *SEs).
3. PATA 3ware controllers are cheap on ebay, if you're
Re:some base hardware (Score:2)
I don't have a problem with Samsung but they don't have anything bigger than 160GB at this point. 8 drives for ~1TB makes it a lot harder to leave room for a 2nd terabyte in a case. Cheap is goo
Re:some base hardware (Score:2)
Got a Mac? (Score:2)
Unlike most of the people above, I can appreciate the need to collect, and to have a perfect collection, so I won't chastise you for having so much media. (Although, I do prune my mp3/mp4 collection often now, as my 30gb laptop HD just can't handle me.)
Something I've been doing lately: I buy cheap hard drives (I'm seeing 200gb for about 100$ US , though I'm living in Tokyo currently), and external firewire cases. Any case is fine, as long as you've got two plugs on the back for daisy chaining. This way
Windows XP and 2000 Have Software RAID Built In (Score:2)
Windows 200 Pro and XP have software RAID 0 and RAID 1 capabilities built in. Should be no problem with FireWire drives (though I haven't tried it).
- Greg
Mod me flamebait... (Score:1, Offtopic)
I haven't had cable in years, and y'know what? When I'm at friends' houses (yeah, I do have friends
The wierd thing is that the longer I don't have it the more I hate it when I do see it. Like reverse withdrawal symptoms or something. TV programming nowadays really is horribly poor quality but it's hard to realise that until you step away for awhile.
My way (Score:2)
I have two Windows boxes with ATI All-in-Wonder cards that're m
Re:My way (Score:1)
Re:My way (Score:2)
Accomplish something (Score:1)
Cooling the disks (Score:1)
Re:Cooling the disks (Score:1)
Re:Cooling the disks (Score:2)
my setup (Score:2)
ReplayTV + DVArchive. ReplayTVs have the ability to play shows recorded on other ReplayTVs, and DVArchive masquerades as a replayTV. Runs under Windows or Linux (and OS X, as well, I believe), and can be queued to automatically copy, transfer, or delete shows at scheduled times. Your replayTV will simpl
simple (Score:1)
Solving your Futurama indexing problems (Score:2)
Re:Solving your Futurama indexing problems (Score:2)
It's been a while since blank discs were $7 apiece.
Also, rather that recording stuff from the Tivo, use Netflix and/or Wantedlist (for pr0n), and never pay for an actual DVD again.
Re:Solving your Futurama indexing problems (Score:2)
Re:Solving your Futurama indexing problems (Score:2)
136G Limit (Score:1)
Terabyte of 7200/133 on Firewire 800 < $1400 (Score:2)
$79.50 at new-egg [newegg.com].
$169.98 each at cooldrives.com [cooldrives.com].
$230 each according to Pricewatch [pricewatch.com].
Total: $1339.46 plus Tax and/or S&H
- Greg
setup. (Score:1)
Re:Donate it to charity (Score:1, Funny)
Yes, but these are immunizations against food.