Data Storage For Home? 114
kuom asks: "Every couple of years, I face the same problem: running out of hard drive space. No matter how big of a hard drive I get, I seem to find ways to fill it up within a few months. The size of my hard drive grew from 2.5G to 13G, to 20G, to 40G, 80G, 120G, and most recently, 200G. Today, I have a combined hard drive space of 280G, but I again find that I only have about 2G of free space left. My collection of family photos, web site content, TV episode captures, music files, and my archive of ISO files for various operating systems, they just eat up my hard drive space so fast. I could get a 400G hard drive, but I figure maybe it's time to think about something long term, something like EtherDrive or StorageWare. But the price tags are definitely out of my range. Slashdot readers, what do you suggest for home data storage?"
DVD (Score:5, Insightful)
Spend $100 and get yourself a DVD burner. Don't just use it for backup, but actually move things that you don't reference all the time--ESPECIALLY those ISOs you almost certainly don't need live--off of the live storage. For things that are important / irreplacable, make several copies. Distribute them far and wide to friends and family if you want.
Re:DVD (Score:4, Informative)
Re:DVD (Score:2)
Stick with the single layer media, even if your DVD drive is dual layer, unless you've got something that needs to be on one DVD.
Re:DVD (Score:3, Informative)
Re:DVD (Score:3, Informative)
Labels can be bad too. [techbuilder.org]
There are also some other good tips for archiving CD/DVD type media in that article.
-- Pete.
Re:DVD (Score:4, Interesting)
This is true. I had several CDs I had burned a few years ago that were unreadable. I removed the labels by soaking the disc in isopropyl alcohol and after that, I had no problem reading the discs.
Re:DVD (Score:1, Funny)
Next time, try putting the labels on the other side of the disc...
Re:DVD (Score:5, Interesting)
When you write on the silvery label you're actually writing on the back of the recording medium.
Of CDs: true. Of DVDs: False. DVDs sandwich the recording medium between two layers of substrate, rather than having it sprayed on the one side of the disc. This has several benefits:
On labels, we have a problem here at my workplace in that we receive data from a certain other department on DVD. Regulations require that the DVD be labelled in a particular fashion (doesn't say use a label, does say $wordy_legal_boilerplate must appear on the disc) and the other department refuses to cease using sticky labels to do this. Our department all use laptop computers (because we all have to work weekends from home occasionally) and the sticky label causes the disc to warp slightly as it heats up in our machines, thus causing it to stop being legible after about 15 minutes unless you keep it 100% spun up the entire time. Labels on optical discs are evil
Re:DVD (Score:3, Informative)
This is true for CDs, but not DVDs. They have the data layer in the middle of the plastic bulk, whereas CDs have it on the label-side surface.
You can see this clearly if you microwave coasters of each :) The shiny layer of CDs begins to strip away, whereas that of DVDs stays inside the plastic.
Re:DVD (Score:1)
Welp, guess I better be looking into getting the good stuff back off of them.
Re:DVD (Score:4, Funny)
Re:DVD (Score:2)
Re:DVD (Score:2)
Re:DVD (Score:2)
Re:DVD (Score:1)
4 GB is half the size of some folk's entire hard drive. It should be more than enough to back up your entire OS, possibly your entire "bin" directory if you keep your PC neat.
But you're right. A more serious/lazy/gadget-happy geek would be better served buying either a 200 GB portable HDD or a tape backup system in addition to the DVD burner. Both of the previous have the problem in that they simply don't work for offline data archival--it's not their intended p
Re:DVD (Score:2)
"It's a horrible pain in the ass to keep anything archived on DVD."
No shit. I can't believe someone seriously gave that advice, and actually got modded up for it. The man has 280 GB of data and is considering getting another 400GB hard drive. Somehow I don't think DVD's are going to cut it.
What he needs (and what I need too, for that matter) is cheap network attached storage for the home, but unfortunately I haven't been able to find such a product so far. I hope some manufacturer realizes the deman
Check out the 'Burly Box' (Score:1)
http://macgurus.com/productpages/sata/sataguide_1. php [macgurus.com]
The 'product page' is at http://macgurus.com/productpages/sata/satakits.php [macgurus.com]
Prioritize what you absolutely need (Score:5, Insightful)
Where it's electronic bits or physical items, some things are more important to you than others. Take a long hard look at what things you absolutely need, and toss the rest. Will your life be that must worse if you didn't have "______" within easy reach at any given moment? Probably not. And you'll feel better knowing that the things you do keep are the important ones.
And don't think of it as parting with things you'd rather have kept. Think of it as making room for more new stuff.
Good luck.
Tom
Re:Prioritize what you absolutely need (Score:2)
How to do it? Easy, make a "long term archive" folder of stuff you *know* you'll want someday, everything else if you haven't touched it in a year (or three or five or whatever arbitrary cutoff you decide on) - delete it!
Delete? Never!! (Score:4, Insightful)
Take a long hard look at what things you absolutely need, and toss the rest.
Not me. For two reasons. First, I've too often found myself wishing I had something that I had deleted and second, it is simply not worth my time to wade through it all and decide what I need and what I don't. Disk space costs well under 50 cents per gigabyte, and even after you add some redundancy (RAID), it's still very cheap.
Re:Delete? Never!! (Score:2)
Something that can easily be prevented by a well thought out short/medium/longterm storage policy.
and second, it is simply not worth my time to wade through it all and decide what I need and what I don't. Disk space costs well under 50 cents per gigabyte
The bigger the disk, the longer it takes before you get the dreaded 'disk full' message, but it will take exponentially longer to recover from it. In other words, you can't prevent
Re:Delete? Never!! (Score:3, Interesting)
The bigger the disk, the longer it takes before you get the dreaded 'disk full' message, but it will take exponentially longer to recover from it.
Who cares? The last disk I added took three days to be fully on-line. But my investment in the process was about 30 minutes. It took 20 minutes to write the script to iterate through my RAID-5 volumes, run "pvmove" to migrate the data off of each volume, remove it from the volume group, rebuild it with the new disk and add it back into the group, then anothe
Re:Delete? Never!! (Score:2)
I meant recover from a disk full condiion, not recover from a failure or adding a new disk or whattever.
Tthere are momenets when a new/extra disk is not an option, ie, you need the space now, and there is no shop open, your storage enclosures are all filled up already etc.
The more capacity you have, thte longer itt takes to figure out how to make the inmediately needed space available.
Re:Delete? Never!! (Score:2)
Tthere are momenets when a new/extra disk is not an option, ie, you need the space now, and there is no shop open, your storage enclosures are all filled up already etc.
Only if you let that happen. It's not hard to stay ahead of your needs.
Re:Delete? Never!! (Score:2)
Not to mention that noone can see into the future, all you can do is predict when your disks will be full based on current usage. That is never gonna give you an absolutely correct picture.
Re:Delete? Never!! (Score:2)
There is a limit to how much storage you can attach. That limit changes a bit depending on how much you are willing to spend. Staying ahead of your needs is simply not always possible.
Unless your needs include very large amounts of video, I disagree. Disks are so large these days that it's pretty easy to stay ahead. When I can start ripping HD video from Blu-Ray disks, then you may have a point. Then again, perhaps fixed disk storage will keep up. Or not lag too far behind.
Not to mention that noon
Re:Delete? Never!! (Score:2)
Or when your needs include saydoing multitrack audio recording? or you want a library of your music in a non-lossy format on your disks, and you happen to have a somwhat substantial amount of music?
Or you happen to be
Re:Delete? Never!! (Score:2)
Or when your needs include saydoing multitrack audio recording? or you want a library of your music in a non-lossy format on your disks, and you happen to have a somwhat substantial amount of music?
No, I don't think audio does it. Consider that 1TB will hold over six *thousand* hours of FLAC-compressed CD-quality audio. Unless you're a professional recording studio, there's no way you'll have multiple terabytes of audio.
Video is different, but it still requires a *lot* to push the limits.
Or you ha
Re:Delete? Never!! (Score:2)
First of all, I know quite a few people who do have that much music around, you obviously are not among those.
Second, I don't have to be a professional recording studio for doing things like uncompressed multitrack recording, one of the very cool things about modern computers and sound
Bad Avice (Score:2)
Adding drive space is cheap and easy (USB storage can be added for well under $1/GB, takes a second to install, and will last for years). It would be quite easy, across multiple hard drive clean ups, to spend more effort organizing and deleting your hard drive content than it costs for you to simply add more
I took the easy way out (Score:2)
Not all that big (only 200Gb total) but if I got 3 of them HDs as payment on an old debt. I'm sure that if I had invested a little bit more I would have bought 4 SATA drives and rigged them up. Software raid isn't quick but it'll store stuff with the added bonus of allowing a drive to fail without losing anything.
If you're into spending more money for better performance, buy a SATA RAID (levels 0,1,5) controler, there are a few on the market that aren't ove
Re:I took the easy way out (Score:5, Insightful)
Repeated again:
RAID5 will fuck you if you depend on it to be your ONLY failsafe on your data.
Your motherboard/controller could screw you. You could delete some files. 2 drives can fail at the same time (power surge,etc.)
There's just no excuse for not backing things up. I personally have a DDS3 tape drive in my file server for once yearly backups. Every 6 months I do a set of rewritables (DVD+RW). Every year or so I make a permanent copy on DVD+ or -R, and I buy decent burnables for that.
I've had instances where controllers, cables, and my own screw ups have lost data. But the cost to my time is minimal since I have backups in place. The way I figure it, the time I spend safeguarding my data is worth its weight in gold WHEN I have to depend on the backup for critical data.
Besides, Corporations back up their data - why can't we?
What's your price range? (Score:4, Insightful)
Or you can just delete the ISO's and TV episodes, if they're not worth the extra cost and you're never going to use/watch them again anyway.
Whatever drives you get, make sure you research their quality first, especially if you don't care for the extra cost of a redundant raid and/or backups.
Try not getting illegal P2P movies (Score:2)
Or you could try a tape drive, but large DLT tapes can be expensive.
I saw an automatic DVD changer. Its a robotic library that can automatically select one dvd out of 200 and load it. Thats 800GB. Get a DVD burner and one of these babies. Got more money? Get a large chassis and just add 500GB hitachi drives as you need them.
So I guess eventually you're back to just buying the largest harddrives, the least expensive and most convenient thing next to cutting down on fat in
Re:Try not getting illegal P2P movies (Score:1)
Mass storage on the cheap. (Score:2, Informative)
Re:Mass storage on the cheap. (Score:2)
You set up
My preference is to then have a root volume group that is also raid-1 to hold the OS and swap partition (you don't want to swap to raid-5). If you have 4 drives, then set up raid 1+0 for your root VG across the first slices of the drives, then set up the remainder of the disk space as raid 5.
Re:Try not getting illegal P2P movies (Score:2)
Right. DLT media costs a good deal more than large IDE drives do per GB nowadays, nevermind the cost of the DLT drive itself.
Really, the only thing that's cheaper per gigabyte than IDE drives is DVD+/-R media. So if you're looking for archival or just backups, your cheapest bets are DVD+/-R (cheaper per GB, don't require rebooting or opening your case to switch media) or large IDE drives (faster, read/write, larger, probably likel
Re:Try not getting illegal P2P movies (Score:2)
Re:Try not getting illegal P2P movies (Score:2)
Just a few steps (Score:2, Interesting)
I'd set up an older computer system (P2 or newer or older..) with a RAID configuration. Something like RAID1 for simplicity and redundancy. Then you just store everything on that central server and your client machines don't matter.
Or get a bunch of chinese kids in your basement to memorize strings of 0's and 1's.
Just keep buying bigger drives (Score:1, Interesting)
I just bought one of these: LaCie Bigger Disk [bhphotovideo.com] for my Mac, which is 1TB RAID for under $1000.
Just buy a TB or two, and if you fill that up any time soon, then, well, start deleting stuff. My personal files take up megabytes, my MP3s take up gigabytes, so 1TB is fine for me. 1TB ought to be enough for anybody.
I'm even think
Re:Just keep buying bigger drives (Score:3, Insightful)
Now, Bill
Re: (Score:2)
Re:Just keep buying bigger drives (Score:2)
Maxtor 300's for $115. Barebones system for under $100. A wide selection of 4-channel RAID cards for under $50 (NEVER use a softraid - which includes the crap solution on most NF4 boards - not to mention you'll someday bow down and thank His Noodlyness that you have your boot drive separate from the RAID).
Under $1000? Try under $600, for either 1.2TB RAID0, or 900MB RAID5. Build two, and sync them once
Consider disk space as a lease, not an investment (Score:2)
You can think of buying hard drives as an investment, and see it as a never-ending buying process, or think it as a lease:
Don't jump on the great new big drive, stick with the better $/G ratio (250G or 320G combinations these days).
I evaluated some RAID, big drive, external multi-drive systems to "definitively solve the problem" and finally
Re-evaluate. (Score:5, Insightful)
Firstly, get a good catalogue system going. Put all your crap in one area. Get a sorted listing of creation time,last access time and categories. Get some hard-and-fast rules going as to when it should be archived offline. Go through once a month (or more often) and work out any new stuff that you need to backup. You do backup, don't you? Of course you do.
Then go through and check the last access times of your categories, and move to offline storage as appropriate.The advantage of getting this sort of regime going is that you've got more chance of having a backup offline somewhere when the inevitable happens and your drive wakes up dead one day.
For example, once you've got categories and last access times sorted:
- Digital Photos, family movies , documents - anything more than 12 months - Offline to DVD. Use par2 [par2.net] for archive copies (sent to distant relatives for storage), but make another set without par2 for normal, semi-random access.
- ISO's of distros? Got a broadband connection? Ditch them.
No broadband? Get them all offline, regardless of age.
- TV Episodes? If it's later than say, 3 months - off to DVD they go.
- Web Site content? 3 months.
And so on. Work the times out for yourself.
You really need a good cataloging system to help find out where the offline files are. Everyone's idea of a good catalog is different - Hell, I just label DVD's and keep them sorted by catagory - so I'll leave it to you.
If you organise your data and find that you still haven't enough drive space to keep all your current data online, then (and only then) go and look at the expensive options.
Re:Re-evaluate. (Score:2)
eRacks, DIY (Score:2)
Ther are MANY tutorials on building your own RAIDed NAS on the net. Some have been slashdotted. Here's one. [librenix.com] Google for others.
Roll your own NAS or SAN (Score:2)
I buy drives four at a time, along with a 4 channel disk controller. I put them into Coolermaster 4-in-3 drive cages th
I don't think you want to know, but... (Score:4, Informative)
And I've spent thousands of dollars on my home network and attendant PCs, to solve the problems that the original poster will only have if he manages to actually get enough storage for his needs.
Presently I have four identical storage servers, with the following characteristics:
Athlon64/3000, 1GB RAM, Gigabyte K8VM800 motherboards, 4 Hitachi 7k250s (RAID5 on 3ware ), 2 Hitachi 7k400s (soft RAID0, stores a daily snapshot of the RAID5, which is the data that is actually shared), 1 Samsung SP1614 Boot/OS drive, a 3Ware 8506-4LP, Intel Gbit NICs.
These machines run a series of scripts that collect and copy (pictures or MP3s) or move (video) whatever I happened to have dropped on my various workstations (each have between 300GB and ~1.7TB) to appropriate filesystems on the various servers (one for porn, one for ripped DVDs and TV shows, one for music, one for pictures); those filesystems are then exported via NFS to another Linux machine where the whole mess is presented back to all my machines as a single file system.
Getting enough storage is simply a matter of applying money. 250GB drives are quite reasonable nowadays and 160GB drives are downright cheap, but dealing with dinky little disks make getting enough SATA ports problematic. Sub-$100 2- and 4-port SATA controllers from the likes of Adaptec, Promise and Highpoint have their own problems. Most don't do online volume management and REALLY only do RAID through a driver, rather than an actual onboard processor. They're fine for storage expansion, a JBOD or RAID0 (note: RAID0 is normally a VERY stupid thing to do, since most people aren't doing STR-intensive things with their PCs and the chance of losing data is substantially higher than for any single disk), but as with everything else, you get what you pay for, and ports on a proper controller are probably worth more than the disks you're attaching to them.
RAID5 is also kind of a bad deal for write-intensive data - lots of little files that get updated a lot, while I'm at it. Do RAID1 or RAID10, (or maybe RAID3 if you can find a controller that supports it) for data you care about. Spend money.
USB2 and Firewire enclosures are NOT a good solution for adding primary storage most of the time. Generic enclosures frequently have difficulty with larger drives, and often have VERY cheap fans that either fail quickly or detriorate to the point that they sound like a penny in a vaccuum cleaner. Additionally, the performance and CPU utilization of USB2 enclosures both tend to be god-awful. Brand-name enclosures have a few different problems: many use 2.5" disks, which in my experience are rather delicate. Others are not properly cooled, and almost all of them are sealed enclosures. Better to put a drive inside a computer if possible. I tend to think of USB2/Firewire drives drives as backup devices only.
Disk-wise I tend to prefer Hitachi 250 and 400GB models, or Samsung 160s or 200s, and SATA over PATA when possible. The Hitachi 500GB models get too damned hot, and it's the only one that's out (available for purchase) at the moment. Seagate and Maxtor ATA products tend toward tepid performance, and in the case of Maxtor, quality hasn't been good since the Quantum merger in 2001. I will not purchase a Western Digital drive for any reason, but specifically I avoid the geek-favorite Raptor 360GD; I was party to the construction of a small 20-drive SAN using Raptors (client's spec, not my idea) a couple years ago where the drive failure rate was approximately one drive every 33 days.
Re:I don't think you want to know, but... (Score:2)
Re:I don't think you want to know, but... (Score:2)
I have a great deal of original content, mostly things that I have video-captured myself over the last 10 years or so. I actually had about 4TB of material on my hard disks before I got broadband (and another five or six on CDs or DVDs), which was only a couple years ago for me.
Re:I don't think you want to know, but... (Score:2)
Simple. (Score:4, Funny)
Re:Simple. (Score:2)
Infrant ReadyNAS X6 (Score:2)
Note I have no financial interest in Infrant, I just want their products.
Re:Infrant ReadyNAS X6 (Score:1)
Infrant uses a dedicated custom CPU (NSP) specifically designed for network storage and a Linux based OS to achieve excellent performan
what works for me (Score:5, Interesting)
It sounds like you may be a digital packrat. If you are, I sympathize as I was one too after I got broadband. I stopped after realizing how much time/money I was wasting on crap that really didn't improve my quality of life. Now I buy new hard drives for my file server once every three years instead of every three months. Following are some of the things that helped me.
1) Download less porn/warez. Or put off downloading more until you've watched/run/played what you have. If you're just one person cranking through that much space that quickly, then you're downloading things just to have them. Stop that.
2) Go through and 'rm -rf' files and directories that you haven't accessed in a year. Don't keep obsolete versions of operating systems around, because you won't use them. As soon as you download a CD image, burn it and rm the ISO.
3) Archive on external media anything that's sentimental but rarely accessed.
4) Make it a routine to burn stuff to CD/DVD at least once a week. Eventually, you'll get tired of wasting time burning crap that you don't use and this will help you realize that you really don't need it at all.
5) If you do a lot of video editing or webmastering that requires huge amounts of data, and you're making money at it, then you need to invest in a proper server to keep all that. Be sure to make backups too. If you do this for a company, have them take responsibility for this.
6) Take a page from Linus's book: Upload it to an FTP server and let the world mirror it.
RAID (Score:2)
Assuming you already have a cheap PC, throw a SATA replication card in (Cheapest way to get RAID-5) and you have yourself an array.
Using cheap drives (WD 7200RPM 250GB has the lowest cost-per-gig here) you can get 1TB of total storage for for $488 USD. Of course to get 1TB in RAID5 that means an extra drive, so the cost for 1.25TB of total storage is $610 USD.
Of course if you don't have a PC to use, a NAS server ($130 USD), two USB enclosures ($25 each) and of the 250GB drives and you h
Phshaw! This is just like owning a home: (Score:2)
Home raid box (Score:1)
Network Storage Appliance (Score:2)
And I don't want to sound like an ad, but...
I also have a Terastation, and recommend it (Score:2)
It runs a really weird hacked up version of PPC Linux:
admin@TERASTATION:~$ uname -a Linux TERASTATION 2.4.20_mvl31-ppc_linkstation #15 Tue May 31 10:18:19 JST 2005 ppc unknown
An sshd went on there about 16 minutes after it was in the house. Mine's the low-en
Look (Score:1)
My approach (Score:2, Interesting)
The master box at home has a cheap DDS4 drive which I got from eBay for ab
SequoiaView (Score:2)
Re:SequoiaView (Score:2)
Treesize [jam-software.com] also does a good job locating where the bigger chunks of data on your drives are located.
man... (Score:2, Informative)
Data Migration Faciliity (DMF) (Score:2, Informative)
I've no idea what the cost is, or if there's a low-end solution.
2.8TB RAID 5 for $4100 (Score:3, Interesting)
Yes, $4100 is a lot of money. But I built it for exactly the reason you mentioned; I've always been of the philosophy of "do it once and do it right," and this way I've taken care of my storage needs for years to come.
Re:2.8TB RAID 5 for $4100 (Score:4, Funny)
For $4,100 he could just hire people to come to his house and fuck each other while he watched.
Grow up (Score:2)
Back when I was your age sonny, I had this problem with disks filling up. My dad did not let me have many disks, and never got me a double density drive. For years I worked hard to keep all my data on my collection of floppies. Over time things grew to over 100 disks, but never as fast as I needed them.
As technology marched on, dad eventually bought a computer with a large 80 megabyte harddrive (Back when 40 was standard), which when combined with dr-dos's disk compression lasted a few months before I
Re:Grow up (Score:2)
Something you might want to consider ... (Score:2)
http://www.pcsuperstore.com/products/G22301-Netgea r-SC101NA.html [pcsuperstore.com] (Use your favourite seller, this was the first link that I clicked on.)
It's a true SAN device, hence block access, which means fast. You can just add more devices as needed. If I remember correctly, you can "team" devices for better performance, or larger virtual drives.
The only problem so far
Also, I am not sure what the access from Linux situation is; I think I saw a
Bigger harddisks (Score:2)
And then when I need something that's archived on CD or DVD I copy it to harddisk again. I just want stuff readily available, so instead of spending countless hours archiving back and forth I now just get that bigger harddisk and usually by the time I run out o
Stay Away From LaCie Products (Score:2)
trick (Score:1)
Horder! (Score:1)
Mind you I've just bought 2x200Gb drives for my fileserver, as the old 40+160Gb was not enough for DVD rips and a load of VMWare sessions.
My routine is:
1. download/rip etc. the files and store them on the hard drive.
2. If I've not used a file in a month:
2a. If I think I may need to lay my hands on it quickly (like ghost images and fedora iso's), bung it on the
Which category is the worst offender? (Score:1)
lotta downloaded video? (Score:2)
windows movie maker is your friend, drop a movie into moviemaker, check the resolution of the movie.. then save your movie in the smallest file size that keeps those dimensions.. chews up a lotta CPU cycles, but can reduce filesizes by half..
I drop most of my videos right off, quality does suffer, but- still very viewable. I just dropped a 1.06 gb download into 602 mb...
Re:lotta downloaded video? (Score:1)
Windows Movie Maker turns movies into crap. Using ffmpegX (GUI front-end for ffmpeg and mplayer) you can make H.264 versions of your movies. What does this mean? It means movies don't loose quality and can be 3 times smaller than WMV files.
Windows Movie Maker is a sham. It doesn't do anyone any good. ffmpeg and mplayer actually are worth a damn in terms of quality. Oh, and they're free and cross platform. Always a plus.
There is no solution. (Score:1)
I have over 2 terabytes on my LAN, and would never dream of buying an HD less than 400G nowadays. I also have about 5000 CD-Rs and 2000 DVD-Rs full of offline-content. Everything is indexed in greppable text files.
There is no escape. Drives become full, and then you must offload files as fast as you add new ones. For me, this is 5-10G a day.
My point is: There is no escape. You can't win, you can't break even
Store what you can afford! (Score:1)
use Paperdisk and Staples (Score:2)
Re:here's an idea... (Score:3, Insightful)
I suspect someone in Redmond might have a problem with this. Using Linux and Samba would be a much better idea, and easy to do with a distribution like Ubuntu, Mephis or Suse.
Re:here's an idea... (Score:2)
Re:here's an idea... (Score:3, Funny)
Noooo, you don't need good hardware for a fileserver. Stability and data integrity don't matter at all. [/sarcasm]
Dimwit.
Re:here's an idea... (Score:3, Informative)
save up the cash and put together a 2 terabyte SATA storage server (the mobo doesn't have to be great, nore the ram)
That all depends on how important your data is. High quality parts should be of top concern. Also, don't skimp out on the ram -- the more ram you have, the more the server will have to cache recently accessed files.
2 TB should be enough space. MAKE SURE YOUR NETWORK THAT THE SERVER IS CONNECTED TO IS 100 MBIT (preferably the client transfering files to/from as well). It might be a good
Re:here's an idea... (Score:4, Informative)
100 Mbit equipment is so cheap now, every PC that's at least 100 MHz should have it. If you have some older workstations, 100 Mbit cards may be hard to find, but these would have to be pretty old boxes to not come with 100 Mbit ...
That's not true if you've got a switch rather than a dumb hub, and switches are pretty much the norm now. Switched networks have almost zero collisions, no matter how busy they are. (Granted, they should have zero collisions, not almost zero, but switches do occasionally have bugsSure, latency does go up when you're pounding on the network, but on a 100 Mbit network it's not that much -- just a few miliseconds at most. And for moving lots of data around, latency doesn't matter much anyways -- it's throughput that matters.
Not quite. You won't actually see 120 MB/s with today's hardware, but you might see 20-30 MB/s, maybe a little more on really serious hardware, which is still a lot better than the 12 MB/s you'll get out of 100 Mbit ethernet. And it's relatively cheap now -- I recently got gigabit PCI cards for $5 and a 5 port gigabit hub for $20 at Fry's.Re:here's an idea... (Score:2)
To hell with that; GigE can totally hammer the dinky PCI bus most systems have these days, you want something with proper interrupt moderation, 100% working checksum offload and decent DMA support; jumbo frames are nice too. Cheapy Realtek cards need not apply; spend a bit more on an Intel Pro/1000 or so, unless you like having 80% of your CPU dedicated to servicing interrupts and copying data around pointlessly.
PCI-E stuff seems much better, thankfully; makes on-board GigE on con
Re:here's an idea... (Score:2)
It's not nearly that bad with the $5 cards.
I ran `cat /dev/zero | nc other-host 6666' with a `nc -l 6666 > /dev/null' running between a dual p3/700 and an Athlon XP 2600, with both commands run on both systems, and saw about 27 MB/s going in
Re:here's an idea... (Score:2)
At ~30MB/s I see something like 3% CPU use on my X2 4400+ and an on-board PCI-E Marvell; once I get my fileserver sorted out with PCI-X (surplus rackmount server) I'd hope to see more exciting transfer rates more in tune with modern HD's. Presumably similar could be said with two consumer-level PCI-E systems now; that makes me very happy
For comparison, on a 3200+ I see
Re:here's an idea... (Score:2)
Check back a few months in my
Re:here's an idea... (Score:2)
And the hub involved is certainly low end as well.
And even /dev/zero isn't unlimited bandwidth -- ultimately, it's limited by cpu and memory bandwidth.
Re:here's an idea... (Score:1)
Although that might be due to crappy Linux drivers for the Intel (it hard crashes the box if you throw too much data at it).
Where I used to work we had a dual 3.2GHz Xeon with 2Gb RAM and a $4000 PCI-E gigabit card with fibre and that could not make much over 600MBps (the Realteks got about 220MBps on a regular PC/CAT5e).
Re:here's an idea... (Score:3, Funny)
Re:here's an idea... (Score:3)
Re:here's an idea... (Score:2)
Re:here's an idea... (Score:1)