Cross-Platform Firewire Networking at Home? 52
Stahnke asks "I have two computers that I need to exchange data between. I do music-production on a Windows-based system and have everything else on a Linux system. I need to exchange HUGE amount of data (5GB at a time sometimes) between the two systems as fast as possible while clients are waiting for me. 10/100 Ethernet is too slow, and fiber is just too expensive. Can Linux (2.4) do Firewire networking with a Windows machine? If so, how, and what tools are out there? I have a Firewire card working in Linux, but I haven't had luck with TCP/IP via Firewire yet."
A Halfway point (Score:2, Interesting)
Admittedly it's not quite as neat a solution at first glance as just firewire-cabling between two machines - but it sounds as good an excuse as any to buy an iPod!
Re:A Halfway point (Score:2)
Here's a non-destructive test to check the read speed of your linux drive (replace
#iostat 1 > iostat.log&
#cat
#tail -f iostat.log
Watch the change from second to second in the data transferred stat. Guess at it. Multiply that number by 4096 and you have bits/second. Even with 2Gbps fibre channel, you're not going to exceed that data transfer rate because of the spindle speed of your drive. The stat will actually drop toward the end of the drive because the sectors are spaced closer at the beginning (inside tracks) of the drive. On a fast scsi drive, you may get 80000 blocks/sec=327 Mbits/sec.
If you just store the data on the firewire drive in the first place, you don't have to wait several minutes for the copy. Just press the switch button and use the data.
Easier way to do read timings (Score:2)
Times a 64MB read from the drive. Most Firewire drives will cap out around 23 MB/sec (Megabytes, not megabits) The same drive/enclosure using USB 2.0 capped out at 12 MB/sec for me.
In my case: 7200 RPM 120GB Maxtor HD in a comboe 1394/USB2 enclosure fron www.newegg.com - Part number is ME720UF.
Re:A Halfway point (Score:2)
the "beginning" of a hard drive is the outer edge. the drive platter has a fairly constant data density, so the outer cylinders(concentric circles) have more sectors(slices of the concentric circle) than the inner cylinders. So on an outer cylinder, more sectors pass under the read/write head per rotation than on an inner cylinder.
with cdroms the data is written from inside out, which means that the slowest part of the disc is used first. this behavior is probably a remnant of the days when the cd drive was designed to maintain a constant data rate under the laser, rather than a constant rotational speed of the disc.
Just One thing to say (Score:1, Interesting)
Re:Just One thing to say (Score:5, Informative)
> Three Words: Firewise Drive
First, "Firewise Drive" is two words.
Second, it's "Firewire Drive". That's a good suggestion.
Third, the poster should investigate gigabit ethernet. Gigabit ethernet can be achieved over copper. Although I believe gigabit ethernet hubs/switches are still awefully expensive. However two inexpensive gigabit ethernet cards can be directly connected (i.e. no hub/switch needed) together with a crossover cable.
Fourth, if the systems are capable of hot-swapping drives, the poster could plug a drive into the Linux box, load up the data, yank the drive and stick it in the Windoze box.
Fifth, when writing the data out to a drive initially on the Linux box, use Raid-1 (mirroring) to write to two drives simultaneously. Then yank one drive out of the Linux box and stick it in the Windoze box. No waiting for copying.
Re:Just One thing to say (Score:2)
Re:Just One thing to say (Score:2)
TCP/IP removes this problem, but the data access will be slower.
Re:Just One thing to say (Score:2)
If you're going to correct someone for mispelling something, then you should take the time to make sure you do the same.
(Yes, I am aware of the fact you think its "leet" to call it Windoze instead of Windows. However, mispelling it is stupid.)
Re:Just One thing to say (Score:1)
If you're going to correct someone for mispelling something, then you should take the time to make sure you do the same
Since he is, in effect, diss-spelling rather than mispelling I'm not surprised that he wouldn't consider the one to be the same as the other. After all, as you point out, he is almost certainly using "Windoze" intentionally for effect...
(Yes, I am aware of the fact you think its "leet" to call it Windoze instead of Windows. However, mispelling it is stupid.)
I'd call 'disspelling' more pointless than stupid, but that's splitting hairs, I suppose. Anyway, given how common the practice is, here and elsewhere on the Internet, I'm kind of surprised that this one mild instance bugged you as much as it apparently did.
Fibre channel any good? (Score:2, Interesting)
Fibre channel is expensive but notable due to higher %age usage in real-world use. The max throughput on Ethernet (you refer to IP) is somewhere between 30% and 50%, whereas fibre channel can realistically get up to about 80% usage. So, on a 1Gb card, that's not too shabby.
Anyway, others will have better hands on experience but it might serve as a useful alternative avenue to explore.
Aegilops
Re:Fibre channel any good? (Score:2)
I would think that would completely depend on the hardware. For instance, if you're using a hub (and I doubt that there are any gigabit hubs out there), you don't want to see utilization much higher (in my experience) than 50-60%, because retransmits after collisions on a busy network will cause the usage rate to climb if everyone is still chatty.
However, we've seen usage rates (peak) on our switches at work as high as 90+ percent. Given, we're pushing 100Mbit from the desktop, through Gigabit links to the Network Operations Center, and from there, into dual or quad gig links to the servers (each).
Ethernet won't fall apart in my opinion, with good equipment. Get a decent switch, or if you're going from machine to machine, put gig cards in with a crossover.
Gigabit Ethernet (Score:5, Interesting)
Re:Gigabit Ethernet (Score:4, Informative)
Dont even need a crossover on most Gigabit NIC's. Intels Pro/1000 cards all have Auto-detect MDI-X.
Re:Gigabit Ethernet (Score:2)
Huh? What about RFC 2734 [isi.edu]? It's a standards-track protocol for sending IPv4 over IEEE 1394. Apple has a beta implementation for Mac OS X and I believe Microsoft has an implementation for Windows XP.
Also see RFC 3146 (IPv6 over IEEE 1394) and RFC 2855 (DHCP for IEEE 1394).
Gigabit Ethernet (Score:4, Informative)
My laptop and my cousins laptop both have gigabit ethernet in it as standard. We connected them together with some twisted pair (you will need a crossover cable) and we got 1000BaseT.
Mmmm, gigabit......
I am sure you can get 2 gigabit ethernet nics at a decent price, get some cat5e or cat6 or whatever it is you need (gigabit uses all 8 wires; 4 pairs, not just 2 pairs like 10mbit and 100mbit) and you are set.
D.
Re:Gigabit Ethernet (Score:2)
Ninnle Linux doesn't exist - mods on crack? (Score:1, Informative)
Re:Perhaps NINNLE is the answer! (Score:1)
Firewire Drive (Score:2)
--Mike
Re:Firewire Drive (Score:2, Interesting)
its possible (Score:2)
if your still stuck visit #linuxhelp on efnet. there are several veteran firewire networkers who swear by it over gigabit ethernet. (which to be any good has to be done with fibre)
state your question in full with many more details then you did for this story. such as machine types what you've tried, what docs you've read. etc etc. and then wait for someone to answer you
Drive Speed (Score:1)
If you have a raid array on each side then you would not be limited by the bandwidth off of the drives, but this would be expensive.I have seen some suggestions of getting a firewire HD and just share it between the machines. I think this would be one of your best bets.
Do not worry about the connection speed until you get more/faster spindles.
Re:Drive Speed (Score:2)
A single drive, even an old one, will easily saturate a 100Mb network. Heck, brand new 7200RPM IDE drives will top 50MB/sec, which also exceeds the capacity of Firewire.
On topic, my suggestion would be to invest in some GigE network cards and ideally a GigE switch (although the latter is unnecessary if there's only two machines). Such a solution will simply be a matter of switching from the existing network cards and then continuing as before, with essentially no reconfiguring and/or changed habits necessary and should reap at least a 3-4x performance boost.
What if Firewire is the only option? (Score:2)
I'm not so concerned about the speed (10/100 would be fine); I just want connectivity. If I go the firewire route, then all I need to buy is a $10 1394 cable, right?
My question is, does Windows 2000 (or Linux, for that matter) support 1394 networking? I haven't seen any definitive answers regarding which OS's support this other than Windows XP, and [sometimes] WinME.
Re:What if Firewire is the only option? (Score:1)
Re:What if Firewire is the only option? (Score:2)
Firewire Disk Mode (Score:3, Interesting)
Although not strictly relevant to your question, I feel the need to point out that if you had Apple CPUs with built-in Firewire, you could do exactly what you're asking for. To wit: by restarting one in "FireWire Target Disk Mode", which involves simply restarting and holding down the "t" key, that unit's HD will appear on the desktop of another Mac just like any other externally mounted FireWire HD would.
Depending on how much you need this feature, you might consider a "switch"--why not use the tools that provide the features you need?
Re:Firewire Disk Mode (Score:2)
However, for his other systems, I don't believe they would be able to read the disk since it's in apple's HFS format.
firewire drive (Score:1)
Theoretical bandwidth comparisons (Score:3, Informative)
Speed close to theoretical speed. Direct PC to PC might possible. Guesstimated speed around 40-45 MB/s.
FireWire 2(?) 800 Mbit/s ~ 100 MB/s
New version, Apple already supports on high-end, but likely there is no support for PC, or even interface, or very expensive. Speed close to theoretical speed. Direct PC to PC might possible. Guesstimated speed around 80-95 MB/s.
Drive Swap 1064 Mbit/s ~ 133 MB/s
Well, you will have to csave on PC, load on the other one. That makes speed at least half of theoretical one. Drives are not physically capable of these speeds, only around 40MB/s sustained. Guesstimated speed around 20 MB/s.
Gigabit 1000 Mbit/s ~ 100MB/s
Direct PC to PC possible. You need 64 bit PCI cards to get colse to 100MB/s. Guesstimated speed around 90-95 MB/s.
Ethernet 100 Mbit/s ~ 10MB/s
Cheap, easy, cheap, but slow. No special needs. Guesstimated speed around 9-10 MB/s.
USB2.0 480 Mbit/s ~ 60MB/s
Cheap, high(er) CPU overhead. Need PC-to-PC host bridge (might be hard to locate). Guesstimated speed around 40-50 MB/s.
Real-world caps (Score:4, Interesting)
So far I've found the 1394 networking support for Linux to be pretty slow... For some reason it seems to put the interface into 100 mbit/sec mode.
USB2 can't even come CLOSE to theoretical max throughput. I have a combo USB2/1394 drive enclosure. In 1394 mode, hdparm -t gives a result of approx. 23 MB/sec. In USB2 mode, the same benchmark gives a result of 12 MB/sec. (For a 64MB sustained read)
Re:Channel bonding (Score:2)
answers, (Score:5, Informative)
this is TCP/IP ONLY!, that means that windows file sharing will not see across this medium, you will have to use IP (//192.x.x.youipwhatever/share) which may not be a problem for you but it's something to consider.
This does work well, i have a firewire400 link between a windowsXP machine and gentoo linux and it works very well. I use NFS for file sharing as i feel Windows is the "guest" on my network and should pay homage to the exsisting linux machines
i do get very close to optimal speed accross this link for file sharing. i have susstained ~45Megabytes/second transfering large video files. on these same machines with 100 speed networking i get about 9.5Megabytes/second and with gigabit i can get about 35Megabytes/second, but only on large files.
The firewire link is by far the fastest but i think the gigabit is being help back by immature drivers on the linux side.
good luck.
Re:answers, (Score:1)
Would you please document somewhere some of the commands and/or settings files you modified to get it working?? If I can find enough information to get it working myself I'm eager to document and contribute that howto info to anywhere/one who could benefit from it. TCP/IP networking over Firewire would be COOOOL!
I wouldn't say that I'm a newbie (I got slashcode compiled and running on my home machine if that helps you), but honestly -- I've looked all over google, slashdot, etc. and not found the specific commands or files that I would need to change.
Everything I've found essentially says "configure your FW card for IP" . . notta big help.
Thank you in advance.
It's not always about the network. (Score:2)
Removable, hot swappable drive bays in each machine, and then use a sneakernet to transfer your data.
Not familiar with sneakernets? That's where you use your sneakers as the transfer medium. Transfer time algorithm works out to (size of drive / walking distance) * number of trips.
In short, use removable drives, dummy, and walk between machines.
use gig Ether (Score:3, Informative)
400 Mbps FireWire makes it sound like you will get 4x better than 100Mbps Ether, but in practice you won't.
Gig Ether will do what you want.
If you still want more info on FireWire networking, visit my research page [cs.dal.ca].
Use multiple 100Mbit/sec Ethernet cards (Score:2)
OTOH, this may not be very effective between linux and windows.
TCP/IP over Firewire (Score:2)
http://web.mit.edu/kkeville/www/firewire/
also try googling for "ip over firewire"