The Quiet Death Of Intelligent NICs?

Captain Novell asks: "Have you ever wondered what happened to the likes of the Intel Intelligent Server Adaptor? I have, one day it was on their site and then, later...*poof*...it was gone, only to be replaced by this Windows 2000 IP/Sec card! With tears in my eyes I went looking for any other manufacturer that had Intelligent NICs, but all I could found was this IP/Sec rubbish! (for those of us who run OSI complient software, a TRUE intelligent adaptor will off-load part of the OSI stack to the card and usually has some form of large cache on-board a.k.a. the old Intel Intelligent Adaptor card. I recommend server adaptors to many sys admins! Does the Slashdot community have any advice on where I can still find these elusive devices?"

Re:They became regular NICs

[Chris Siebenmann]

Direct IO and DMA to or from user space gets discussed periodically on the Linux kernel mailing list. The general commentary I've read is that this is almost never a win until you are dealing with huge amounts of memory. Since doing this with huge amounts of memory is fairly rare (especially doing this with huge amounts of memory the driver can't supply to the user, instead of the user supplying it to the driver), code for this stays out of the general kernel.

Why is it so expensive? Because to do it properly you must do page table manipulations (including TLB flushes et al) and these are surprisingly expensive, especially on multi-CPU machines (where you must insure all CPUs have done the page table and TLB updates before continuing). This makes it faster to do the copying for most if not almost all of the user-level write() and read() operations.

This is one of the less obvious wins for sendfile(), and why zero-copy TCP sendfile() shows such a significant win: with sendfile() you can avoid all those page table manipulations because the data never becomes visible at the user level.

I believe the Linus-approved solution for things like video drivers is that the driver should provide an address space that the user can mmap() into his own process's memory space, and which he can then read, write, and write() as the driver scribbles information into it. Other people consider this a less than entirely satisfactory solution for various reasons.

Alacritech

[Erk]

You should look into their NIC - As usual, the info is here [alacritech.com].

I've met with these folks to talk about this specific card. Quite cool, as it offloads the IP stack on to the card to lessen the processor load on the actual CPU. They claim that for some applications, you will see as much as 100% performance increase.

Now, I have my doubts about that, but I can see something in the order of 30-45%. It has DMA and all sorts of other goodies that you'd expect, and it works quite well for fileservers (if that's what you're looking for.)

-Erk

Re:They became regular NICs

[SEWilco]

So you're saying the problem is creeping featurism. What used to be called an "Intelligent NIC" is now just an ordinary NIC -- and intelligence is now indicated by other features, such as encryption in firmware/hardware.

Re:Open Source Initiative??

[pyite]

I'm not sure if he's just joking or if its pure being naive.

Re:Open Source Initiative??

[Russ Nelson]

It's not a troll, you silly git. It's a celebration of the death of the old OSI, and the birth of the new. I mean, really, is *anybody* talking about OSI protocols supplanting IP at this point?
-russ

Re:Open Source Initiative??

[Russ Nelson]

OSI *used* to stand for the Open Systems Interconnection. OSI is dead, long live the OSI!
-russ

They became regular NICs

[h2odragon]

See the zerocopy networking discussion [linuxcare.com] from the latest Kernel Traffic (emphasis mine):

With zerocopy, when you issue sendfile(), the kernel does the network DMA straight from the page cache, avoiding that extra copy. In the case where the network card is capable of doing the TCP checksum in hardware (as a lot of newer cards can), the kernel doesn't even have to look at the data between the disk DMA and the network DMA.

This is almost consumer electronics, after all; there's no such thing as a static product line anymore. Even the venerable 3c509b is being (has been?) phased out for a -c version; which almost certainly has nothing fundamentally new or worthwhile in the way of capabilities, but might be a cheaper design, and is just different enough to cause trouble with old drivers. See also backward combatability [tuxedo.org].

Re:Open Source Initiative??

[autocracy]

Hehe, you're an idiot!

OSI stands for ISO's Open Systems Internection Reference Model. The idea behind it is that there are 7 layers running from the physical media to the application level. Each layer talks only to the ones directly above and below it. For an example, TCP/IP can send data over any media without caring whether it is 100baseT, 10base2, FDDI, phoneline, etc... Do a google search and you might just learn something!

As for your previous post, this one leads me to believe that it is wrong. If you know so much about networking, how come you don't realize what OSI stands for here?

The problem with capped Karma is it only goes down...

oh bleh.

[Zurk]

just use this one instead. [3com.com] besides the fact its better than intels, a linux driver is also available.

Intelligent Nics Considered Harmful

[Russ Nelson]

Didn't Van Jacobsen show that intelligent nics actually slow down your networking? The main loop of the TCP/IP protocol is quite small. If you're careful to cause the error conditions to be the taken part of the jump, then on modern machines the prefetch will always get it right.

With an intelligent nic, you're going to spend just as much memory bandwidth on the transfer as you would if the protocol is in the main processor. You're just not saving anything with an intelligent nic. The price premium causes them to be *much* more expensive that a regular nic, plus you have to write the stack that runs on it, plus you have to have operating system support for a stack on the card. All for what? No extra performance.
-russ