Traffic Control and Quality of Service In Linux?

MikeBabcock asks: "Quality of Service (or QoS) and Traffic Control (TC) have been options in the Linux kernel for some time now (stable in the 2.2.x series at least). Unfortunately, several sites that refer to these features lament the less than perfectly documented nature of the tools that configure them. Has anyone successfully implemented effective (often complex) quality of service rules with Linux and feel like telling the rest of us how you did it?"

Re:You might find this useful

[MikeBabcock]

After seeing the lack of information available, I've created a new web page [fibrespeed.net] with Linux QoS & TC as the focus. I will only add information to it that I couldn't find elsewhere, and link to the information I do find elsewhere.

Re:QoS is non-trivial, particularly in Linux

[Cato]

Good to hear you are working on a tool to make this simpler!

I suspect that doing all classification in iptables (formerly ipchains) is the simplest approach - this lets you do anything that the relatively mature iptables code can do, including classifying by port numbers, src or dest IP, ICMP subtypes, and so on.

Once this is done, CBQ seems to be the queuing mechanism of choice - although it may be best to support a non-hierarchical setup of CBQ to start with.

Policy rules for QoS split nicely into conditions and actions - the conditions classify packets, and mark them (either using the internal fwmark value, or a DiffServ codepoint value in the TOS byte); then the actions take the marked packets and assign these to queues. Ultimately this can be much more powerful, e.g. the actions can be to re-route the packet to another destination etc. DiffServ marking should be optional, since in some environments the markings will actually mess things up (ipmasq does, in particular).

Linux QoS does really need a tool like this to take off, otherwise it is just too complex to use. I'd also like to see text-based and web-based tools, as most Linux routers won't be running X11 - or does your tool work on remote Linux routers via ssh, perhaps?

Re:QoS is non-trivial, particularly in Linux

[MikeBabcock]

The goal is to create an output file that describes the queuing and traffic parameters (perhaps in XML?) which can then be parsed (Perl / Python) to give the appropriate iptables, ip, tc, etc. commands.

You might find this useful

[Anonymous Coward]

http://www.ittc.ukans.edu/~rsarav/howto/

Bandwidth throttling

[XoXus]

IIRC, in 2.2.x (and thus 2.4.x), you can throttle outgoing bandwidth via the ipchains mechanism, for firewall machines. I haven't tried it myself, but you may want to look in that direction.

QoS is non-trivial, particularly in Linux

[Cato]

I've been doing IP QoS for four years, starting with ATM and RSVP and ending up using DiffServ and MPLS QoS on Cisco routers.

Some reasons for slow uptake of Linux QoS, IMO:

- IP QoS is generally not well understood - few networks have actually deployed QoS.

- there are quite a few decisions to be made before you even compose a QoS setup command (e.g. how many classes of service, which applications need QoS, how much bandwidth per app, and so on - sometimes taking months for a large company).

- the Linux QoS infrastructure is very complex and powerful - it has far more features than most commercial routers, even Cisco (which has a large number of QoS features).

- the tc(1) command to control QoS setup is complicated to use, and not installed in all distros (e.g. Mandrake 7.0). Red Hat 6.2 includes tc but doesn't include the man page, at least on my setup.

- the Linux QoS docs are highly technical rather than user-oriented, e.g. the examples involve CBQ, which is a complex mechanism that many initial users of QoS will not need.

Despite my QoS background, I find the tc(1) command line examples for CBQ rather complex - CBQ is not easy to set up, because it involves a hierarchy of traffic classes, but it can be made easier than this (see Xedia's manuals at www.xedia.com, they make a successful CBQ based router).

It would be good to have some very simple docs that provide templates for Linux QoS setup corresponding to common requirements - for example:

1. Bandwidth allocation to individual servers in a web farm - use WRR (weighted round robin, no hierarchy, just guarantees bandwidth in the event of congestion).

2. Prioritise interactive applications over downloads - most Internet users could use this one. It would also use WRR, and would simply use port numbers to prioritise telnet and maybe http (ideally only certain MIME types to avoid including HTTP downloads) into a guaranteed bandwidth class (say at least 30 Kbps on a 56K modem link); all other traffic would get the rest of the bandwidth in another class.

Alternatively, a simple set of setup shell scripts (or Webmin/Linuxconf type modules) would make life a lot easier - the key point is to limit options so it would be much easier to just say 'give bandwidth X to ports A, B, C'. Is anyone working on this?

One interesting alternative, for large-scale commercial use of Linux QoS, would be to extend CISH (a Cisco CLI emulation for Linux router setup) to cover Linux QoS by emulating Cisco QoS setup. There are a lot of people who know how to do Cisco QoS, and quite a few tools to provision QoS on hundreds to thousands of routers automatically (e.g. my company's Orchestream tool, see www.orchestream.com). By emulating Cisco QoS syntax, Linux QoS could tap into a huge pool of experience and tools.

For more info on QoS in general, see www.qosforum.com - for some more links, see www.orchestream.com/support/links.html.

Re:QoS is non-trivial, particularly in Linux

[MikeBabcock]

Thank-you for your response. It heartens me to just see the very few responses as an indicator that the Linux implementation is as complex as I've felt it is. I've been trying to work on a Gtk+ application that would allow an administrator to prepare the tc, ipchains & ip commands necessary to implement some of the QoS & TC options given, but its very difficult to get a handle on the implementation.