Apache Bandwidth Limiting? 44
IOOOOOI asks: "I work at a high traffic web hosting company and we're trying to find a simple effective way to limit bandwidth hogs, some of whom we've clocked pulling over 4Gb/hr off our servers. We've tried mod_throttle and have looked into QoS/fair queuing as well as a couple of custom solutions in-house. None of these quite did the trick. Has anyone found an effective way to do this, one that can handle individual connection streams?"
You're missing the point (Score:3, Insightful)
"Blocking" network appliances such as Packeteer can't handle these high rates, and even if they had gigabit interfaces, they would only be able to do 600-800mbps on them.
None of the kernel QoS/queueing options I've seen allow for anything other than classifying traffic or "fair" queueing. None of this seems to help someone that wants to limit all webserver connections to 2mbps - everything here is expecting an IP range, ports, or something to distinguish by. What if I don't want to?
Apache needs real per connection, per user, and per IP rate limiting. mod_throttle and everything else I've seen has to starve connections after they perform too well. How about something that hard limits connections to 2mbps/sec. I will pay for anything that can do that for Apache today...
Forgive me if I have overlooked the obvious...
Use Zeus (Score:4, Insightful)
You want Zeus because it is high performance (it doesn't use the toy process-per-connection model). It comes with an easy to use, powerful web based GUI. The GUI doesn't just hold your hand. It lets you set everything, and then will show you the exact lines that are changing in the config files.
It doesn't use extremely complex format for config files that Apache uses. A good comparison is BIND and djbdns. Do you want to try and deal with the incredibly complex BIND zone files, or the simple, one record per line data files that djbdns uses? Zeus config files are one record per line of the form "modules!throttle!enabled yes". It also comes with tools that let you do everything from scripts. But only if you want to. Otherwise, use the GUI.
And speaking of throttling, Zeus does it correctly, unlike any other web server (at least any of the freely available UNIX ones, as that is all I am familar with). It will let you set a limit on the number of users, or set a max number of bytes per second on a virtual server or subserver level. It doesn't serve some people at max speed and then start dropping connections (mod_throttle) or set the throttle speed at the beginning of the request, then start dropping connections (thttpd).
Virtual servers in Zeus actually make sense. There is no master server configuration like in Apache. Instead, you create one or more virtual servers. As such, each virtual server has its own separate configuration. Virtual servers can serve a single website, or any number of websites, via subservers. Subservers all share the configuration of the virtual server (kind of like Apache's mass virtual hosting only much better). No more restarting the server to add a site. Simply create the directory, and it starts serving the site.
There are plenty of other reasons why Zeus is superior to Apache, but the ones I listed should be enough to start considering it. No, I don't work for Zeus or own stock (don't think they have any) or anything like that. I'm just a satisfied customer.
For some things, Apache works just fine. But for anything high traffic, requires throttling or needs a flexible or scripted configuration, Zeus beats Apache hands down. It's worth every penny. Check it out. I doubt you'll be dissapointed.
(subconscious message to Apache developers: stop being lazy and make Apache more like Zeus!)
Re:Maybe I'm missing something... (Score:3, Insightful)
The issue is not about our customer's bandwidth consumption and how much they can/can't use. It's about being able to provide services to all of their users without experiencing slowdowns because of the occasional hog.
Are your slowdowns bandwidth or CPU based? If you are serving lots static content (like porn), then Apache is going to kill you, due to its process-per-connection model, which the developers refuse (read: are too lazy) to fix. Zeus doesn't have this problem. Neither do the open source boa or thttpd (but they unfortunately lack many important features that may stop them from being used for commercial web hosting). Zeus will allow you to max out your network card (100mbit) on a modest machine (P3/500 w/ 1gb RAM).