6437287
story
Comments: 180 +- Ask Slashdot: Affordably Aggregating ISP Connections? on Thursday October 15, @06:01PM
background: url(//a.fsdn.com/sd/topics/topicnetworking.gif); width:71px; height:37px;
networking
background: url(//a.fsdn.com/sd/topics/topicit.gif); width:75px; height:61px;
it
An anonymous reader writes "Has anyone setup a system to aggregate multiple ISP connections to form a high bandwidth site-to-site link? Load Sharing SCTP looked interesting, but it doesn't look like it has been widely adopted. Multi-Link PPP appears to be more widely supported for clients, but I can't find any good guides for setting up both sides of the connection for a site-to-site link. The hardware solutions I've found are expensive for a small business. Does anyone have experience using hardware solutions from Mushroom Networks (Virtual Leased Line, p2 of this document), Ecessa (site-to-Site Channel Bonding), or others?"
Related Stories
Submission: Aggregating multiple ISP connections? by Anonymous Coward
One person's error is another person's data.
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.
Peering (Score:2)
The hardware solutions I've found are expensive for a small business.
And leasing the line is too if you want multiple ISP's on it. One is easy, after that it gets hard.
If you want something fail-proof, just go for co-location in an established datacenter with many peers.
The more interesting question here is that if someone has done *peering* outside of major datacenters? It's of course costly, but it's something the submitter is looking for.
Re: (Score:3, Insightful)
I think that the poster was intending to agreggate a cable, DSL, and satellite link to make a more reliable connection, not get multiple ISPs on one line.
HTTP proxy doing range requests (Score:2)
I had an idea for how to do this - has anyone tried using a HTTP proxy, and having it split up large downloads across multiple HTTP range requests, each going out of a separate WAN connection?
In other words, given N connections to the internet: Small requests are load balanced by simply doing round-robin. When the response starts coming in, if the object size is more than say 10MB, the proxy goes and issues N-1 additional range requests across each of the other WAN connections for equal sized chunks (or si
Re: (Score:2)
Re: (Score:3, Informative)
I've been using the hotbrick LB-2 for years to aggregate dsl and cable lines. Works like a charm.
http://www.hotbrick.com/produto.asp?tipo=3&catpro=2 [hotbrick.com]
I thought they had up to a 4connection version, but I don't see it anymore.
Re:Pering (Score:3, Interesting)
Its long, at least read about Greenlight in N.C. and learn!
I am 100% positive you could do this with hardware that will run the DD-WRT, here is a list of DD-WRT supported devices [dd-wrt.com], they have a search link, but I find that it does not work very well if you do not know the name of the router / firewall that you are looking for. So use the list and find a supported device.
You would need two of them and two different providers. You could even get a third one and do some special VLAN stuff to put some ports
Bonded VPNs (Score:5, Informative)
I have bonded 2 IPSec VPNs running over 2 ISP's to create a bigger (and cheaper) site-to-site link on the cheap.
http://www.zeroshell.net/eng/faq/vpn/
Read Point 5 in the link
What are you really trying to do? (Score:5, Insightful)
What you have presented us with here is a "B C" problem. You want to achieve C, so you ask us how to do B. Unfortunately, you never specify what A is, so the best we can do is give you some pointers for B which are probably going to be irrelevant and useless to what you are really trying to achieve.
Most of the comments will probably be about trying to figure out what your A problem is. To that end, why don't you just get a faster line in the first place and forget about this line aggregation stuff you're asking about?
Re: (Score:3, Insightful)
As the other poster noted, it's not always easy to just add more bandwidth. Where I live, the absolute fastest DSL line I can get is 1.5 Mbps. Fortunately, my cable company offers faster options, up to 22 Mbps. If they didn't, I'd be screwed if I actually wanted a decent connection relatively cheaply.
Also, one nice thing about having multiple links over multiple ISPs multiplexed together is that you have redundant links. If one ISP is having problems, you sti
Re:What are you really trying to do? (Score:4, Informative)
If he wants the last one, does he want:
Some of these are trivial, some require a little bit of client-side configuration, some require additional support from the ISP. Without knowing what he actually wants to achieve, it's impossible to make a recommendation. You can do all of these things relatively easily with a stock OpenBSD install on your router, but exactly which ones you want depends a lot more on the requirements. For somethings, you want to run a VPN between the two sites with packets sent over some of the link with the most bandwidth. For others, you could get away with just a couple of routing rules. If you want more than just the two sites and you want existing connections to work then you need the ISP to support updating the routing tables when their link to you goes down.
Parent
Multiple bonded connections (Score:3, Informative)
In theory, you can bond multiple DSL, multiple cable, multiple T1, or even multiple dialup connections from the same vendor.
Even if you are in a small town where the best service you can get is 1Mbps DSL, if you've got enough wires running from the neighborhood box to your house you can ask for 2 or 3 or more separate DSL lines and see if the local telco will support bonding them.
Even 15 years ago some telcos offered on-demand, 0-24 circuit, bonded dialup. The idea was a business would use it as up to 24 v
Have experience with Ecessa... (Score:2, Informative)
We've been using 2 Powerlinks from Ecessa (back when they were Astrocom). They work really well, and the price is tough to beat. We have one in our Dallas branch (with a T1 and business cable ISP) and one at our home office in Baton Rouge (a dual bonded T1 and business cable). They are channel bonded with each other, so the site-to-site VPN is more stable. They made my life a lot easier!
tomato (Score:3, Informative)
The cheapest way to do this is use the mlppp version of tomato on a wrt type router. You can check it out here: http://fixppp.org/ [fixppp.org]
Re:tomato (Score:5, Informative)
Only works if you've got DSL, and then again only if they use PPPoE. Then the remote DSLAM needs to support mlppp as well.
I would suggest OpenBSD + PF and just load balance the connections using PF. Takes all of 10 lines of code to get it up and going and is well documented. This won't aggregate your bandwidth, however if you have multiple streams open, it'll bounce those between two or more connections. I've personally done 4 lines like this (2x adsl2+ and 2x DOCSIS 2) and hit about 95% utilization across all lines.
Also with PF, both lines don't need to be the same speed, or even with the same provider, which gives you some additional fault tolerance.
Parent
Re: (Score:2)
Psst.. If DSL is not PPPoE, then it's typically PPPoA. Should work either way.. You just have to find an ISP that will support you.
pfSense (Score:3, Interesting)
Re: (Score:2, Insightful)
I use pfSense too for my multi wan needs and it really is a wonderful distro imho.
However, there is a difference between grabbing a bunch of wans and throwing them together vs linking them together like one giant pipe. I'm not completely sure what the author is trying to do but if this person wants the multi lane freeway approach instead of multiple separate parallel roads than pfSense does not currently support protocols such as mlppp and may not be what they are looking for.
However, if the person wants to
Talari Networks? (Score:2, Informative)
Have you looked at what Talari Networks (http://talari.com/ [talari.com]) is doing? I'm pretty sure their products do EXACTLY what you're talking about. Might be pricy for you, but it should do the trick.
Re: (Score:2)
printers/newpapers (Score:3, Informative)
Re:printers/newpapers (Score:4, Funny)
Even in really depressed metros, diners are still pretty expensive compared to T1's.
Parent
Dreytek (Score:2, Insightful)
Ask Willie Nelson, he uses Mushroom Portabella (Score:4, Interesting)
Need More Infos (Score:2, Insightful)
Sounds like you're trying to take a DSL, cable, and possibly a T1 or other technology and trunk them for combined throughput. That isn't possible because you'd have packets in the same stream taking different routes and TCP/IP doesn't allow for that, that I know of. I don't think any technology allows for that. For example an 8mbit DSL, 6mbit cable, and a T1 can't be combined to make a 15.5mbit connection. I suppose the same would be true if you were wirelessly connected to multiple networks.
You can, howeve
Re:Need More Infos (Score:5, Informative)
"TCP/IP doesn't allow for that, that I know of"
It sure does - it doesn't care what route the packets took - just that they got there. THe problem is if you split the stream over 3 links with varying latency - you won't see the performance gains you wan t- it'll more likely hurt.
If the goal is to end up with a virtual point-to-point link between two offices using multiple ISPs, you can certainly leverage multiple connections to do that. It also depends on the nature of the traffic.
You can set up multiple VPN tunnels and then run whatever protocol you want - you could do MLPPP - but that'll get ugly if the links don't have very similar characteristics.
The solution you mentioend in the end - Iv'e found that' susually the best - you can get most common *nix systems to do some kind of weighted load balancing of outgoing sessions... whether it's per-source, per-destination, per-protcol, or based on any other weird usage combination you had.
For an office situation Iw as once in - we had 1 2mbps and 1 x 4mbps lines (from separate providers) and a very high latency 1Mbps satellite connection.
I gave them a web page that had four buttons on it.
The first was "normal operation - 2MB + 4 MB". TCP sessions would be randomly routed over one orhte other, with double rpeference given to the 4 meg line.
The ohters were "ISP1, ISP2, and Satellite" respectively. At the push of a button the routes would flip, the state tables would flush, and everything would work. For practical puproess, it worked really well.
There is no magic way to simply aggregate bandwidth from separate providers over consumer connections with meaningful results... not like bonding multiple direct lines or anything like that.... 2 + 2 won't equal 4.... but depending on the use case, it can be just about as good.
Parent
Re: (Score:2)
The problem is that many stacks treat out-of-order packet reception as packet loss, which causes the sender to throttle the outgoing stream. When the GP said "TCP/IP doesn't allow for that," they probably meant, "The congestion-
Re: (Score:2)
What stacks are those?
Re: (Score:2)
Sounds like you're trying to take a DSL, cable, and possibly a T1 or other technology and trunk them for combined throughput. That isn't possible because you'd have packets in the same stream taking different routes and TCP/IP doesn't allow for that, that I know of. I don't think any technology allows for that. For example an 8mbit DSL, 6mbit cable, and a T1 can't be combined to make a 15.5mbit connection. I suppose the same would be true if you were wirelessly connected to multiple networks.
I think you need to review your basic networking knowledge. We use packet switching, not circuit switching. Different packets within a single TCP/IP connection can most definitely take different routes to their destination. It might not be the optimal situation, but it is built to work that way.
Only Half a dozen BSD and Linux Appliances... (Score:2, Informative)
I'm not one to yell at noobs but I really can't imagine timothy did more than a Bing search because I see that pfSense comes up on the first page of results on Google when you query "multi wan".
PfSense is probably the go for this, but you are free to choose any other BSD or Linux based distro which gives you a nice pretty point and click web interface out of the box and good online documentation on how to use the features.
Hell, you don't even actually need physical hardware for this provided that you have
Mac OS X? (Score:5, Interesting)
Admittedly, I have no idea if it works, nor do I have any idea how it decides to load balance between the connections.. But I ran across the feature the other day and it looked pretty cool.
In Mac OS X you can create a new "Aggregate" network device from any other devices and, in theory, do exactly what your describing. Again, I just ran across this the other day in Network Preferences and have no idea if/how it works, but it might be worth a shot (especially since it seems a lot easier to configure than a roll your own router with dd-wrt or tomato, though those likely offer more fine-tuned configuration).
Policy-based routing (Score:2)
If you're trying to combine different types of access (leased-line, cable, DSL), I think you're out of luck with trying to aggregate everything into a single "super circuit". However, you can certainly utilize all of those individual circuits. Look up policy-based routing. Most every platform out there should support it through some method. Set it up so that email goes over the DSL, your database queries goes over the cable connection, and your VoIP goes over the leased-line. You'll probably need to tweak i
iproute2 (Score:2)
"Routing for multiple uplinks/providers" [lartc.org]
Once the load balancing has been stablished you can set up OpenVPN to encrypt the traffic between the two (I like using openvpn + brigde to do a Layer 2 vpn). You can even get more fancy and add traffic shaping to distribute bandwith, prioritize packets (for a lower latency in ssh or terminal server traffic for example).
LISP Routing (Score:4, Interesting)
Some people (Cisco, etc.) are working on developing the Locator/ID Separation Protocol [google.com] as a core component of the Internet infrastructure.
If that ever takes off, you'll be able to buy a Provider Independent IP address block, advertise it through multiple ISPs (even Cable/DSL), and transparently load balance your upstream and downstream traffic across them, without bloating the core BGP tables.
The downside is, you'll have to use an MTU that's smaller than 1500, but I'd say it's a fair trade.
Advanced Routing Howto (Score:3, Informative)
Linux Iproute2 is all you need (Score:4, Informative)
A few years back I did this with a colleague, we actually investigated 3 solutions; 2 commercial and one linux script based, in the end the one that won easily was the Linux script.
Basically using iproute2 and some nice scripts gives you the ability to load balance your outbound packets and then using some relatively simple scripts to monitor each remote peer for automatic failover.
A quick google turns up this blogger who sounds (from a quick skim) like he's doing the same thing: http://blog.taragana.com/index.php/archive/how-to-load-balancing-failover-with-dual-multi-wan-adsl-cable-connections-on-linux/ [taragana.com]
Unfortunately I can't remember the commercial solutions we tested (this was 4-5 years ago!), but although they did exactly what you wanted perfectly, our problem was that we were doing this for a managed services company who ran 150+ IPSEC VPN's over those (at the time) 3 bonded ADSL connections, needless to say the commercial solutions had never imagined anyone trying to statefully balance that many VPNs! However with some tweaking (to be honest a LOT of tweaking) we got the Linux solution working a treat, even with nearly seamless failover.
Google is your friend on this one.
Re:Don't be so cheap (Score:4, Insightful)
All of them?
Um, yeah: Whatever you say, kid.
We usually just use a Roadrunner account for the main office, just like all the other small business out there. It's faster and cheaper than a T1, and has better reliability than the PRI that handles our phones. (We also have a freebie account with a local WISP that we do some business with for manual fail-over, but we haven't had to use it in years.)
Parent
Re:You can't do what you want to do (Score:5, Informative)
Sure you can.
http://www.openbsd.org/faq/pf/pools.html [openbsd.org]
One simple example. Plenty of other options available with other software. As long as you load-balance per connection instead of per packet there aren't many issues with this, and those often don't apply outside of special use cases.
Parent
Re: (Score:2)
Perhaps English isn't your first language...
The OP wrote "Has anyone setup a system to aggregate multiple ISP connections"
Had he written "Has anyone setup a system to aggregate multiple connections from an ISP" you'd just be a jerk, but as it stands, you're an ignorant jerk.
Re: (Score:3, Informative)
Wow. I'm not the AC but after that response I fully agree with him.
Your use of selective quoting is amazing, you got some big-ass internet cojones to ignore the rest of the very same sentence that you quoted.
Re: (Score:3, Informative)
What difference does that make?
It makes all the difference in the world. All you need is the appropriate device at each site - not at the ISP. Set up a VPN tunnel across the multiple links that terminates at the other site and you can aggregate at the packet level just like any of the site-to-ISP aggregation methods. The only case where the ISP has to support link aggregation is where it is site-to-internet-at-large, not site-to-site.
If so, the internet cojones apparently don't require intelligence.
Considering that it now appears you've been proclamating without investigating, it is quite appropriat
Re: (Score:3, Informative)
An ISP provides a connection to the internet, by defintion. So, "site-to-internet-at-large" is what was the topic of discussion.
That's some funny ass shit dude.
The OP said site-to-site link and you think he meant not site-to-site link!
You crack me up. Are you stoned or just high on your ego defense mechanism?
Been fighting for peace too?
Fucking for virginity maybe?
Re: (Score:3, Informative)
What's funny is how you keep ignoring the original premise and want to infer based on subsequent statements
Subsequent statements in the same sentence that serve to clarify his intent.
You just keep right on denying the obvious dude, safe and warm in your little house of meaningless semantics
Re: (Score:3, Informative)
Your meaningless semantics really are meaningless - they certainly aren't details that make a difference to solving the actual problem as stated.
As someone who has done precisely what the guy asked for, as previously described with a VPN, this 'not a tech' laughs at your continued denial of the obvious.
PS, this "not a tech" has 20+ years of tcp/ip stack and other misc internals experience, he knows exactly what he's talking about.
By your own demonstration in this thread, you don't.
Re:You can't do what you want to do (Score:5, Funny)
a fucking book on how routing works
Now there's a fetish you'll only run across on Slashdot.
Parent
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
I imagine the way sharedband works is that it's a VPN endpoint. If you use VPNs (essentially creating another IP layer on top of the existing one), you *can* aggregate multiple connections and even get faster single-session transfer speeds.. You just need an endpoint to connect to that has at least that much bandwidth. This appears to be part of what sharedband offers. The main issue I'd be concerned with, however, is latency.
Re: (Score:2)
And they all work like crap...
If you want to truly multi-home, get an ASN and do BGP.
Re: (Score:2)
Re: (Score:2)
> ...their cell phones strapped to the outside of their waists...
You strap yours to the inside of your waist? I'm trying not to visualize that...