Working With 2 ISPs For Home Networking? 356
An anonymous reader writes "This is, I think, a simple question — but one which I can't get the answer to.
As a typical, but perhaps high-demand home user I would like to use 2 separate ISPs. ADSL is pretty cheap nowadays, and 2 x ADSL seems a better value than one fast one — especially in terms of reliability.
If one breaks, at least the other will work.
Using an old box as a router/firewall, how can I configure a system to use two completely separate ISPs in a sensible manner?
Ideally, I'd like the load of my browsing to be balanced, but at the minimum, I'd want some kind of 'fail-over.' If I leave torrents running over night, I'd like the router to use whichever connection doesn't block the traffic — and preferably for it to reset the errant connection.
Ideas?"
Re:Point of failure (Score:1, Interesting)
Look at sharing with your neighbours.
Create a WLAN with those who have differentt ISPs.
Have a look aat cringely.com and serch in the archives of his pbs.org site.
Greetings from Norway :-)
Re:pfSense + two independant ISPs (Score:1, Interesting)
Agree with parent - I use pfSense on an old P3 Gateway with four NICs. I have it set in Failover (not load balance) mode for two ISPs. Only uses about 40 watts, one nic to the LAN, one nic to Time Warner cable (10mb/1mb), one nic to AT&T $10/mo DSL (pppoe, 768/128), and one dedicated to a Meraki free wifi community network that can't talk to the LAN. I initally set it up for load balance but ran into issues. The TWC connection was so much faster - when I went to get a new ISO or even just surfing, half the time I'd notice it going slowww. TWC actually went down today for me and I am on the backup DSL right now, pretty coincidental.
Re:DSL+Cable (Score:2, Interesting)
Re:Why bother, seriously? Why? BECAUSE... (Score:2, Interesting)
YES
I live in the Rockies on the western edge of a mountain ridge at 10k ft elevation - in other words a lightning magnet. I'm a full-time telecommuter for a multinational, & I work daily with people from 5 different time zones. Teleconferences, webex's etc. are my daily work life. Loss of connectivity to our source code repository can be a serious problem.
EVERY time there's lightning with 1/2 mile of here my phone & DSL go out. Last year I was out 7 different times for more than 24 hours. I lose track of the number of times I'm out for just a few hours.
I have a secondary ISP - WisperTel, a wireless WISP - that's a lot less reliable than DSL. Latency is bad, it's down a couple times a day at least, although usually for short periods.
To top it all off, I'm outside of cell phone coverage... and I have 3 DIFFERENT carriers. I'm only 1/2 mile to the nearest coverage, so I can drive or walk to make the necessary calls when both ISPs are down. This is fun when there's 3 ft of fresh snow on the ground, and it's -10F. Thank goodness for snowshoes... (Last year alone both were down at the same time 3 different times).
If I could also get cable here I probably would... although I do hate Comcast with a passion.
Re:That's not the usual Point of failure here (Score:3, Interesting)
There are four main reasons that DSL goes down
I've had DSL fail four times in the last 10 years. One was my DSL router. Two were when phone company installers working on boxes down the street disconnected me by accident. One was a billing problem (but that was when my ISP was providing beta service, and they mixed up things between my home account and work lab, and I was customers #1 and #2 in the western half of the country :-) Some of these can cause both circuits to fail, some can't - and backhoe events are pretty rare. On the other hand, cable's more likely to have common failures than DSL is, unless you're one of those rare people with two cable providers, because there's more shared infrastructure between the two circuits.
Even so, I'd recommend going with two different providers because they're going to have different performance issues and probably different policies. If it won't interfere with your cable TV service, I'd recommend cable and DSL - cable's usually faster, though more likely to be flaky, and more likely to have obnoxious limitations on your service like not letting you run a web server at home or giving you 20 Mbps of download speed with a monthly download cap that limits you to an average of 50kbps if you use it 7x24. DSL is more likely to be reliable (because infrastructure gets fixed along with lifely telephone service as opposed to television), probably slower depending on your distance from the telco, and you usually have a choice of dozens or hundreds of ISPs if you don't like the policies or pricing your telco offers.
One obvious way to mix the two services is to have a DSL with a static IP address, and do most of your own downloading from the cable modem. You'll need some kind of router to deal with keeping track of the two services, and some kind of firewalling, so you probably want to use an OpenBSD to do that and whatever your favorite Linux, Mac, Windows, or game boxes behind it. (I'm picking OpenBSD because it's usually the best at security and firewalling and at least OK at routing, and you probably won't be putting anything requiring fancy hardware drivers on your firewall.)
LISP is intended to do this. (Score:3, Interesting)
To do this properly with load sharing and immediate failover, at the moment the professional solution would be that you should
- get business class connections and
- run BGP over both links.
If you don't already know what BGP is, this solution is probably too complicated for you. Worse, the global BGP routing table is a shared expense, and your extra route would impose a (slight) extra cost on literally every other ISP running BGP. (The business class connections are because you will need several static fully routable IP addresses to do this, plus run BGP, and that requires more than a consumer class connection.)
There is a lot of discussion at the moment about this at the IETF, and people are working on something called LISP (no relation to the computer language), which would provide true multi-homing without the bother of running BGP and adding to the global routing tables. Things like immediate failover and load balancing should follow more or less automatically.
There is a lot more information available at Lisp4.net [lisp4.net]. I have heard of some initial testing, but in my opinion this is still a ways from commercial use.
Re:Point of failure (Score:3, Interesting)
I don't mean to dog cellular/wireless as a backup, but anything based on the POTS network is going to be more reliable in terms of being strong against blackouts and disaster. Latter day technologies are less likely so because generally the legal requirements for that strength are not there or are significantly less.
High-speed cable and DSL aren't that cheap (~$100/month and up) and T1's are cheap as hell nowadays (~$400/month is not uncommon, can be less) and you've got a 4 hour repair guarantee - if you're CO is online (they are built like bunkers), you'll be back up in 4 hours from almost any outage. Check with Speakeasy.net first as I think they have about the best service going, but there are other providers as well.
So, if your goal is primarily to gain additional uptime, go with a T1 - back that up with some kind of "unregulated" connection (cable/DSL) or wireless.
Another tack to pursue if cost-efficiency has a higher priority is using wi-fi to link to a neighbor [popsci.com]. Using some simple technology [instructables.com] expands the range of potential connectees considerably. Find someone with a different ISP than you (different Layer 1, that is) and get them to share with you - share both ways and you both get a reliable backup (as long as your network gear is on a nice big battery) for $0/month. Make sure neither of you scrimps on that battery equipment though! (Speakeasy encourages connection sharing and would even facilitate billing if desired even on their lower-end DSL connections if that becomes a problem/need.)
-Matt
P.S. Both of those links are step-by-steps, not theoretical articles. :-)
P.P.S. I'm not connected to Speakeasy in any way other than as a very satisfied former (for now) customer.
Aggrate routeing is a bad idea - here's why (Score:3, Interesting)
OK, so you have two routes to the internet. One packet departs, but is returned by the other route. How to glue those together is a very non-trivial problem.
Sprint tried that in 1997-2001 time frame with bonded T1 & T3 services. The bonding never worked for persistant connections, and only slightly better for transiant connections. UDP worked best. And that was using a routing system that understood it was bonded, not one completely unaware of another route.
These days $DAYJOB uses OC3's and SONET rings for Internet, so there may have been advances I'm unaware of, but back then, it really, really sucked. Off the cuff, I'd say use Linux and the Zebra package on a old computer, and try that, but no promises. Personally, I don't think it will work well.
Re:2 ISPs? Single provider. (Score:3, Interesting)
Umm, who are you to tell someone else what's worth it and what isn't? I can see a lot of situations where one would feel the extra $50 or so per month is worth it. For instance, if you're day trading from home, a 20 minute outage at the wrong time can cost a LOT more than $50. Additionally, being able to automatically choose the connection with the lowest ping time could be a benefit.
This is just one of many many examples of why somebody might want to have redundant home connections. Just because YOU don't have a use for it, that doesn't mean there aren't many many other people who would find this useful.
I think I'll be looking into exactly this in the next year or so, so this topic is very interesting to me.
Re:DSL+Cable (Score:1, Interesting)
I cannot believe anyone has yet piped up about OpenBSD. (www.openbsd.org) It is "free, functional and secure". In my opinion - when it comes to network tasks, I always use OpenBSD over any other *nix or let alone *BSD distro. As with anything, what you take in is what you can make of something. If you want a system that you control, understand and can fix up - I find OpenBSD very straight forward. I can barley install Fedora ("it's too easy") but give me an openbsd console and I am happy. Again, it is all personal preference. I'm just touting it as I've used it for nearly a decade in hundreds of places with the utmost success.
While it doesn't have any fancy front end, basically you follow the first FAQ (http://www.openbsd.org/faq/index.html) which is ridiculously well written and every general function is documented. It will run on virtually any bit of hardware and is fully configurable.
Comments such as 'get a redundant house' are obviously quite useless I don't even get why people make them. woot! mod them up funny! Never the less, fail over is great in case of hardware failure or basic service failure. I've lost power many times, but never my internet (or any system as they are all on adequately sized UPS's).
Simple fail over protocols such as VRRP, or HSRP the Cisco proprietary solution allow a hot spare to take over if the primary router/firewall fails. The term hot spare denotes the fact that the backup router/firewall doesn't actually do anything while it's waiting for the primary to fail. The newer Cisco proprietary protocol GLBP load balances traffic between two router/firewalls maximizes utilization of resources meaning you don't just have a piece of equipment sitting there, powered on, waiting for the primary to die. The OpenBSD group was originally going to code and deploy a VRRP implementation. VRRP, the virtual router redundancy protocol is an industry standard router redundancy protocol. It allows for failover to a hot spare secondary if the primary fails. The problem was when development was about halfway completed they were contacted by Cisco's lawyers, saying that even though VRRP is an open standard it infringes on the patents Cisco has on HSRP, and thus incompatible with OpenBSD's license structure. Basically they got a cease and desist. Not to be deterred the OpenBSD guys set out to design their own high availability fail over protocol. The result was CARP, or the common address redundancy protocol. Just like VRRP it allows for failover between two devices sharing the same address, unlike VRRP however it has another mode called ARP-BALANCE that allows traffic to be dynamically balanced between the two devices. CARP has very quick failover times, most time quick enough for TCP sessions not to be reset. In a demonstration of CARP technology the link between two routers was severed with a hatchet, and the secondary router took over fast enough that the Internet radio stream that was being listened to was not interrupted. Although this is a feature that is more useful to business I have a secondary router that I use to fail over in case I need to update or mess with the primary so I never lose my connection to the Internet. My torrents are important to me.
Broadbond (Score:4, Interesting)
I produce a system that can do this. It's called Broadbond [broadbond.org].
You can bond several ADSL lines, even from independent providers, and it will deliver the combined upstream and downstream bandwidth of the two. All traffic is load balanced across the two lines and can also be transparently compressed. The throughput of the lines is automatically measured to determine the optimal load balancing. Differences in latency on the two lines are compensated for.
The catch (there's always a catch!) is that you need to have a partnering system co-located with an ISP to handle the far end of the tunnel -- although I can also provide this if you would prefer.
The system is available as a software package that you can license to run on Linux or OpenBSD and also pre-installed and pre-configured on a couple of small embedded Linux boxes -- very low power (under 5W), no moving parts, good for up to 90Mbit/sec.
I bond two ADSL lines to my office, 4.4Mbit and 9.6Mbit, and I get around 13.5Mbit on file transfers.
If you're interested, contact me (details on the broadbond.org web page).
Dynamic routing idea (Score:3, Interesting)
I have DSL and cable. I also have a D-Link DL604 load balancing router. It sucks.
The router seems to think that as long as the physical ethernet connection is up, the provider is up. It tends not to detect network failure. There are ways to set up a periodic monitor of some host to detect if the network is up, but it does not seem to work properly.
What I want from this thing is:
Lock SMTP to one port and thus one provider. My AT&T DSL SMTP server will not accept mail from my Comcast account. (this is correct behavior for anti-spam). The DL 604 does this correctly.
I want the router to send any new connection for a naive (not currently in routing table) external network to both providers. I want it to measure the response time ( over a number of packets ) and then lock the route to the network which provides the best performance. It can periodically re-test the routes - perhaps every 5 minutes or so. This should address the problem of non-neutral peering between various providers. It is not always true that the higher bandwidth cable connection is the best connection to where I want to go. If I am accessing a client's machine who is on AT&T DSL, my DSL connection may be faster than my cable connection. I want the router to deeply inspect the traffic and be able to detect if a session breaks on a particular WAN port, and try the other. I also want it to quickly recognize when all sessions on a particular WAN port break and switch to the alternate port, while testing the original port.
I want built-in diagnostics that can show me how often a provider drops the ball, shiny graphs of bandwidth and latency etc. It would be cool if the router would allow me to see what the instant connection graph between my LAN and external networks looks like. ( which of my hosts connect to which external domains at the moment ).
I would like to be able to see graphics of IP address / port scans.
I want the router to be able to do some intrusion prevention, particularly if no one is using my network at the moment - someone tries to scan - shut the thing off for a while. ( do I care if I DOS myself if I am not using the net? NO! )
There is a hardware provider http://www.routerboard.com/ [routerboard.com] that can provide multi-wan multi-lan and wireless router hardware for cheap. They also have software but nothing that does all the tricks I want...
Coders, here's a base spec, send some bits!
OZ
Re:Point of failure (Score:5, Interesting)
You know, this is something I haven't been able to figure out. I live in Japan where we are hit by strong earthquakes at least a month, and typhoons (like hurricanes), thunderstorms, minor flooding, etc. almost every day during the rainy season. And no I don't live in central Tokyo, I live in the middle of a farming town and have to walk through flooded rice paddies to get from my apartment to the station. But my power and internet have NEVER gone out once in the 6 years I've lived here. We don't have anything special... the power and phone run on overhead lines on metal poles just like most places in the US.
Meanwhile, at my mom's house in the DC Metro area, USA, the power & internet go out every time there is anything more than a gentle breeze. What's going on?
There's always Cisco. (Score:2, Interesting)
Re:Point of failure (Score:2, Interesting)
In the US, the utility companies find "break and fix" to be less expensive than continuous maintenance.
We used to have tree-trimming crews come around every year. Now they wait for a big storm to knock over the trees, then bring in crews to clear the damage and fix the lines.
Re:Point of failure (Score:3, Interesting)
You can probably do some sort of ghetto load balancing with ipvs/keepalived and iproute2.
I'm just thinking out loud... all in all, you can probably do this without a whole lot of difficulty, but it really is probably going to require a linux router and 3 network interfaces... unless you want to plug both internet connections into a switch with all your other computers and use a bunch of static IPs and routes and whatnot...
Probably [the internet x 2] --$gt; [linux router] --- switch ==== other pc's.
Set it up with iproute two such that assuming 1.2.3.4 is your link to ISP1 and 4.3.2.1 is your link to ISP2: /etc/iproute2/rt_tables such that there's a table called ISP1 and ISP2 /sbin/ip rule add from 1.2.3.4/32 table ISP1 /sbin/ip rule add from 4.3.2.1/32 table ISP2 /sbin/ip route add default via 1.2.3.4 dev eth0 table ISP1 /sbin/ip route add default via 4.3.2.1 dev eth1 table ISP2
up both these IP's on eth0 and eth1.
set your default gateway to one or the other, i guess
edit
then ip route flush cache for good measure
up your 192.168.1.1 address on eth2
set up a DHCP server that serves out 192.168.1.0/24 addresses...
Then I guess you can set up ipvs on the linux router in some sort of NAT mode (i think it can do this)...
so you can make 192.168.1.1 your "virtual server", and set up... see this is where I'm not really sure about it, but i guess the remote gateway of both your ISP's to be the "real servers", set it up either weighted least connections or something, add persistence if you want, adjust the weights. Add keepalived to that, and tell it to ping the remote gateway and if it's not responsive to ping, to fail over to the other link (it'll insert and remove stuff out of the ipvsadm -L -n table).
Yeah, something like that. That's a metric asston of work, though, and i'm not sure it'd all work. You probably should just buy one of these:
http://tinyurl.com/5v5b8g [tinyurl.com]
I mean, they're a couple hundred bucks on froogle:
http://www.google.com/products?q=RV042&btnG=Search+Products [google.com]
and they've got 2 internet ports, four switchports (i mean, gigabit plox, but whatever), and a fancy web interface.
Meh.
~W
Re:Point of failure (Score:3, Interesting)
And your system was totally rebuilt after WWII. Ours has just been "maintained" by the "if it's not broken don't touch it" scheme since Edison.