Slashdot Log In
Dealing With ISPs That Use NXDomain Redirection?
Posted by
timothy
on Wed May 13, 2009 01:17 PM
from the looking-at-you-charter dept.
from the looking-at-you-charter dept.
Vrtigo1 writes "I work for a small company that has about 50 staff on the road relying on VPN back to our office at any given time. Many ISPs have implemented NXDomain redirection services that hijack DNS traffic to show you sponsored links and other related ads when you mistype a domain name. These services are incompatible with most VPN software, since they prevent the computer from resolving internal hostnames. Large ISPs typically provide an opt-out on their sponsored links page that immediately opts you out of the DNS redirection, but I've noticed that some smaller ISPs and CLECs have opt-out links that don't actually appear to do anything. I don't have a good solution for employees using these ISPs, and our employees are getting frustrated because the problem is becoming more prevalent and we can't fix it for them. I've tried calling a few of these smaller ISPs for help, but it's been like talking to a wall. Manually changing DNS servers works temporarily, but the user can't resolve internal hostnames when they connect to the office LAN again. Have you had to deal with ISPs using non-standard DNS servers? What is your solution?"
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
This is an easy one. (Score:5, Insightful)
If the small local ISP is screwing up, and refuses to respond in any useful way despite your best repeated efforts, it sounds like its time to take your business elsewhere, maybe to one of those large ISPs you mentioned. And make sure you tell them WHY. Who know, maybe the threat alone will be enough to get them to make a sudden change in policy for you, with a month or two of free service to boot.
Re:This is an easy one. (Score:5, Insightful)
Parent
could someone explain what the issue is here? (Score:5, Informative)
This guy sounds like a manager or IT worker who is having problems with his employees connecting to the work VPN.
it sounds more like he has not stated the problem correctly.
how is it possible that a VPN connection is doing DNS to an external name server? Should not every internet request flow over the vpn from the client to the server. once it reaches the internal vpn server the server should know how to route the internal addresses and for external addresses it could use an external domain name server. the problem described seems like it should not exist. what am I missing?
Parent
Re:could someone explain what the issue is here? (Score:5, Insightful)
Even if all traffic goes down the VPN wire, it's probably making those requests to the same DNS servers OVER the VPN. Bust since it's still the same DNS servers, it still gets the same results.
The IT guy would have to intercept all DNS requests over the VPN and proxy them to his own DNS server. That's not a bad answer. Too bad I'm buried in the middle of this thread.
Parent
Re: (Score:3, Insightful)
Re:could someone explain what the issue is here? (Score:4, Informative)
Parent
Re:could someone explain what the issue is here? (Score:5, Insightful)
Some VPNs only route traffic meant for certain destinations through the VPN as one network interface and allow traffic to the public Internet use the actual established connection.
They should be checking the internal DNS servers first (which should not promulgate requests up to public servers), and then the public servers.
Doing in the other order sends internal information (server names) over the public network.
Parent
Re:could someone explain what the issue is here? (Score:5, Informative)
Parent
Re:could someone explain what the issue is here? (Score:5, Insightful)
That's a hell of a security risk, having a client connected to both your internal network and external networks simultaneously.
Every corporate VPN I have ever used has, as part of its function, disabled all network interfaces other than the one it was using once a connection was established. In addition it would prevent any traffic from going through the "normal" connection. The idea was that a machine should never have connectivity to both the internal network and the outside world simultaneously.
The article poster doesn't need to fix their users' ISPs, they need to fix a horrifically broken and insecure VPN system.
Parent
Re:could someone explain what the issue is here? (Score:5, Informative)
I have never seen that enforced, and only twice ever as the default setting. It is a client-side configuration option in most VPN software (Cisco, SecuRemote, most Linux VPN clients).
You want VPN users to stream video or download game patches or do other non-business-related bandwidth intensive operations over the VPN, when they have a perfectly (ha!) good internet connection locally? I hope you have a REALLY big network pipe.
Parent
Re:could someone explain what the issue is here? (Score:4, Insightful)
Enforced at my work. In addition we don't allow user's personal machines onto the VPN. Since it's a company notebook on the VPN and all traffic goes through the VPN, we also enforce the internal AUP on remote users using the VPN. That means downloading a game patch will get you a stern talking to, downloading porn or torrents of wares, etc. will get you fired.
Again, this is all acceptable, because you are on company equipment (even if you are at home). If the case is that employees are being allowed to attach their personal equipment through the VPN to the company's internal network then I really hope you totally trust your employees, because one rogue could catastrophically hose you.
-nB
Parent
Re:could someone explain what the issue is here? (Score:5, Insightful)
And this is an excellent plan for convincing your users to only connect to the VPN occasionally. Good if you want to maximize security. Bad if you want to maximize productivity.
Parent
Re:could someone explain what the issue is here? (Score:5, Insightful)
I'm not sure what your threat model is, but I suspect you are claiming one of two things: either that the VPN node might act as a router, forwarding packets around your firewall, or that the VPN node might be compromised and used as a stepping-stone onto your network.
In the case of the router vulnerability, this is something that you can control on the corporate side of things by simply not accepting packets down the VPN tunnel that don't come from the IP address that's the far endpoint of that tunnel. I'm not a VPN expert, but I would be surprised if this isn't how your VPN is configured by default.
In the case of the stepping stone, this is a fairly weak threat model, for two reasons. First, if your machine has been rooted, there's a good chance that it will phone home out through your firewall even if you route all internet access through the VPN. So it will be a stepping stone to your network anyway.
Second, if your machine has been rooted, and is running any sort of virus platform, it's going to try to infect machines on your network even if it doesn't have a link to the outside world. If you are genuinely concerned about threats originating on employee laptops, you shouldn't allow them to VPN into your network at all.
So the point is that forcing the VPN'd node to access the internet through your site is probably going to be a big inconvenience for your users (the kind of inconvenience they will hack around, possibly making you even more vulnerable) and it's not going to buy you any meaningful security.
Firewalls are great for slowing the spread of infection, and raising the cost of attacking you, but you really do need to secure every node as well, and if someone really wants to get past your firewall, and is willing to expend substantial effort to do so, you probably won't stop them without much sterner measures than the one you're advocating.
Parent
VPN does not preclude packet filtering (Score:3, Interesting)
In the case of the router vulnerability, this is something that you can control on the corporate side of things by simply not accepting packets down the VPN tunnel that don't come from the IP address that's the far endpoint of that tunnel. I'm not a VPN expert, but I would be surprised if this isn't how your VPN is configured by default.
You can also filter packets on the receiving end of the VPN. That's how I configured our firewall at work. The VPN tunnel simply looks like another network interface to our firewall, so I apply a slightly less restrictive set of rules to that connection than I do to the default external interface. Giving someone keys to your network just because they are an authenticated VPN user is not a very good idea.
My main complaint with DNS tampering is the outright DNS hijacking that Sprint does with their AirC
Re: (Score:3, Interesting)
Machines that connect through a VPN client are only behind your firewall some of the time. They cannot be trusted to be virus free. The firewall needs to keep them, the LAN clients and the servers separate anyway.
Another point is that if the machine has been infected, that means that the software on it has been altered. The VPN client software is not immune to this. You may THINK split tunneling is disabled, but are you sure?
There may be other issues as well. If you WANT people to come in through their pers
MOD PARENT UP: Re:could someone explain what th... (Score:5, Informative)
This is in fact why NXDomain breaks things in the way the poster describes, however, unless you're the kind of employer who wants to see EVERYTHING your subordinates are doing it's not actually the best practice to filter everything through the VPN.
Filtering everything through their VPN increases overall costs in bandwidth and hardware as Intron indicated. These are very real, very costly expenses that many employers overlook when implementing broad policies... and it's a fantastic point you raised that all too many companies forget.
Why should my connection to slashdot.org, for example, be secure on the company VPN? My ssh and nfs connections have very real reasons to be secure however!! On the other hand you could fix this by filtering DNS traffic through the VPN, but not web traffic. The cost of DNS traffic is marginal comparatively to other services, but the benefit for companies facing these specific issues is obvious.
Parent
Re: (Score:3, Informative)
SSH tunnels get around that without difficulty. If you know the address, it's as simple as assigning local port 2222 to 10.1.0.100:22 and you can now SSH to that machine by connecting to localhost:2222. Get a SOCKS capable SSH client, and you don't need to set up the tunnel for each connection.
Re: (Score:2)
Good luck in finding an ISP that doesn't screw up the DNS in some way.
Re:This is an easy one. (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
Good point. They should thank the ISP for this alert.
Re:This is an easy one. (Score:5, Informative)
Either way, what I do when I have some kind of weird situation where a user needs to change their TCP/IP config routinely is just put a couple shortcuts with pretty icons on their desktop that point to batch scripts that run a netsh script. You should be able to completely change an IP configuration on a Windows box with this utility, the user just runs "home.bat" when they're home and then "office.bat" when in the office. A Google for "netsh exec" should give enough info to get started.
Parent
Provide your own DNS? (Score:5, Informative)
Last time I setup a VPN, was with a Cisco PIX firewall, (its been awhile) but there was a spot to specify which DNS servers to use when connected to the VPN. I had specified that when connected, they would use our DNS, since they otherwise couldn't resolve \\file-server\share or whatever..
Re:Provide your own DNS? (Score:5, Informative)
Parent
Re:Provide your own DNS? (Score:5, Insightful)
You're right. It all boils down to misconfigured VPN
Parent
Mod parents up (Score:5, Funny)
Mod parents up, please.
And then we can all go home. This is an easy problem to solve once you see it from the right angle, and that angle is described above.
Parent
Change VPN settings . . . (Score:5, Insightful)
to force use of internal DNS servers while connected.
Done.
Re: (Score:3, Insightful)
I guess I did not know there was an option not to use the internal servers.
Our unit has its own domain and dns servers. The zone does get replicated to the central dns servers, but we have to use the Fully Qualified Domain Name of our servers when on computers outside our unit.
Have the users try the full name of the server and see if that helps.
Use Full Tunnels (Score:5, Informative)
If you're splitting your connection between a VPN tunnel and a non-VPN protected internet connection, you're a security risk to your infrastructure.
Have your administrator configure full tunnel support where ALL of your traffic goes through the encrypted tunnel. That solves a security problem AND it fixes your DNS problem because you don't use your local internet provider's DNS servers.
Re:Use Full Tunnels (Score:5, Informative)
This is called split tunneling. If he disables split tunneling and specifies the DNS servers in the VPN config his problems would go away.
His users however would tunnel all their traffic through the corporate lan while connected so you may need to setup some kind of filtering or route the traffic through whatever filters you already have. Otherwise these remote workers in hotel rooms will be pulling buckets-o-pr0n through your corp network.
Parent
Re: (Score:2, Informative)
But that's only a problem when they're connected to the VPN. Don't surf porn while on the VPN, don't get fired. Win/Win
Just disconnect to download your porn and you're good.
Re: (Score:2)
Re: (Score:3, Insightful)
Split-horizon DNS (Score:3, Informative)
http://en.wikipedia.org/wiki/Split-horizon_DNS [wikipedia.org]
Easiest solution: Get them to change ISP. (Score:2)
Failing that...
Why does manually changing DNS servers work only temporarily? Can't you just host a DNS server and give your employees the IP for that? It'd mean having to service DNS requests for all your employees private internet usage plus it might break some CDNs but it seems like the simplest solution.
You could also loan employees suitable ADSL / cable routers that you configure, something with a decent small DNS server in it that you can configure to serve your intranet hostnames but defer to the user
Re: (Score:2)
Failing that...
Why does manually changing DNS servers work only temporarily
Perhaps that's the reason cron exists: to make sure your DNS servers are reset to your preference despite DHCP mangling them.
Our Solution (Score:2)
Our Company webserver and mailservers serve as DNS servers as well.
There are four in total. We are an ISP, but we are dependant on a larger backbone - so we registered our own DNS servers.
Also, DHCP on the lan with your own DNS server on LAN side should be fine, and you can also edit the hosts file if all else fails. We have a few (Vista) laptops where we needed to hardconfig LAN side server addresses in the hosts file - but I suspect this has less to do with nxdomain problems than with a larger config issu
Stop filtering your DNS, or run a local cache. (Score:4, Insightful)
What's the benefit of blocking your internal DNS? You're firewalled off, or they wouldn't need the VPN. What's going on here is that you're doing something broken - you must have some kind of NXDOMAIN redirector running on the remote machine, and the ISP is doing something wrong, because its NXDOMAIN redirector is fooling your NXDOMAIN redirector. If you just follow the standards, the fact that they have a broken NXDOMAIN redirector wouldn't affect you.
Another option is to set up a DNS resolver that's reachable from outside your network, and also inside your network, but only answers for your internal names if the query comes from inside. Then configure all your VPN machines to always use that nameserver, and not use your ISP's nameserver.
Even if your ISP filters DNS and answers in place of your nameserver, you're okay, because as soon as the VPN is set up, all the queries will go across the VPN (since this server is on your local network). At that point you'll start getting answers for local domains because now the query is coming from a local (VPN) IP address.
This second solution is a bit more work, and of course being a DNS geek I'm biased toward just doing the right thing in the first place, so I recommend just opening up your DNS, but either way ought to work.
What small ISPs? (Score:5, Funny)
There are still small ISPs left where you live?
Re: (Score:2)
I was going to laugh because this is probably true for most people in the US... then I realized that this wasnt funny because this is probably true for most people in the US. I think we need a "Sadly, humorously, funny" Mod...
Plenty of other DNS options... (Score:2)
You can run your own DNS servers... (this opens a lot of other possibilities for it's use as well - such as blocking certain sites at the DNS level, or setting up local domain entries for your internal network (without the expense of registering a domain name or three): just make sure you dont set such up using a real, existing domain name that you may at some time want to visit.
A Linux box with BIND or similar can be a cheap, old box and perform fantastically in this respect. An OS/2 box (if you've got so
Uhhh (Score:2)
No VPN software or hardware I ever used does this. It always checks the VPN DNS server first before going to the main one.
Reconfigure your VPN software, something is wrong.
YES, NXDOMAIN redirection sucks, but it does not by default interfere the way you think it does.
If it's servers on your network you need, you could just stick a hosts file entry on their computers to resolve "webserver" to 10.1.200.34 etc.
Re: (Score:3, Insightful)
I wonder if the actual problem is this:
1. User goes to internal site, gets ISP not found page.
2. User goes "Whoops, need to turn on VPN". Turns on VPN
3. User hits refresh. Still goes to ISP not found page.
Is he sure this isn't an issue of just needing the user to close their browsers to clear the browser dns cache?
This is an unethical practice (Score:3, Insightful)
Redirecting my web request to somewhere else, as far as I am concerned, is equivalent to re-routing my snail mail to their own office if someone has moved. That is not acceptable. I want a "not at this address" notice, nothing else.
hosts file? (Score:3, Informative)
A logon script here loads a hosts file that null-routes a lot of known bad (spyware, etc) sites.
Could you do the same for your internal hosts so that when on the VPN it doesn't even need to do a DNS lookup?
Your VPN is busted (Score:3, Insightful)
The first thing your secure VPN tunnel should be doing is altering the client's DNS profile to only use the DNS servers on the other side of the tunnel. Anything else is totally insecure.
Large ISPs (cough, verizon, cough) lie about it? (Score:3, Informative)
That's not limited to small ISPs. Verizon FiOS, for example:
"Oh, sure, we will let you opt out - just click on the link that shows your router"
BROKEN LINK
Hmmm, guess I will click on a similar router...
THEY ARE ALL BAD LINKS
Gee, I guess I will click on the "change OS settings" link then...
BAD LINK
Somebody's going to point out that you can Google and find where helpful geeks have posted the instructions to opt-out without Verizon's assistance. But that's not the point, really, is it? Verizon had working opt-out links exactly long enough to get a favorable review in Consumer Reports, and then it all mysteriously broke. I cannot explain this coincidence, personally, you will have to come to your own conclusions.
Re: (Score:2)
Then don't even do/suggest it.
Quick and dirty only gets dirtier and wastes the time you saved upfront and more later on.
Doesn't OpenDNS also use NXDomain Redir? (Score:2)
It's been awhile since I looked at OpenDNS, so maybe I'm mis-remembering, but I could swear that OpenDNS's business model is based around generating ad revenue from doing NXDomain redirection, isn't it? If that's the case, swapping one NXDomain redirect for another doesn't seem very productive.
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)