Mirror Listings Though TXT DNS Records? 32
mackman asks: "I was wondering if anyone has ever though about using their DNS servers to provide mirror information? A specially formated TXT-record could easily provide a DNS-cache-friendly mirror listing. A TXT-record would just need a list of servers and paths, or perhaps a more complicated mapping for servers which only mirror a subset of the original site. This would allow for much more flexibility than a basic round-robin A-record scheme. For example, instead of pounding the Red Hat web server to get a mirror listing (or relying on Slashdot posts for that matter), why not do a 'dig -t txt mirrors.redhat.com'? Of course we could build this into download managers like wget."
Good idea! (Score:5, Insightful)
But for more usability the record should include the location (country and continent) of each mirror, so your favourite download manager can try the nearest mirrors first.
Another question is how quick such lists would be replicated/cached by your ISP's DNS server...
Interesting... but no (Score:1)
There's alot to be said for keeping something that basic to the working of the internet as we know it as simple as possible.
Also, if I understand correctly, that kind of thing would increase the load on the domain servers, especially with that sub-site idea. And I just can't see any widespread adoption of that kind of thing anytime soon.
Re:Interesting... but no (Score:5, Insightful)
Exactly.
DNS is supposed to provide nameIP mappings, in an application-neutral fashion.
By trying to make it provide "mirror" information, you're moving into the realm of the application - which doesn't really map properly to the concept of IP address.
The question (of course) is "mirror for what?"
HTTP content? Why only HTTP? FTP sites are commonly mirrored as well. How do you distinguish between the two? If you do a lookup for the example provided - "mirrors.redhat.com" - which service is mirrored, the HTTP, or the FTP? How do you know if they have different content? How do you know if they have the same content? And what happens if you have multiple HTTP servers on the same machine? (Say, one running on port 80, and one running on port 8080.)
Which one gets mirrored? Since you can't do a DNS lookup for "www.redhat.com:8080", this makes it impossible to provide a mirror for anything that doesn't use the WK port.
Re:Interesting... but no (Score:3, Informative)
Then what are these mx records being used to relay mail. That's not exactly application neutral. You can place any random text you want to in a dns record. If you do a host -a you will occasionally come across people who have thrown completely random shit into their dns record. The A record is what is used to resolve a hostname to an ip address. Here is a list of other record types from nslookup(1):
A the host's Internet address.
CNAME the canonical name for an alias.
HINFO the host CPU and operating system type.
MX the mail exchanger.
NS the name server for the named zone.
SOA the domain's ``start-of-authority'' information.
TXT the text information.
WKS the supported well-known services.
Other types (ANY, AXFR, MB, MD, MF, NULL) are described in the RFC-1035 document.
I seem to recall something about jabber using a dns record similar to mx. A quick google search comes up with this page [vanrein.org] that claims both ldap and jabber use srv records and that jabber formerly used the mx record.
So yeah, dns, more than just ip addresses and hostnames.
Re:Interesting... but no (Score:3, Informative)
I said that DNS isn't the proper way to do this, and here's why - and you completely ignored all of the "why"..
Then what are these mx records being used to relay mail.
They're used to find the MX for a domain. - To find an IP address, just like I said.
That's not exactly application neutral.
Yes, it is. MX records don't include information on where the mail aliases to, or where to find the administrator for a domain (for example). It's used to provide a record for a Domain. (You know, the "D" in DNS.)
You can place any random text you want to in a dns record
Just because you can do something, that all of a sudden makes it a good idea to do it?
By that logic, I bet you can jump off a cliff.
Here is a list of other record types from nslookup(1)
What's your point? In every case, the records provide information about the domain, or about a host in that domain. In no case do they provide information about pathnames, mirrors, the name and address of the administrator of the domain, or anything that's handled by an application-specific protocol.
Re:Interesting... but no (Score:1)
okay. so you use an mx record to find a server that handles mail for that domain. What's wrong with having an FX (file exchanger) record to find a server that handles ftp for that domain? This idea does that without requiring a new standard for dns, by using a currently available field in a DNS record.
Re:Interesting... but no (Score:2, Insightful)
My point is, you can nitpick details about the existing protocols/DNS record types and for what they were originally intended but that doesn't mean it's the wrong type of architecture.
If you were building a system from the ground up to be a mirror directory for various well-known services it would probably end up looking a bit like DNS in its hierarchy (and probably, semantics).
That being said, I think the parent post's idea is problematic in more significant regard in that it tries to assign a standard meaning to an unstandardized record type which could only wreak chaos with DNS servers that did not support it.
I think a more "correct" approach could involve adding an additional record type to DNS that enumerated available mirrors for a host's well-known services that were listed in its WKS records. It could return results similar to an MX query (service [e.g. ftp], host name, priority) along with port and country information.
How about ENRP? (Score:5, Interesting)
The IETF has some great things cooking. RSerPool [ietf.org], e.g. -- though it's not designed primarily for load-distribution (rather uptime, AFAIK). Perhaps I'm suggesting you use a hammer to crack a nut, but this might abate the need for posting mirrors, like you said.
Open Source is such a beautiful thing. It's unfortunate that so much of its energy is spent writing competing implementations of various similar protocols. But I digress.
Re:How about ENRP? (Score:2, Informative)
Seems more than appropriate for mirror information to me
Already done, of a sort (Score:2, Interesting)
Re:Already done, of a sort (Score:2, Informative)
It provides reasonable load balancing, but isn't fault tolerant (i.e. a server goes down, requests aren't redirected).
Works well enough for me though...
Doug
Look up what Hesiod can do.. (Score:2)
RFC 2052 (Score:5, Interesting)
Who will impliment it? (Score:2, Insightful)
If you are talking about websites like the slashdot effect, good luck on getting anyone to implement and maintain the table. A viable option for slashdot effect would be to mirror it in your journal. Or you could use kazaa.
Re:Linux is dying (Score:1)
It sounds like you're quoting accurately, but years ago I was talking to a guy at a Microcenter who seemed to know what he was talking about. Then he insisted that Apple had gone out of business. I couldn't convince the guy that there was still an operating business at 1 infinite loop. But he _sure_ sounded like he knew what he was talking about.
How about _specific_ URLs to back up your quotes?
Go for it! (but don't use TXT) (Score:5, Informative)
I wouldn't use TXT records, though. SRV records might be more appropriate, but people are apparently using SRV for LDAP nowadays so you're probably better off just defining a new type. Use the procedure laid out in the RFCs for designating a new type of "experimental" DNS record, so that everyone will be compatible with you (well, everyone that follows the RFCs, anyway, which might leave out MS and DJB) or figure out how to use one of the obsolete record types.
You can look at some of the other types of records in RFCs 1035, 1183, 1706, 1886, 2672, 2782, and 2874. Most of the important ones (well, other than IPv6 stuff) are in RFC1035 [rfc-editor.org].
Incidentally, you're going to get dozens (if not hundreds) of posts here that will say "You shouldn't do that with DNS!". Pretty much any question you ask around here that even mentions DNS will get the same result, so just ignore them. I think these people are a lot more interested in shooting down ideas than in actually testing them, which is the only real way to find out if they will work.
Re:Go for it! (but don't use TXT) (Score:1)
The larger problem is how to model the information. DNS already supports the concept of storing geo-positional information. The problem is that the raw location isn't indicitive of the best mirror. The network topology needs to be understood and properly recorded, never mind the security implications.
An alternative (Score:2, Interesting)
The exact method of the guessing is tricky, a rDNS can be used, but it migt be slow, and not all non-US ISPs (or companies) use their country TLD in thier canonical or alias names. The visual-traceroute style locactions might be more accurate, but the lookup time might be prohibitivly expensive.
* This generally true unless the program does its own lookups (some versions of Netscape), or the machine is running its own DNS server.
Why not use HTTP? (Score:2, Interesting)
I think HTTP is a better place for this.
Consider the following:
So, you could have a server "downloads.mycompany.com", which takes a request for file "/dir/file.tar.gz", and then issues a redirect to an appropriate download server ("http://useastcoast.downloads.mycompany.com/dir/f ile.tar.gz") based on a number of variables including HTTP headers, Client IP Address, and server load.
Since the functionality can be implemented on an HTTP server transparent to browsers, I would not be inclined to avoid changing the DNS (or any other) protocol to implement this.
As a side note, I do think that sites should implement this transparently, whenever possible. I get annoyed when I am asked to "pick a mirror"; I think that is the server's responsibility.
Just my two cents.
URN (Score:1)
Squid implemented some basic URN stuff a while back - It'd run a cgi script at a URL which returned a list of sites in a specific format. Squid would then ping the sites and return them to you w/ the RTTs. You could then choose the 'best one'.
Not very effective - and I don't think it (the squid side) ever progressed past the initial stages - but maybe its the beginning for someone to tackle this issue again.
(IMHO it could be made more
Note - there's RFCs covering URNs for various things (IETF documents, ISBNs, etc)..
No, DNS isn't really designed for this. Yes, DNS has been perverted enough, so I guess this'd just be another extension. Write an RFC, get it published..
Akamai (Score:3, Informative)
Akamai [akamai.com] already do something like this, and they didn't need to extend the DNS protocol in any way. They even provide the mirrors for you, if you use their service, and take care of ensuring that users get redirected to the nearest mirror. Sure beats having to scroll through a long list like you have to on sites that mirror the old-fashioned way, particularly since a site physically close to you might actually have less bandwidth than a faraway one.
This way lies madness (Score:3)
Drop a note to the IETF, get yourself a proposed RFC number, and go for it.
xml or text format? (Score:3, Insightful)
The sort of information the file would contain would include (5 second analysis):
If the browser sees that the current server's address is not the same as the listed orginal then it knows than it is a mirror. Also, the mirror file would have the ability to list all the sites that are mirrored on the server.
This file could then be be read by a search Engine and then when it shows the site in the search results it could list the associated mirrors. The file would probably be called mirrors.txt or mirrors.xml, depending on the format used.
This is my 5c worth. If you like the idea, then consider it free for the taking