Geographical Mapping of Website Traffic? 20
inputsprocket asks: "Having gotten into a project that involves doing some geo-statistic data mining analysis on Apache log files, I couldn't believe how there seems to be such a large chasm, in open source's vast library, for such a powerful analytical tool. Unless I'm wrong, then there is no such open source project available for *nix variants. There appears to be a great deal of brand software (heh — only for Windows off-site analysis though), yet there is are great open source tools for generic mapping tools (GMTs). The available GMTs seem to be overkill for mapping of weblogs and require a lot of in-house programming to map IP addresses to cartographic data using free geo-location databases. The closest I found was Palentir. Does one really have to rely on the powerful needs of Map Server, MapTools, Carto Web, or Map Builder to draw simple maps with statistics on web usage, given the extra programming needs required to do this?
"To see what is possible, have a look at DeepMetrix's LiveStats.XSP — a commercial option, which isn't a problem. It's just a shame that it's only for Windows and for of-site browsing.
Somebody, please tell me there's more to open source geolocation of web traffic than Analog."
Somebody, please tell me there's more to open source geolocation of web traffic than Analog."
Try hostip.info (Score:4, Interesting)
On their front-page right now is a geographic map of website traffic, which is a "public-beta" of a mapping system (upload IP addresses, get a map).
They don't actually appear to give out the code (from only a quick look on the site, that may not be correct), but this ought to be pretty trivial to do yourself if you wanted... The public API of hostip.info returns the lat/long of any IP address it knows, and plotting lat/long on an image of the world ought not be too hard.
I designed the whole system so you could download and set up your own geolocation database, slaved from the master at hostip.info with regular updates. Then you can query IP addresses locally and generate charts however/whenever you wanted...
Just a thought,
Simon
NeXT (Score:3, Interesting)
perl and cpan are the answer (Score:3, Informative)
Google? (Score:4, Informative)
Azureus (Score:3, Interesting)
That's Open Source -- so look at their source.
Azureus-RIAA. (Score:3, Funny)
Thanks
ip2location (Score:3, Informative)
Re:ip2location: accurary != precision (Score:1)
I can't see much point in anything that is supposedly more "accurate" than by country. Using the demo site there I got either just "UK" or "UK, London" for a whole bunch of addresses dispersed around the UK. Seeing as most home users will be on some xDSL connection which is probably peered/registered in London/$MAJORCITY, but they are perhaps a few hundred miles away it doesn't seem very "accurate" just more precise (but wrong). Maybe for the US you'll at least get an idea of which time zone they might be i
Re: (Score:2)
If you want to find out where your site's users live, ask them!
Re: (Score:2)
Hee hee ... I just went to http://ip2location.com/ [ip2location.com] , and they informed me that I was located at "(US) UNITED STATES,
VIRGINIA,
HERNDON." Mm...no, I'm in Los Angeles, and no, AOL isn't even my ISP :-)
I have at least two misgivings here:
OpenLayers.org, OSGeo Foundation and GeoRSS (Score:3, Informative)
While your at it, GeoRSS is also open and, in my opinion, the future of RSS. Learn more about it here [georss.org].
Invented On Slashdot (Score:3, Informative)
Perl Module Geo::IP and MaxMind Database work ... (Score:4, Informative)
So with that, it wasn't too hard to write another Perl script that parses the IP address from the Apache logs, acquires misc. geolocation info, and then dumps out the lat/long data in Javascript for the Google Maps API - see this graphically shown on my christmas maps page. [komar.org]
Re:Perl Module Geo::IP and MaxMind Database work . (Score:2)
Re: (Score:2)
please get in touch if you wish, but I did some Apache benchmarks with Geo::IP and Pear's GeoIP, both using Maxmind's free database and while Pear was quering at 1/5ms on 100 concurrent connections, Geo::IP was answering 1:180ms but with apache balking after 2000 request...
This really disappointed me and as such we only run PHP on the maxmind dbase....
I never looked into why the delay was there, but care to comment?
phpmyvisites (Score:4, Interesting)
Replying to my own question, but I think I found the most complete OS project out there for geolocation log statistics....
I've been playing with phpMyVisites [phpmyvisites.us] and it looks a very nice and thorough implementation of what I was looking for.
Although it's method for country geolocation is a bit *cough* rough, I think a module to hook into Space Cowboy's hostip.info [hostip.info] shouldn't be too difficult...
The only problem with it is it uses mysql to store the logs *grrrrr*
OpenLayers [openlayers.org] looks slick!
Re: (Score:1)
They don't distinguish between Time-Warner customers in Chicago versus New York, so you get a lot of DC-area longitude and lattitude URLs.
ClustrMaps (Score:1, Informative)