Digitizing and Geocoding Old Maps? 235
alobar72 writes "I have quite a few old maps (several hundreds; 100+ years old, some are already damaged – so time is not on my side). What I want to do is to digitize them and to apply geo-coordinates to them so I can use them as overlays for openstreetmap data or such. Obviously I cannot put those maps onto my €80 scanner and go. Some of them are really large (1.5m x 1.5m roughly, I believe) and they need to be treated with great care because the paper is partly damaged. So firstly I need a method or service provider that can do the digitizing without damaging them. Secondly I need a hint what the best method is to apply geo coordinates to those maps then. The maps are old and landscape and places have changed, it maybe difficult to identify exact spots. So: are there any experiences or tips I could use?"
dig camera (Score:2, Informative)
Faster than the scanner and stitching.
Is this in the US? (Score:3, Informative)
If so, give USGS a call. They may well be interested in helping you with this and obtaining data from the maps. I can't say for sure, of course, but this is the sort of thing they do. When it comes to map data for the US, they are the go to guys. Call them up, tell them what you've got and what you want to do, see if they can put you in touch with someone in their agency who'd be interested in helping.
Triangles (Score:1, Informative)
Not ALL features will change. Once you get them to a scale you can use, use three points on both maps, such as mountain peaks or a particular coastal feature. When overlaid and aligned, it should be fairly accurate.
Some Inexpensive Methods for Digitizing (Score:5, Informative)
Assuming all those avenues are exhausted, let's look at some cheap and dirty DIY methods. I'm assuming you've got a MP digital camera. There are sub $100 ten megapixel cameras [amazon.com] out there but don't get anything with a fancy digital zoom. Next you'll need mosaicking software [aolej.com] or if you're into software, you can try your own implementation of the KLT algorithm [clemson.edu].
First off, practice all of this on layed out newspapers while developing your preferred methodology.
Your cheapest and most haphazard option is going to be lay the maps flat on the floor and cut a length of string with a washer on it (two to three feet?). Try to use brightly diffused lighting so that is normalized in the mosaics with no shots of your shadow over the maps. Now this is backbreaking but hold the camera flat over the map with the string extended in front of it so you can keep the distance to the map consistent. Don't angle the camer as this will slightly distort that tile and hinder the mosaicking. Put plastic bags on your feet if you need to walk on the maps. Take a picture, move a few feet in a grid style, take another picture. Rinse, wash, repeat until you have images covering all of the map. Collect the images and put them on the computer and verify the mosiacking works before preparing the map for storage forever.
A better method would be similar but to construct a large wooden rectangular box with plexiglass as a top so that you can fit this structure over the largest of the maps. Then cut holes in the plexiglass so that you can set your camera at a plane level to the surface of the map into the plexiglass. You might want to put an adapter on your camer that allows the lens and flash to be free of obstruction. You could make the tiles more uniform and save your back some work but you need to build and buy the materials for the structure. I think this is more time consuming but your best bet and will allow you to gather more images with less distortion.
Above all, remember to save the original images! It's probable that later better algorithms will be developed to normalize the images, remove distortions, light problems, shadows and increase clarity on your overlapping sections. If you do the plexiglass route, you could manufacture it so that every bit of the map is photographed three or four times.
Not professional, not flawless but cheap and dirty. Hope this helps.
As for the geocoding, what are the maps of? You should actually check out the feature extraction of the KLT algorithm and consider using that methodology for syncing these up with maps. That will require human intervention though to identify the features, I'm sure.
Hard to Do (Score:5, Informative)
Re:Contact a Museum (Score:3, Informative)
I agree, you might try contacting someplace like David Rumsey Historical Map Collection [davidrumsey.com] to see if they would be interested in helping, or might otherwise make recommendations.
A collection of other links that might be of interest:
Historical Map Web Sites [utexas.edu]
Re:Digital Camera (Score:2, Informative)
There has to be somewhere that does this.
Where I work we have a medium format scan back we use (it is essentially a 100 mega-pixel camera that scans across over about 15 minutes to get an image).
We are in Wilmington, DE, which is a fairly small city. I know there is a similar device in DC, which is not too far. We went to the Library of congress and actually scanned maps there as a test before purchasing it.
It actually allows for some interesting capture as one can adjust the lighting in different ways to get details a flatbed may not (we replaced a 2'x3' flatbed with it).
It could probably do the 1.5m one in two scans, and most smaller ones in one scan.
Look around your area for a place that offers art-reproduction for the service.
Comment removed (Score:5, Informative)
Talk to a curator (Score:5, Informative)
Another possibility would be a really high resolution digital camera. My wife (historical linguist) has dealt some with manuscripts, and that was their method of digitizing them for further study. OTOH, she's not a museum curator or archivist; they probably have even better methods. If you want to do it right, talk to a curator or archivist of some sort. They deal with much more fragile and much more valuable documents on a regular basis.
I don't have any good ideas to contribute about the geocoding, unfortunately.
We can help! (Score:5, Informative)
First, ask on the OpenStreetMap mailing lists. There's lots of us who've done this kind of stuff before, and we'd be really pleased to help. I collected, scanned and rectified the Ordnance Survey's New Popular Edition - a complete set of England and Wales maps from the '50s, now out of copyright. It's all available in OpenStreetMap as a background layer and loads of people use it for adding rural roads, rivers, placenamese etc. Others are scanning other old Ordnance Survey series right now. Seriously, we love this kind of stuff. (#osm on OFTC can help too.)
Secondly, GDAL [gdal.org] is definitely your friend. It's the most amazing set of command-line tools for rectifying and reprojecting data. gdalwarp and gdal_translate are probably the two you'll use most.
Comment removed (Score:2, Informative)
Re:Handheld scanner (Score:3, Informative)
I was going to suggest the same thing -- two birds with one stone. I personally use Hugin for things like that. You take many high-res, overlapping photos. You can automatically match them up with autopano-sift and then use vertical and horizontal alignment points to stretch them out as you would prefer. If the results aren't close enough to use as an overlay as-is, if you had a hires modern map, you could load it and set the FOV to roughly match up with the FOV of your fully aligned pieced-together map, and then define control points between it and your map pieces. Optimization will then stretch the pieces to try to fit to your modern map while still being pieced together.
Re:Handheld scanner (Score:3, Informative)
Yowza, that would be a royal pain to get results.
Two ways to go.
1. Wide format scanner. These are usually at more specialized digitization shops. Find someone who scans blueprints in your area. http://www.amazon.com/Designjet-Large-format-Scanning-Software-Intergrate/dp/B000E8Z0XU [amazon.com]
Only you can judge if the documents will be okay through the feeder. The feeders aren't hard on documents. I'd give your best one a shot. Naturally, you want to be there. So, not every service provider will be okay with that.
2. You most certainly can use a flatbed scanner. The key will be stitching software and memory/cpu resources and refining the scan/stitch method. Make them big-ish files, maybe 300ppi. After 300ppi, any information is useless for a 1:1 reproduction.
Lastly, overlaying geocoordinates info won't quite work as elegantly as you think. Ignore my doubts and go for it. I think the end result would be more art than science if done well. If done well, there will probably be a couple of false starts.
QGIS or ArcGIS for georeferencing (Score:3, Informative)
Hi there, I am a spatial guy so thought my 0.02 may be worth something. I am not too sure about digitising them, maybe a print shop or as suggested in other posts you could talk to your local university geography department or a government mapping agency
Once they are digital though you need to georeference them. As mentioned in the title of my post, it is easiest to use GIS to do this and you can use QGIS with relative ease. Install it using osgeo4w [osgeo.org] on windows or the ubuntu ppa for qgis [launchpad.net]. Alternatively if you have a license then use ArcGIS. If you have a map of the underlying roads for the maps you are digitising then what you do is find points on the roads and match them to points on the scanned images, this provides data for a transformation and will shift the map onto your coordinates.
Geocoding (Score:5, Informative)
Others seem to be describing some good solutions to getting the map scanned, so here's how to geocode and rectify the image using the open source Grass GIS software:
Step 0:
- You need to have a location already created in grass, with some contemporary data in it (physical features, roads etc where there's some concurrence with the map you're trying to geocode). The projection you've used doesn't matter much - a later step is going to be rectifying (ie distorting) the scanned map to match the projection of the digital map. The created location does need be at least as large as the scanned map (ie if the map is everything in a 5 km radius of some town, the grass location also needs to encompass at least a 5 km radius of the same town).
Step 1:
- Come up with a list of features/points which exist on both maps. Depending on the scale of the map, this could be intersections of specific roads, locations of towns, peaks of mountains etc. You're going to need an absolute minimum of five points for the rectification process to have any chance of working; more than fifteen is much better. Try and select points which are unlikely to have moved over time (coastline or river features for example). In grass, mouse over each point and record the coordinates.
Step 2: import the scan
In grass, do: r.in.gdal input=[path to scanned file] out=[Mapname] location=templocation
Quit grass
Step 3: target, point, rectify
Open grass, but this time in the 'templocation' you created in step 2
i.target group=groupMapname location=[modern map location name] mapset=PERMANENT
i.group group=groupMapname in=Mapname
d.mon start=x0
i.points groupMapname
d.mon will open a window; i.points will display the scan in it. Select the mapname in the dialog that appears, then one by one select each of the points you've identified as having concurrence with the modern map. In the terminal window, enter the coordinates for the point taken from the list you created in step 1. When done marking points, click 'quit'.
i.rectify -a group=groupMapname extension=_1 order=1
Depending on the size of your map and your processor speed, this bit may take a while. When done, quit grass.
Step 4: admire output
Open grass in the modern location. The scanned map will be available as a raster layer for display. The scan will have been rectified so the map matches the projection of the modern map layers - ie you'll be able to see what's moved and changed, and what exists now that didn't then etc. There's other grass commands which will help you convert features of interest (rivers, roads, contour lines, whatever) into vectors if you really want.
If all this seems too hard, have a look at qgis - also open source mapping software; it's more gui-oriented and I know it has a georectifying plugin. I've just never used it.
Good luck.
Read up on how David Rumsey's doing it ... (Score:2, Informative)
Saw a great presentation at OSCON a few years back about the massive digitzation effort undertaken by David Rumsey (http://en.wikipedia.org/wiki/David_Rumsey): See http://conferences.oreillynet.com/cs/where2005/view/e_spkr/1867 [oreillynet.com] -- Drew a well-deserved standing ovation.
In the course of the talk, I think he said that he'd scanned the first 10,000 maps (though even 1,000 sounds ridiculous -- maybe it was 1,000) before hiring assistance.
Of course, he had more money to play with, so he probably had a pretty big scanner ;)
timothy
Re:Handheld scanner (Score:5, Informative)
I do this for a living, We Use a wide format scanner and Global Mapper to georectify them.
Contact mikes@wavefront.pro if you would like a quote.
We do everything from old torn maps to vellums to Tifs, We can Georectify them to load quickly as a geotiff. or we can digitize the data on the maps into Arc compatible Shapefiles.
Use panoramic stitching / Gigapixel technology. (Score:1, Informative)
Hi, I'm a panoramic photographer, (I have shot panoramas up to 18 gigapixels in size, which currently seems to be a world record) so I hopefully am speaking from experience. What you want to do is basically shoot a panoramic image of your map.
You should situate your map as flat as possible, and shoot photos of it from a perpendicular point of view - best to hang it on a wall I guess. In the case of a very fragile map, sure, you can lay it on the floor. Take care to light it evenly.
You can then use PTGui (my favorite panoramic stitching software by far - only a satisfied customer talking) to stitch the images together into one perfectly seamless image.
Some key things to remember are to keep the map as flat as possible. A long lens is probably better, but you can use a macro lens also.
PTGui has a very handy feature called "viewpoint correction" which can join different images *of a flat surface* together, even if you didn't shoot these images from the same point (this is useful for panoramic photography when we want to capture the area below the tripod). In the case of shooting your maps, this will be very handy indeed.
Feel free to contact me via my website's contact form (360 cities dot net) and I can help advise further if you want.
good luck,
Jeffrey Martin
Re:Handheld scanner (Score:5, Informative)
We can also output Google Earth KML's. It's neat to be able to click a link and get all your contours and well locations to pull up in 3D. And to have this file work on any machine with google earth.
Re:Contact a Museum (Score:5, Informative)
Quick post since I am heading out (Score:5, Informative)
Contact a local Licensed Land Surveyor. We are in the business of coordinating maps and making sure they are properly referenced. We also know the difference between NAD83 and NGVD29. This and the other coordinate system conversions and the proper use of scale factor in SPCS (State Plane Coordinate Systems) is something we do every day. Plus, most of us are really into local history and could possibly show you some other really neat uses for that data. Historic societies are always looking for ways to map past events. When speaking with a Surveyor, we can usually know what the practice for a given time period was. There are three different lengths for a foot that I have come across working. International Foot (not used in surveying, but sometimes engineers use the wrong foot), US Survey Foot (standard) and the Philly Foot. Philadelphia has a different set of standards for how a long a foot is, depending on what part of the city you are in and what you are trying to do. This is not something most historians would accurately pick up. Surveyors will. We also know who was the good and not so studious Surveyors in the area and what tricks each used to mark corners, turning points and reference markers. A local Surveyor in the area the map is of would be very interested in helping you with your work. He/she may have already done the heavy lifting for you. We have to trace maps back as far as possible, so sometimes (I am in New Jersey) we have to go all the way back to the Proprietors to get maps so that we can run lines that control our current work.
Long story short, if it deals with cartography or local surveying, seek a professional Surveyor.
Re:Contact a Museum (Score:2, Informative)
If the maps are in decent shape, you typically use a large-format scanner. These are extremely expensive, though, so you'll preferably want to find a local university or friend at a company with one. Most larger copy shops will have one (for making architectural plans/construction documents) but will likely charge you a pretty penny to use it. And as others have pointed out, uni or a local historical society may have been through this so be relatively set up to guide you along (or even do some of it for you!).
If they're really brittle or on non-standard material, digital photography will likely be your only option. And if you want a nice orthographically correct version it will take a lot of patience as you'll get a fair bit of distortion on those large maps. So, as described by other folks in this thread, you'll need a setup so that you can take a number of tiles and stitch them together. To truly take a line from the 'pros' (as in the way they actually shoot aerial photography) you'd want to very carefully mark out a grid pattern on the map itself so you have something to correct against. One other thing: find the smallest real aperture you can get - if you've ever seen pictures from pinhole cameras you'll notice that everything is in focus. (And if you're debating using a point and shoot vs a nice DSLR, make sure to convert to equivalent focal length when comparing - in most cases you'll find that as long as the optics are decent on the P&S, the effective aperture will be better unless you have a really fancy lens/camera setup.)
That all said, if they are old, and you're more concerned with georeferencing them than having a high-quality reproduction, you likely needn't spend too much time getting a photo of the final version. As there will almost indefinitely be some distortion from the true coordinates, you'll likely need to do some 'rubbersheeting' to get the maps to match up with their real-world locations. That process will likely introduce way more distortion than that from from your digital camera. If you have access to mapping software such as ArcGIS, it will do it easily for you. Otherwise there are lots of free products out there that will allow you to distort the image appropriately.
Good luck!
I've done this (Score:5, Informative)
I have done this for a grant-funded historical map digitization project at a university library. We used a $40k large-format scanner (from Betterlight) which can scan the whole item laid out flat. Trying to stitch together camera images will result in distortion across the image—if you didn't need to distort it, you wouldn't need special software to do it; you could just line the pictures up.
But even once you have image files, there's about zero chance you can just replace Google Maps' tiles with your own and expect geotagged stuff to line up where it should. If you have a finite number of places of interest, you could manually locate them on each map and then try to distort each map to align, but if you expect arbitrary geolocations to need to be right, give up. Non-satellite/GPS-based maps are examples of practical cartography, not theoretical. They will be even less perfect than you think, no matter how professional they appear. Or do what we did: keep the geotag display on Google's maps, but show your historical map of the same general region side-by-side and allow the user to calculate the precise correlation in his own brain.
Re:Blue print company (Score:4, Informative)
For best results generally you would use a SLR with Macro lens. This type of lens generally provides the flattest field at reasonable cost even when it is not used in macro mode.
Then you've got architectural lenses, but those cost an arm and a leg and a foot - but then again, renting is always an option...
Geomorphic stability (Score:3, Informative)
Just a pedantic little thing -- as a geomorphology instructor, I can tell you that rivers and coastlines are very, very likely to have changed. Check out pretty much any river mouth in Victoria, Australia, or any island off Maine, US in google earth vs google maps satellite mode for examples of how much they can change inside of just a few years. If something catastrophic has happened (big storm, big earthquake...), huge changes can shift the coastline inside of hours.
If you're going to use geomorphic features for your geocoding, find out what's most stable in your region (keyword search academic journals for geomorphology + your location + change and see what doesn't pop up, or ask a local university geo prof). Vegetated topography can be pretty stable over decades, especially if you only need relative shapes.
Re:Talk to a curator (Score:3, Informative)
I'll assume you are a European and suggest the nearest large university cartographic library. They are knowledgeable, helpful and it's the sort of thing they do. They are also all in touch with the other universities, so you will have lots of resources to draw upon.
If you are in an ex-colonial country (England, Holland, Belgium, France or Germany), then the military mapping agency may be able to offer you some help. Your national geographic service may well offer you some good advice.
There are a couple of other considerations. Are you willing to share the digital data with those who help you? Are you concerned with keeping your maps intact (I hope so). The projection of the maps is going to affect geo location code as well. The older the map, the less likely its projection will be accurate at all. Do you have any idea of the scale of the maps? At any rate, you need professional help and the most likely folks are at the nearest large university.
Re:Blue print company (Score:3, Informative)
If I had this project, I would start with digital SLR with telephoto macro lens.
Lay the map on the floor and use a mat frame that has some weight to hold the section to be photographed down.
Weight should be such that it holds the map down but doesn't press so hard that it damages the map. The idea of the frame is that it delimitates that area that you are photographing so that you have a reference to the next spot to photograph (should overlap a little). In addition, you have a reference for squareness of the picture should the camera not be perfect parallel to the map - in which case you can use a program such as Photoshop to correct the perspective (at least the versions that I have used it is in the "crop" tool - there is an option for perspective)
Re:Handheld scanner (Score:3, Informative)
GlobalMapper > Arcinfo.
It all comes down to price, Globalmapper is like $300. I don't even know what Arc costs, but i know blue marble is cheaper and it's a couple grand.
GM might not have the same intuitivness, but It's perfect for the given problem.
Considering I mainly deal with Oil and gas we usually use well locations as a secondary x/y point.
just be glad you don't have to convert from meets and bounds
Re:Handheld scanner (Score:2, Informative)
I've also found that a lot of university libraries will have staff and equipment in their rare books room that is appropriate to this task. Having said that, what they told me to do is what everyone else has said: snap and stitch, and what they felt they could offer was a more secure rig to hold the camera and more even lighting for the exposure.
Re:Talk to a curator (Score:2, Informative)
Actually, Autostitch probably won't work for this application.
From the FAQ:
Q: Does AutoStitch support planar stitching, such as flatbed scans or aerial photographs?
A:The demo version of AutoStitch assumes that the camera is rotating about a point, so distortions will be visible when stitching multiple views of a planar surface. We hope to add planar stitching functionality in the future.
Re:Blue print company (Score:2, Informative)
No, a tilt-shift would be the lens to go for. Architectural photography favours this type of lens because you can adjust it to match the plane of the subject, and it would be pretty easy to do this for the very easily identified image plane of the map under glass.
Distortion-free in one easy step.
More of a worry, really is getting all sections of each image evenly lit without reflections. Not as easy as you'd first assume.
Re:Talk to a curator (Score:3, Informative)
you could also try hugin and play with the various projections that are rectangular. hugin uses autopano-c and seems to work pretty good. ideally get yourself a copy of photoshop cs4 and just use file->automate->photomerge and try automatic and then "reposition only" if it is for some reason trying to project on a spherical surface. the lack of seams and stitching errors in photoshop's tool really amazes me compared to other pano programs I've used. I really liked hugin and did a lot of cool stuff with it, but as soon as you throw in water or anything that moves it just really seems to break down. photoshop always gives me perfect water with smooth looking waves.....i'm sure I could probably get more smoother results from hugin, but why bother when I can just click twice and be done?
David Rumsey Collection in SecondLife (Score:2, Informative)
http://slurl.com/secondlife/Rumsey%20Maps%201/133/247/56/?title=David%20Rumsey%20Maps&msg=David%20Rumsey%20map%20collection [slurl.com]
See also http://www.davidrumsey.com/ [davidrumsey.com]
Re:Talk to a curator (Score:3, Informative)
Raster reprojection is done all the time in GIS, so this would be nothing new. Google Earth has plenty of georferenced old maps available as overlays, and I've made a few myself. Yes, some distortion would be introduced by the camera (as opposed to scanning), but since the projections may be arbitrary or off anyway, all this gets corrected at the same time.
It is true that you have to have some common points to current georeferenced maps in order to do this, of course. But there should be enough, and if not, then at least you are not introducing new errors.
You can do this either in the closed source ArcGIS georeferencing tools, or using open source GRASS (http://grass.osgeo.org/wiki/Georeferencing)