using google maps and microdem TOGETHER

[soccersetter643]

gumboots, what do you mean by projection?

[Gumboots]

http://en.wikipedia.org/wiki/List_of_map_projections

[soccersetter643]

Oh I see what you're saying. the other option is doing a slight lens distortion filter in photoshop to correct it. It would just mean searching for more coordinates on the google map so that the distortion is accurate. Further testing should be done to confirm this.

By large area, you mean something as big as north america, right?

[Gumboots]

I was finding noticeable divergence over an area of about 500 miles square.

[Gumboots]

Just did a bit of checking. The DEM's are done on a basic cylindrical projection, such that the 5 degree x 5 degree tiles are rectangular. This will be least distorted at the equator, with progressively increasing distrortion as you move further north or south.

So, you could do an "around the world in 80 railways" map at the equator without worrying about projection discrepancies, but if going the other way (from pole to pole) you'd have to make a lot of corrections before you could get satellite imagery to overlay onto the DEM.

Although Google Earth uses a basic cylindrical proejction for its maps, this doesnt apply to the satellite image view you see on the rotatable globe.

[Gumboots]

I'm going to try this out. I've just tiled up a 1024x1024 image out of Google Earth satellite images as a part of a test run. It's a bit tedious, but not difficult. I haven't cropped/merged the dems or generated a .gmp yet, but the texture picture is ready for overlay. The procedure went like this:

The area I'm covering is about 750 miles east to west, and 930 miles north to south. Specifically, it runs from -26 S and 141 E, to -39.5 S and 154 E. Yes, it's Australia again.

Anyway, I found that when taking the screenshots from Google Earth, a two degree by two degree square worked best. Any smaller was too many tiles. Any larger started to give problems with the curvature of the meridians. If I stuck to two degrees by two degrees, the shapes were close enough to a basic tapered quadrilateral. This meant they could be stretched to rectangle and tiled, without mismatching problems at the edges (bearing in mind that I was turning the Google Earth sat images into a composite cylindrical projection, to match the dems and Google maps).

The finished map will be rectangular (I'll be using 768 wide by 960 high), but the BMP that has to be made first (for the bmp2gmp tool) has to be 1024 x 1024. So, break out the calculator and do some number crunching to stretch the 768x960 to 1024 square, then calculate the pixel positioning of the degree grid for latitude and longitude. Set up said grid. You now have a blank psd with longitude and latitude grid on it. Tiles can be made to fit this.

Photoshops wont do weird distortions/warping on multiple layers at the same time. It will only do a basic tapering and corner moving on a quadrilateral (it's called the Distort tool, but doesn't really "distort", if you know what I mean). To get the corners right, I had to have a Google Earth shot with the latitude/longitude grid on it, and another screenshot just underneath without the grid. These were pasted into a second psd, that was purely for making tiles. Once each tile was all sorted out and cropped, I resized it down to fit the grid on the main 1024x1024, and pasted it into that.

With me so far? Cool.

So, first set up a new 1024x1024 file in Photoshop, with grid positioned according to calculations. Set up a second, blank, psd that is for making tiles. I made this quite large, so I could get the tile edges more accurate without going cross-eyed (actual size was 750 wide by 830 high).

Find the bit you want in Google Earth, line it up nicely, turn grid off (Ctrl+L), take shot, paste shot to the second blank psd. Turn grid back on (Ctrl+L), take another shot, paste this on straight on top of the first one. Select both together, move them until they are centred in the PSD. Crop the image to grid boundaries (Ctrl+A > crop image > Ctrl+D). Now adjust corners with the "Distort" tool until the screenshot grid intersections are at the corners of the psd. Crop again (you need to do this double crop or the bits hanging out will cause problems). Turn off the top layer.

Voila. You now have one 2 degree by 2 degree tile. Now resize that down to fit the tile size of the main, composite psd. In my case this was 158 wide by 152 high. Paste the result into the big psd. That's the first one done. Step backward on the second psd (Ctrl+Alt+Z) to bring it back to its original size. It's now ready for more screenshots to be pasted and distorted to suit.

Now just repeat the process until all the tiles are done. It may take a couple or few hours, depending, but it's easy enough once you get into the swing of it.

Yes, I could have saved a few keystrokes by tiling everything without resizing every tile down to fit the 1024x1024 image first, then just resizing the finished composite image once. I chose not to do it that way, because the composite would have been huge and a bit awkward to work with (10000x11000 images are not a lot of fun to work with, IMO).

So, that's how you make a composite satellite image, at 1024x1024, with a cylindrical projection, from Google Earth screenshots. The one I did had 49 tiles, and once finished looks immaculate.

[Wolverine@MSU]

Very nice work Gumboots. I'll have to give it a try the next map I make. I've always been looking for a way to overlay Google Earth imagery on a RRT map.

[OilCan]

Wow! Very nice. I'm very interested in seeing the final outcome.

[Gumboots]

I might make another one and see how streamlined I can get the process. I wasn't really thinking of efficiency this time around, since I was mostly doing this one in the ad breaks while watching tv. I just did it one or two tiles at a time and then saved. I think the best way really would be to just paste straight to one, very large, image that had the lat/long grid. This would be slightly disconcerting to work with, partly due to the size and partly due to the minor mismatches between the tiles probably being visible at that scale. However, once merged into one layer and scaled down to 1024x1024, it should look perfect. It should also be faster to do.

The way I'm thinking of it, it'd be paste the screenshots, then turn off top layer once stretched to fit the grid, select lower layer only with the Select tool, invert selection, then Cut. Top layer (now invisible anyway) could then be deleted.

The other point is that when I scaled down the tiles from 750x830 to 158x152, I got a partial tranparency coming in on the last pixels along each edge. This was purely an artifact of the scaling algorithm, and left visible, sort of greyed out, lines between tiles. I got around that by cheating. What I did was duplicate all the tiles, then merge the duplicates into one layer, then set that behind all the other tiles, then offset it one pixel left and one pixel up.

That filled in the partial transparency along the edges of the tiles, and is not noticeable to the eye. It looks just as good as if it hadn't been cheated, but it was (again) more keystrokes etc, so not the most efficient method overall. If using a more efficent method, I probably could have done the whole thing in an hour.

---------------------------------------------------------------------------------------------

The other thing worth mentioning is the choice of vertical and horizontal scales. A degree of latitude is the same distance over the ground at any latitude (or near enough to it, if you're not that fussed about oblate spheroid stuff) but a degree of longitude varies from roughly 70 miles over the ground at the equator, to zero at the poles.

The cylindrical projection used by DEM's and Google Maps assumes that a degree of longitude is the same distance at all latitudes. It starts from the equator and goes from there, so the further you get from the equator, the more your map will be stretched east to west. What I decided to do was effectively change the radius of the cylinder that the map is projected onto, so that it matched a point roughly halfway down the map.

What this does is to nullify the stretching at that latitude, so that a mile east to west will be the same number of pixels as a mile north to south. At the top of the map, east to west distances will be slightly scrunched (around 7% in this case) and at the bottom of the map, east to west distances are slightly stretched (again, around 7% in this case). Overall, however, doing it like this gives the least distortion of distances while still using a cylindrical projection.

It had another advantage, in that because I was using 33 degrees South as a reference, the finished map only had to be 84% of the width that it would have needed if using the equator as a reference. This means a more compact and manageable map on screen, while still keeping the same miles per pixel ratio. If using a basic (equatorially based) cylindrical projection straight from DEM/Google, I would have had to take the map out to 896 pixles wide to cover the same area, and the east-west distances would have been more distorted. Having tried an 896x1024 test map on my 20" monitor, I'm convinced the 768x960 will be superior during actual gameplay.

[Gumboots]

Ok, have tried applying a .bmp and, being on W7 64 bit, I get the problem talked about here: Re: Bmp2gmp dummies guide?. IOW, it applies but is all stripey and dark, and generally useless.

However, the good news, taking what I can get in the way of good news at this stage, is that even though the textures/colours are borked, the overlay does appear to exactly match the .gmp heightmap. So, as soons as I can get that texture application to render properly, it should be fine.

I tried downloading Wolverine's PDF tutorial from this post, but my copy of Reader wont open it. Says something about "raise without handler" and quits. Maybe the old attached PDF is corrupted.

===================================================

ETA: Ok, I know what I did wrong. I had saved as 32 bit .bmp without alpha channel visible. If I save as 24 bit .bmp with alpha channel visible, it imports to the .gmp perfectly

BUT

there has to be a glitch of course.....

...although the textures are applied almost perfectly (will need minor touching up around some coastal bits) the remaining problem is that they are only shown at a distance. When you get close in to the terrain, the applied .bmp textures fade out and are replaced with a basic, sort of pale olive green/cracked dirt sort of texture. Screenshots follow.

The other problem is that although the map looks awesome "from space", once you get down to "Cessna altitudes" the textures degrade. Obviously this is not a bug as such, merely a limit on the amount of information the original 1024x1024 .bmp can contain. As with any raster image, once you zoom in enough it will begin to pixellate badly. This is a separate problem to the one mentioned just above.

My current conclusion is this: although the application of a sat image .bmp to the .gmp is easy enough, it will probably have to be considered only a useful guide to manual terrain painting, at least on fairly large maps. It may work better on very small areas.

Would be delighted to be proven wrong about the larger maps, of course.

===================================================

AHA! I think I know what is going on. I think the two problems are actually related. Think about it this way: the overlaid .bmp is raster, therefore pixellates badly when viewed from "low to moderate altitude". Considering that, the problem will obviously get worse at ground level. I have a suspicion that what is happening when zoomed right in at ground level is that you are effectively "looking between the pixels". IOW, the .bmp has run out of information to feed you at that scale, so you just get the default RRT3 general purpose ground texture.

I opened soccersetter's Durango map and the same thing happens. When you zoom right in, you see the generic/fallback RRT3 ground texture, rather than the fancy stuff applied via .bmp.

[Hawk]

I tried downloading Wolverine's PDF tutorial from this post, but my copy of Reader wont open it. Says something about "raise without handler" and quits. Maybe the old attached PDF is corrupted.

Nope! Nothing wrong with the pdf file in the post you linked to. I just tried it and it opens fine.
What pdf reader are you using? I'm using Sumatra, but Foxit opens it as well.

[Wolverine@MSU]

I think there's a way to "paint out" the ground texture in the Editor.

[Gumboots]

What pdf reader are you using? I'm using Sumatra, but Foxit opens it as well.

I'm using Adobe Reader. Anyway it doesn't matter. I got it the .bmp to work without reading the PDF. Turns out you don't even need an alpha channel on it. Just plain RBG channels is all it needs. The key point apears to be using a 24bit .bmp instead of 32 bit. 32 bit was a mess. 24 is fine (on W7/64 bit OS).

I think there's a way to "paint out" the ground texture in the Editor.

Sure, you can do that, but then you're not really using the .bmp, because you're replacing all the textures with ones you have painted yourself. I did do a bit of messing around with painting manually, and that overrides the .bmp. Anything you paint manually will, assuming it's a large enough feature, be visible even from high altitude.

Like I said in the other post, I think this will have to be a case of using the.bmp as a guide to manual painting of textures. It's still a big help, because you can easily grab the ground colour for an area just by Ctrl+clicking on it from high altitude (where the .bmp textures look solid) then using that to paint manually when zoomed in. This should still give very good results, even if it's not quite as convenient as we might like it to be.

The thing about the map I'm playing with is that large areas of it look more or less like the surface of Mars. The Strezlecki Desert is a bit like that. So although RRT3's default ground colour would not be that noticeable when zoomed in if the ground colour you wanted was some some of green anyway, it'll really stand out badly if the place is supposed to look like Mars.

[Wolverine@MSU]

I think there's a way to "paint out" the ground texture in the Editor.

Sure, you can do that, but then you're not really using the .bmp, because you're replacing all the textures with ones you have painted yourself. I did do a bit of messing around with painting manually, and that overrides the .bmp. Anything you paint manually will, assuming it's a large enough feature, be visible even from high altitude.

I guess I mis-used the term "paint". I wasn't referring to actually painting to the bitmap, but using the Paint tool to remove texture (see image below). You can see that the ground texture has been removed on the left hand side compared to the right hand side. I used the Pencil tool in this example, but you can use the paintbucket set at "fill below" and find the highest point on the map to remove texture from the entire map. There's still the grainy appearance that remains, but you only see that when zoomed in really close.

Untitled-1.jpg (117.55 KiB) Viewed 6068 times

[Gumboots]

Ok, tried that. Perfect! Thanks for the tip. If you start right down on the coastal fringe and use the "fill above" option, one click fixes the ground texture for the entire map, and the .bmp colours go right down to ground level. I left the ocean alone for now, but as you said you could do it the other way, and start at the high point then fill down.

Obviously it could still do with some manual touching up here and there for local detail, but overall it's great. Looks like we have a viable system.

[Gumboots]

One other detail. Google Earth puts a heavy drop shadow around coastlines by default. There doesn't appear to be a user option to remove this. It seems to be a hard-coded feature that you have to accept.

What this does is (not surprisingly) put a heavy dark border on your coast when you make your 1024 x 1024 .bmp. This shows up once transferred to your .gmp. It's not that bad, and you'll probably be painting beaches etc anyway, but it's something else to deal with in the editor. Because of this, I've decided it's better to remove it when I'm making the coastal tiles. More work in PS, less work in RRT3 editor.

Another point worth noting is that RRT3's ocean edges will match the .bmp very accurately, but RRT3 puts the breaking waves on coasts several blocks back from the actual ocean edge. This means that to get a truly realistic coastline, you'd have to artificially extend the ocean several blocks further inland, so that RRT3's waves appeared to break where they really should.

The problem with that is that it will mess with coastal elevations, since oceans always have a height of 0. That may be a problem in some situations. It may be possible to cheat this, and use a lake tool to artifically extend the ocean edges to where you want them, without changing elevation. I haven't tried that yet (just thought of it) but will give it a go sometime and see what happens.

The only other alternative would be to paint an artificially wide beach, or rock shelf, or whatever, so it goes out and meets the waves. That's possible, but is more work, and rather defeats the point of getting an accurate coast from satellite shots.

[Gumboots]

Another point worth noting is that RRT3's ocean edges will match the .bmp very accurately, but RRT3 puts the breaking waves on coasts several blocks back from the actual ocean edge. This means that to get a truly realistic coastline, you'd have to artificially extend the ocean several blocks further inland, so that RRT3's waves appeared to break where they really should.

The problem with that is that it will mess with coastal elevations, since oceans always have a height of 0. That may be a problem in some situations. It may be possible to cheat this, and use a lake tool to artifically extend the ocean edges to where you want them, without changing elevation. I haven't tried that yet (just thought of it) but will give it a go sometime and see what happens.

Tried this. It works.

You can join a lake straight onto an ocean and, by using the land area smoothing tool, actually convince the "lake" part of it to slope up your beach. That way the waves break where they should break, and you don't have to paint unrealistically wide beaches.

The trick is to bring the ocean in as close as possible, then put a narrow strip of "lake", then run the smoothing tool along the coast. Only catch here is that it is going to be difficult to get it to work properly when you have cliffs on the coast*, but it seems fine for beaches.

*I'm not sure yet how much slope the "lake" can be convinced to take.

[Gumboots]

Ok, now I know how much slope a lake can be convinced to take. I'll never trust a water level again.

Shots are attached for the ocean part of the gfx, the lake part, and the resulting cliffs with waves breaking at their base. Trick was to start with the basic terrain from the heightmap, then go along the cliff using Shift+click to reset lake height as you go before setting each square to lake.

This will result in layer changes that are more angular than you want, so break out the smoothing tool. After that, it's just a matter of playing around at bit. I used the "plateau to sea level" tool a bit, as well as the "increase height" tool, to get the cliffs looking right at the end. By tweaking which cells are set to land, and which to lake, you can adjust exactly where the waves appear to break. I went for the classic "cliff with protruding rock shelf below high tide line" look.

I think the end result is pretty good for a first stab at it. Only took about ten minutes, and I could do it faster next time. Admittedly this is only a very short section of coast, but fortunately Australia has a lot of beaches anyway.

ocean.jpg (80.88 KiB) Viewed 5975 times

lake.jpg (74.65 KiB) Viewed 5975 times

result_cliffs.jpg (62.68 KiB) Viewed 5975 times

[soccersetter643]

Wow guys, this is amazing! Gumboots, GREAT JOB on the in-depth research on this project. Concerning the texture degrading when you zoom in on my Durango & Silverton map, I had always just assumed that that bug was just a side affect of me using the mac version of RT3, and it always bugged the hell out of me! So glad to hear that there is a fix to this problem! I'll get to work on updating that scenario ASAP.

On a side note, sorry I've been gone, but I've been in the process of moving from California to New York, which has taken a good deal of time and energy. But I'm getting settled, and I'll try to follow through on this post more!

GREAT WORK EVERYONE!

[Gumboots]

Oh here's another detail: trees. I've just been playing OilCan's new Central America beta map and it has no trees on it.

As I found out myself when playing around with Google Earth sat images for RT3 maps, none of the default Poptop trees sit will with the colours of such images.* They always look out of place. I assume this is why the new map was made without trees.

I'd found a similar model with maps of Australia, in that even with default PopTop land textures the available trees never looked right. They're obviously made from an American perspective and don't necessarily fit other places at all.

The problem with leaving the trees out of a map is that doesn't look right either. It leaves the map feeling rather dead and flat. I had thought that at some point there might be some value in making some extra tree packs. These should be quite simple to create, since the default trees have very simple models and would be easy to re-skin. Custom tree packs could be made for extra locations to fit custom maps, or to fit Google Earth colours, and just dropped into UserExtraContent as required.

Since these would be simple re-skins they wouldn't break anything, so even if you had the wrong pack in place it wouldn't really matter. It'd just look a bit odd. Might be worth thinking about, if anyone wants to have niftier maps.

Another thing I thought of was ground textures. The Google Earth shots give a very good overall effect particularly at higher altitudes, but are lacking in detail when viewing from near ground level. The thing is you don't want the default RT3 ground texture applied full strength either, since that overpowers the sat image at close range and looks stupid too. This got me thinking that the best option might be to apply suitable RT3 ground textures at low opacity, so that they enhance the sat image at low altitude/close range rather than fighting it. I think this technique might make it possible to have a map that looks convincing at all viewing distances.

-------------------------------------------------------------------------

*While I think of it, I think one problem with GE sat shots is that they're dependent not only on the general colour of the area in question, but also on its albedo. This tends to make forested areas look darker than you'd expect, with the very dark ground texture not sitting well with RT3 trees. There may be some value in playing around with brightness and contrast in some areas of GE sat shots to get things looking better.