Terrain Creation from DEM Data

[Hawk]

I came across this site in another forum I frequent and was wondering if it would be of any help to map makers.

http://terrain.party/

Here's a brief help on how to use the site.

1 - place the blue square over your target area
2 - use the + and - buttons at top-right to select area size (8-18 km2)
3 - use green button to select data source
4 - use down button to download data.

As far as I know all data sets include a grayscale height map (of varying quality)

[Hawk]

Bumpty bump.

[Gumboots]

Yes, there will be bumpity bumps in terrain. Unless it's a salt lake or something, in which case the bumpity bumps will be very small and won't show up on your game map.

Ok, seriously, the limit of 18 km x 18 km is pretty small. It's only about 11 miles square. This one might be handy if anyone was doing a very large scale map of a very small area, like a specifically urban scenario, but it's probably easier to go from DEM's* for most maps.

Also, I don't know how others have fared with grey scale height maps but I've found them less well-behaved than the coloured ones. Specifically, I've found that it's very difficult to get the shoreline to sit well with the ocean, without having the rest of the terrain go nuts. It might not matter if you didn't have any ocean on your map.

*Meaning the usual, larger scale, coloured DEM height maps that the editor will convert to a game map by itself. Trying to construct a game map from actual data points sounds pretty tedious.

[Hawk]

Actually it will go up to 60 KM square, but I guess that's still a little small.

terrain.jpg (10.59 KiB) Viewed 15726 times

I didn't realize the limitation, so - never mind.

[Wolverine@MSU]

Also, I don't know how others have fared with grey scale height maps but I've found them less well-behaved than the coloured ones. Specifically, I've found that it's very difficult to get the shoreline to sit well with the ocean, without having the rest of the terrain go nuts. It might not matter if you didn't have any ocean on your map.

That's because a grey-scale heightmap has only 256 levels (1 byte) whereas a colored heightmap can have 10,000 (~21^3) levels if using Microdem to produce them.

[Gumboots]

The thing there is that you can't export the DEM with the default colour scale if you want the RT3 editor to digest it. It has to be exported as "blue scale", with all the heights done as combinations of blue and black. I may be wrong, but I would have thought this would be limited to the same range as greyscale, since it's effectively just substituting blue for white AFAICT. Does this format in fact have an extended color range compared to greyscale?

The main difference appears to be the use of pure red for the ocean (zero altitude) baseline. That seems to be what lets the editor figure out the ocean level without borking the coastlines. With greyscale, the lower altitudes are so close to black that it all gets a bit mushed up, particularly once you scale the image down to a usable 1024x1024. I think the scaling is what causes the real problem. The pixels around the coasts sort of lose the plot a bit.

[Wolverine@MSU]

Microdem doesn't have an RT3-compatible coloring scheme for elevation data. The creator of the program kindly made a database file (.dbf) that can be read by Microdem (as described in the tutorial). I'll try an attach an Excel spreadsheet that shows the coding of RGB channels for different elevations from sea level to 10,000 meters, and also the .dbf file (enclosed in the ZIP) in case anyone wants to look at it.

[Gumboots]

Aha. So it's not just blue and black. It uses the green channel as well as the blue.

[Wolverine@MSU]

Yes it does but it only gets to a value of 39 out of 255 so it's very difficult to discern any contribution from the green channel.

[Gumboots]

I figured out why I was getting weird results with TGA heightmaps exported from MicroDEM.

If I use the old PopTop MapBuilder to generate a TGA heightmap it will import into the RT3 editor without any problems. The heights and terrain will all look normal even if all modifiers are left on the default value of 1. It may not be the best solution, but it will be close enough that a bit of tweaking with modifiers will get it really good. You won't need to use crazy values.

The catch with the MapBuilder is that it's not as accurate as MicroDEM. The latitude/longitude scale is off by a degree or so, so you don't get the area it's telling you you'll get. The data is also coarser than the more modern DEM's, so you get blockier coastlines, etc.

If I (previously) exported a TGA heightmap from MicroDEM (using the blue/black 0-9999 table) it didn't behave the same way when imported into the RT3 editor. It was dramatically different. If all modifiers were left on 1, the terrain generated by the editor would have massive cliffs around all the coastlines, and inland areas would have mountains that make the Himalayas look like molehills.

To get a playable map from a MicroDEM heightmap used to require using an overall height modifier around 0.005 or so (yes, 5/1000) and then playing around with the other modifiers to tweak coastal heights and mountaintop heights. And naturally it ended up losing detail due to the extremely small value of the initial height modifier.

So this is fixed now.

All I did was uninstall the old version of MicroDEM that I had, including deleting the remaining old C:\microdem folder and its contents, then do a complete clean install of the latest 64 bit version.

That obviously needed the 0-9999 DBF (see Wolverine's post above) dropped into the C:\microdem root folder to give the correct heightmap scale for RT3.

Then all it needed was adjustment of the settings inside MicroDEM. From the top menu:

Modify > Elevation > Check "Colors from table" > Select "Elevation table" > Select ELEV_COLORS_0-9999.dbf

> Select Missing (upper right of the pop-up) and set the color to pure red

> Click OK.

You will now have your DEM in the correct colour format.

Crop out the section you want, and save that as a .bmp.
Open in Photoshop or GIMP or whatever.
Resize to a suitable RT3-compatible 64+1 sizing, including scaling laterally to correct for latitude.
You can use an online latitude/longitude calculator to get the correct scaling. This one is a good one.
Use the standard bicubic algorithm for resizing. Don't mess around with sharpening or smoothing.

Once you have the proportions sorted, simply save/export as a 24 bit TGA. This will then import into RT3 without problems.

[Nickel Plate]

All I did was uninstall the old version of MicroDEM that I had, including deleting the remaining old C:\microdem folder and its contents, then do a complete clean install of the latest 64 bit version.

That obviously needed the 0-9999 DBF (see Wolverine's post above) dropped into the C:\microdem root folder to give the correct heightmap scale for RT3.

Then all it needed was adjustment of the settings inside MicroDEM. From the top menu:

Modify > Elevation > Check "Colors from table" > Select "Elevation table" > Select ELEV_COLORS_0-9999.dbf

> Select Missing (upper right of the pop-up) and set the color to pure red

> Click OK.

You will now have your DEM in the correct colour format.

Hi, I have a question about this. I'm a long-time player, and map download browser, and was playing around trying to create some maps tonight, for either RT2 or RT3. Long story short, I followed the guide Wolverine@MSU wrote at http://web.archive.org/web/200503021557 ... orial1.htm (its original site went down in 2021, but it's still on the Internet Archive), and it more or less worked despite some buttons having moved around over the years. I actually wound up getting a successful RT2 map by following the guide and improvising a little bit. But the elevation colors is the stumbling block. I found the download in this thread, but I can't seem to find the "Colors from table" option.

I can open the DBF as a "Database without map", and I can open my combined GeoTIFF file of the area I want to create a map of. If I right-click on it and choose "Display Parameter -> Elevation/grid values", then my "Modify" menu subsequently has an "Elevation" option, which takes me to the same place as "Display Parameter -> Elevation/grid values". My options look like this:



In this case I've chosen "Fixed color palette", and pasted in the path to the DBF file (other options in that field are "dNBR", "NDVI-1", etc.). However, it doesn't use the colors from the DBF file.

My best guess is something in the MicroDEM UI changed over the past few years, but I could be looking in the wrong spot. Can anyone confirm where this option is supposed to be in the current version of MicroDEM?

(I've also found viewtopic.php?f=65&t=3708, and plan to try OilCan's method; the BoundingBox tool it mentions is exactly what I'd been hoping to find. But since I'm 82% sure that I'm just one small step from completing the MicroDEM method, I'd like to cross that finish line as well. For those curious, I used data from https://srtm.csi.cgiar.org/srtmdata/)

[Gumboots]

No idea. Your screenshot looks like you are running XP or something. I'm using the 64 Bit version of MicroDEM, running on Windows 10. Which version of MicroDEM did you download?

Also, be aware that the old ELEV_COLORS_0-9999.dbf is rubbish, in that it only has a height resolution of 40 metres, when the DEM"s themselves are 5 or 10 metres (depending on which set you choose). This means you can get much more accurate terrain by using different tables. See the following posts for more information:

viewtopic.php?p=46287#p46287
viewtopic.php?p=46294#p46294
viewtopic.php?p=46297#p46297 (and the one after it)
viewtopic.php?p=46308#p46308 (includes a set that will handle areas below sea level)

[Nickel Plate]

No idea. Your screenshot looks like you are running XP or something. I'm using the 64 Bit version of MicroDEM, running on Windows 10. Which version of MicroDEM did you download?

Also, be aware that the old ELEV_COLORS_0-9999.dbf is rubbish, in that it only has a height resolution of 40 metres, when the DEM"s themselves are 5 or 10 metres (depending on which set you choose). This means you can get much more accurate terrain by using different tables. See the following posts for more information:

viewtopic.php?p=46287#p46287
viewtopic.php?p=46294#p46294
viewtopic.php?p=46297#p46297 (and the one after it)
viewtopic.php?p=46308#p46308 (includes a set that will handle areas below sea level)

Oh, the theme is just a Stardock Curtains theme that looks like XP, but it's Windows 10. My MicroDEM version is "Build 2020.8.20.1", according to its start-up splash. It's the latest one from their official USNA website; I didn't see any links to older versions.

Thanks for the links, I'm investigating them now. The below sea level one could be useful on one map; I'd noticed that the Netherlands was below sea level in my RT2 testing. It'll take some time for me to follow up and see what's working. My current ideas are a less flat West Virginia map for RT2, a West Virginia map for RT3, and maybe a larger France map for RT3. Of course once enough detail is added to make something good and playable, I'll be happy if I wind up with one good map with all the other hobbies contending for my attention.

[Gumboots]

Ok, I have build 2012.12.20.1, which I suppose tells you how long it's been since I looked at their site. I can give you a copy of that build if necessary, but I suppose I should try the latest build and see what I can make of that. I can always fall back to the older build if I have to.

Making the maps themselves, in terms of the actual terrain, is a piece of cake. It's all the events, and testing for balance, that takes time.

[Gumboots]

Just got around to looking at this in more detail. The relevant help link is this: https://www.usna.edu/Users/oceano/pguth ... alette.htm

Short version is that, yes, they do appear to have changed the format required. It's different to the default c:\microdem\color_palettes.dbf in the older version I have, and the new help link also seems to say that individual custom tables are no longer accepted, and all palettes must now be defined in c:\microdem\hard_limit_color_palettes.dbf

Which is a bit of a nuisance, but not an insurmountable one. All it requires is tweaking the formatting of the old custom .dbf files, and bunging the results into the new c:\microdem\hard_limit_color_palettes.dbf, after which the right elevation tables for RT3 mapping should be selectable.

I'm sure I can do this, but I'm frankly not excited about doing it (and testing the result) right now. Checking their update history, it looks like the new format was introduced in February 2020:

Feb 2020
New database, hard_limit_color_palettes.dbf for defined palette display of grids

So I'm guessing any versions before then would accept the old custom tables. Offhand I'm not sure how you would get hold of a pre-2020 version that is more up to date than the version I have. I've had a look around, and I can't find any download options for them. However, as mentioned, I can offer you a copy of 2012.12.20.1 if that will help.

---

Just had a look around the docs for QGIS, which is another useful app. It says you can create custom palettes, and it says you can import and export them as a file, but it doesn't give any method of creating such a file from scratch (and I've looked elsewhere too).

What I will try sometime is creating a rough and basic custom table, then export that to a file (of whatever format it uses) and see if I can open that file in some sort of editor and (hopefully) make sense of the content. If what is workable it should be possible to recreate the RT3 height data tables and import them into QGIS. It may (dunno yet) even allow having all of them (from 0 to 10,000 metres altitude) all in one palette, which would be more convenient than MicroDEM's limit of 255 heights per table.

Although it is also possible (would need to test) that the latest MicroDEM will also accept more then 255 heights per table, which would make an update worthwhile.