Map TGAs

[RulerofRails]

I know you were discussing dds file compression in the other thread, but perhaps there are tips and tricks for compressing map tga's. So work on those in a larger format and then lose as little detail as possible when reducing to 1024x1024. That seems to be the only way that I can see the Japan Quakes special features (ships, planes, whales, submarine) showing up in the game.

About the Poptop devs, questioning their intelligence personally is a bit unfair (not saying that you were, I also have been whining about stuff they did lately, just had a little reality-check). When some of them attempted to get funding for a new rail game last year, they seemed to be making more comparisons with RTII. The guy who built the engine commented here.

Additionally, if you would've worked with us, I think you would have a much better understanding of what went down back then at PopTop and why RRT3 turned out to be what it turned to be... and believe me Milo, I, amongst others and above most of all, wanted it to be so much more... specially after I poored 1 year of my life, all my soul and brains into building the engine for it.

[Gumboots]

TGA is a non-lossy format. DDS is different. If you start with a TGA or PSD or PNG or whatever and convert it to DDS, without changing the size, that conversion will throw away 3/4 of the pixel information in the image. This is how DDS gets its good compression.

[Just Crazy Jim]

No matter which way I turn this one issue, it still seems ridiculous to have an embedded high quality TGA in the GMP file, then to use DXT or JPEG/JFIF or whatever to generally wreck it in the saves. In fairness, I don't think the fellow who coded the game engine was responsible for every aspect of the game that we find fault with. Many of these things appear grafted on in the manner of an afterthought.

I still haven't found the precise part of the EXE responsible, but my gut still says it is in the routine that creates the map preview for the scenario selection screen, which does make use of a 256x256 JPEG/JFIF format. I am certain that after resizing and converting it to JPEG/JFIF for the preview/thumbnail with black and red track lines painted as necessary, the same routine then resizes the now 256x256 grotty base image to 1024x1024 and embeds it in the saved GMP in place of the superior original. And I am increasingly confident that it is JPEG/JFIF format compression and not DXT, the pixel and color drift just aren't enough like DXT to account for what we see in-game. For one, after a series of successive saves, we should see banding if the compression was DXT, which we don't. Instead, things get increasingly blocky, which is an old telltale sign of low percentage JPEG/JFIF compression.

The number of occurrences of a JPEG/JFIF headers in the GMP files is a bit ludicrous. Milo remarked on the number of redundancies and apparent errors in the GMP files. I haven't extracted the JPGs out as I have with the TGAs because finding the JPEG/JFIF header and footer is a great deal easier than selecting all the data in between. But, based on the differing byte counts between them, my gut tells me that when I do get around to it, to expect to see mipmap scaling in JPEG/JFIF format.

[Gumboots]

I copied out the hexes after the last (non-random) instance of FF D9 from the GMPs and saved them to a new hex file. I saved the resulting files with the file extension BMP and TGA. Adobe Photoshop could not open the file with the BMP file extension, but opened the TGA version without any problem. Further testing with a couple of other image editors and Cerious Software's Thumbsplus confirm that the map texture is indeed a TGA missing the optional Truevision-Xfile footer and other optional extension data. There may be a thumbnail buried in the GMP (also in TGA format) as well.

It was in those first 18 bytes after the FF D9 that started me thinking. I've noted this 18-byte cluster in a number of other games I've modded. Specifically, the 3rd byte always being 02 that suggested to me that the format was TGA. In a TGA header, the 3rd byte indicates compression type. 02 in that position indicates an uncompressed true-colour image. Once my suspicion was set off, I checked to see if the rest of the header file info conformed to TGA header specifications. It does.

Every GMP file I have scoped has this series of values after the final (non-random) FF D9:

Code: Select all

00 00 02 00   00 00 00 00   00 00 00 00   00 04 00 04
20 xx 

02 at position 0002 (3) indicates uncompressed true-colour image
00 04 at positions 000c-000d (13-14) indicates 1024 pixels width
00 04 at positions 000e-000f (15-16) indicates 1024 pixels height
20 at position 0010 (17) indicates 32-bit colour
xx at position 0011 (18) indicates alpha channel depth and direction and this varies in the files. In the files I have scoped, the value at position 18 is usually 08 (8-bit alpha channel), but in some it is 20 (32-bit alpha channel).

Ergo, the GMP map texture is TGA.

I just had a thought about this, while I was playing with extracting stuff straight from a test .gmp.

Smaller maps generally fare quite well with the usual 1024x1024 map texture, because the texture pixels are far smaller than the map pixels, so it looks pretty decent. The larger the map gets, the blockier the texture pixels get, so things like detailing around coasts tend to turn to crap.

Since we now know the visible in-game map texture is a TGA, rather than the BMP that the RT3 editor requires as a starting point, and since we know where it lives in the .gmp, there may be scope for improving the resolution of game map textures.

It will depend on whether the devs hard-coded any other restrictions we don't currently know about, but if they didn't then it may be as simple as making a TGA texture of whatever resolution you fancy (ie: 2048x2048) and then shoving the hex for that into the relevant part of the raw .gmp.

If that works, suddenly you'll get far better textures on larger maps. I think this is worth testing.

(PS: I already know it can't be done with loco or building textures, because I've tested that and the game will simply refuse to load anything bigger than 1024x1024, but there may be a loophole with map textures.)

[Gumboots]

w00t!

Ok, I just tested that idea of throwing a 2048x2048 TGA into a .gmp. It works. No problem at all. It appears the game is coded so it just looks for the TGA header, then assumes everything after that is texture stuff it needs to apply to the heightmap.

Although I ran the test with a 2048x2048 I suspect the game wouldn't care what size the thing was, although games are usually coded so they like powers of 2 best for image sizes. But a 2048x1024 would probably work too, or a 2048x512 for a very long and narrow map (can test this later).

Attached images show the base texture (scaled down to half size and saved as jpg for this example) and a screenshot of the result in the game editor. The weird hibiscus grotty carpet effect was me just taking the basic green original texture, scaling it up to 2048, then applying a translucent pattern overlay in Photoshop. The idea being that by counting the repetitions of the pattern across the in-game view I would be able to tell if it was chopping the texture at 1024 or not.

As it turns out, it doesn't chop it. It's using the full texture size. This is on a 320x320 test map (the Chunnel_test_92.gmp I made a few weeks back). You can tell by looking closely at the pattern that it is far less pixellated than a texture usually would be on a map this size.

So it works as far as the basics go. Likely effect on in-play performance? Probably not much. The 2048x2048 TGA is roughly 16 megabtyes. That's an extra 12 meg compared the standard 1024x1024. These days, with reasonably new gfx cards, an extra 12 meg of VRAM for the map texture is probably not a big deal. The real loads will be elsewhere.

In case anyone else wants to mess with this idea, the procedure goes like this:

1/ Make your basic map, of course, of whatever size you want and with whatever terrain, etc.
2/ Save it in basic form.
3/ Open the .gmp in a hex editor, and search for the start of the TGA header.
4/ Delete the TGA header, and everything after it.
5/ Open your nifty new much bigger TGA texture in the hex editor.
6/ Scroll to the end of the file, and select the last 26 bytes (the TGA footer that RT3 doesn't want).
7/ Invert selection.
8/ Copy.
9/ Paste onto the end of your cropped .gmp.
10/ Save .gmp.
11/ Hey presto, you now have teh hi-re map texture.

---

PS: Oh yeah, and I also tried doing a quick bit of terrain painting on the new texture. It works, and what's even better is that the painting follows the pixels of the new texture. In other words, suddenly your in-editor painting brushes are half the size, and therefore twice as good at detailing, as they used to be.