Patched the blurry texture issue (properly)

[TruePikachu]

Firstly, many people know about the blurry texture issue. And those who know the currently-recommended fix, the hacked d3d8.dll file which returns 512MB for GetAvailibleTextureMemory, might not know that if you launch the game through Steam (I have the disc version somewhere around here, I just launch through Steam), the "fix" causes the game to crash whenever it changes resolutions.

I have been doing work on trying to find a fix, and nothing has ended correctly so far:
* The d3d8.dll fix doesn't seem like it will be a solution, since Steam hooks into DirectX to provide e.g. the Overlay (and does so even with the Overlay disabled). I had checked the vtable for a known IDirect3DDevice8, and it pointed into the SteamOverlay.dll; chances are, Steam ends up trying to make use of the hacked DLL as well.
* Running my disassembler and debugger, I tried to find all calls to get the availible VRAM through the patched function. It was only referenced once, to write a value to an unaccessed (in the scope of my testing) memory location, probably for the debug info stuff. Meanwhile, the textures continued to go into the low-res versions.
* Replacing the textures with their _A.dds versions. This had the interesting effect of making everything appear normal at first, then this happened. I have since reverted to the original .pk4 files, but have my "conversion program" still availible.

I'm running 1.05 on Win7x64, GPU is an AMD Radeon HD 7520G (3026MB) running at 1366x768x32. I _don't_ think I have any unofficial fixes installed, excluding the nocd.

Anyone have any more musing to share with regard to the issue? I can't say anything about the Steam version of the game, as I said, I'm on the disc version, just launching with Steam.

EDIT: Changed topic, hopefully, to clarify that Steam is only being used to launch the game; it was not purchased through Steam.
EDIT: Another change, and here is a link to viewtopic.php?f=5&t=3734&p=37988#p37988
EDIT: Another topic change

[Hawk]

Have you tried the multiple install option, discussed in this thread:

viewtopic.php?f=82&t=2363

By copying the RT3 part of the Steam install, that might get it away from Steam and let you play it without all the Steam garbage.
Buying RT3 from Steam is not recommended. Then of course, I wouldn't buy anything from Steam. Been there! Done that! Didn't get a T-Shirt (didn't want one).
If you were to use the above option, you might need this.

Here's some links to threads about RT3 from Steam.

viewtopic.php?f=82&t=3419

viewtopic.php?f=82&t=3569

viewtopic.php?f=82&t=3710

There mat be other threads in the RT III Installation forum that might be of some help.

[AdmiralHalsey]

Buying RT3 from Steam is not recommended. Then of course, I wouldn't buy anything from Steam. Been there! Done that! Didn't get a T-Shirt (didn't want one).

Yeah much as the Steam fans don't want to admit it it's just a big heaping of DRM. I've never bought anything off Steam and never will. If I buy a game from the store and it turns out to have forced Steam integration I just take it back.

[TruePikachu]

Did you even read my post?

I can't say anything about the Steam version of the game, as I said, I'm on the disc version, just launching with Steam.

I'm on the disc version, not the Steam version. I just launch it with Steam, so I have the Overlay.

EDIT:

So that it is more evident to others, the only call I found into GetAvailibleTextureMem() was at 0x51F990*:

Code: Select all

// public: void WorldDriver::CheckTextureMemory()
void __thiscall WorldDriver__CheckTextureMemory(WorldDriver *this)
{
  IDirect3DDevice8 *d3d; // eax@1

  d3d = this->ID3D8;
  if ( d3d )
    this->AvailibleTextureMemory = d3d->_vt->GetAvailibleTextureMem(d3d);
}

where WorldDriver is the class with constructor at 0x51FF90*, and WorldDriver.ID3D8 is a pointer to an IDirect3DDevice8. "_vt" in d3d just refers to the vtable, since it is a virtual call, not an imported one. With a hardware breakpoint set at the actual address called (the start of GetAvailibleTextureMem), I can pretty much confirm that this is the only place where it seems to be called. That said, I have no idea how or why the fake DLL works, given that I had tried patching this function here to always result in 512MB, and it didn't seem to work. I might not be seeing something, or the fake DLL works for some reason none of us expect. I'll see what happens if I patch the DLL to return 3GB/-1GB instead of 512MB.


*These are the memory offsets after the program has been loaded into memory, not an absolute offset into the executable. I am using the nocd version with MD5 BFF386B889BEE8E5F5941733E5E95248, but I'd expect offsets to remain the same for the real version of 1.05, given the considerable effort needed to actually insert/remove bytes

Quick edit: Just patched it, binary offset 0x984 gets changed from 20000000h to C0000000h, textures degrade. However, they don't with it set to 60000000h (1.5GB). Testing was done with Explorer launching, since Steam would just crash the game when the resolution changes.

[Hawk]

Did you even read my post?

I did read this:

(I have the disc version somewhere around here, I just launch through Steam)

Which lead me to think you bought and installed the game from Steam as you couldn't find the cd.

I either missed this line:

I can't say anything about the Steam version of the game, as I said, I'm on the disc version, just launching with Steam.

or you added that line when you edited your original post to change the title.

I'm on the disc version, not the Steam version. I just launch it with Steam, so I have the Overlay.

Why in the world would you want to do that? What overlay are you talking about?
What are you trying to accomplish by running the cd installed version through the Steam environment?

[RulerofRails]

I am not a programmer so can't be of much service in that regard. I guess you have read what the Steam forums have about this issue? Some people have reported the same problem you are having without a fix being suggested (crash at the change of resolution to display the campaign screen). If you only have problems launching the game through Steam, I would suspect something in Steam itself. I am guessing though. The funny thing is that some people never need a texture fix, I believe that graphics cards are the real issue.

Seems you were able to convince the game to run fine with 1.5GB of VRAM? That's great. Can I do the change you did with a Hex Editor (how do I read the coordinates you gave?) or do I need something else? I hope it will allow me to have a bigger empire going before I get hang-ups.

[TruePikachu]

What are you trying to accomplish by running the cd installed version through the Steam environment?

If you launch a game through Steam, the majority of the time (OpenTTD and Flight Simulator X are the only things which have given me trouble in the past, for unrelated reasons), you have access to the Steam Overlay, which allows you to e.g. chat with friends. It appears that, in order to provide that functionality, it hooks DirectX, in a way which is apparently incompatable with the fix the community uses right now. For the edit on the first post, I only added the line prefaced with "EDIT:", and changed the title.
Since you don't use Steam, let me explain. While Steam (acting as a distribution service) has a heap of DRM (granted, nothing like e.g. Spore had), it also has community features, which also hooks into some games (e.g. for searching for other players). As a convience to players, you have the ability to launch non-Steam games through Steam, which hooks into the game in order to provide its Overlay and, by extension, Community features. (see 2nd ¶ of here)

I am not a programmer so can't be of much service in that regard. I guess you have read what the Steam forums have about this issue? Some people have reported the same problem you are having without a fix being suggested (crash at the change of resolution to display the campaign screen). If you only have problems launching the game through Steam, I would suspect something in Steam itself. I am guessing though.

The only way I have managed to get the high-resolution textures to work is by using the D3D8.dll fix. It acts as a proxy to the system D3D8.dll file, passing the calls to the system, I believe with the only exception of GetAvailibleTextureMem. The vanilla community fix (without hex editing) makes that function always return 512MB (40000000h); I edited that file to see what happens with 3GB (C0000000h, my actual amount of VRAM) and 1.5GB (60000000h, half my actual VRAM, which is under the 7FFFFFFFh limit).
However, the IDirect3DDevice8 class (which GetAvailibleTextureMem is a function of) has said function accessed through its vtable. This allows a so-called "child" of e.g. the IDirect3DDevice8 class to do something different with its GetAvailibleTextureMem function; it is almost exactly how the unofficial fix works. However, Steam apparently hooks the vtable itself, so while normally, RRT3 would ask Direct3D how much texture memory is availible, it is instead asking Steam. From there, things I don't know happen, which ends up causing the game to crash whenever the resolution changes. I suspect it is related to Steam trying to call the function itself, which it would be getting from the community fix, and having a fit or something.

Seems you were able to convince the game to run fine with 1.5GB of VRAM? That's great. Can I do the change you did with a Hex Editor (how do I read the coordinates you gave?) or do I need something else? I hope it will allow me to have a bigger empire going before I get hang-ups.

Unfortunately, you seem to misunderstand. While you can patch the community fix DLL with a hex editor (I gave the offset and values), it was just changing how much VRAM the game thinks exists, at least with regard to the GetAvailibleTextureMem call. If having it always return 3GB had the same result as the original 512MB (having the game use the high-res textures), it would reveal that what many believe is the problem (GetAvailibleTextureMem returning a value ≥2GB being interpreted as a negative number) isn't actually the problem, or at least isn't relevant.



In summary, I'm trying to fix the problem with a modification to RRT3 itself, rather than using an external DLL file. The community fix does work, but I can't manage to fix the issue or integrate that workaround within the executable itself; it isn't a matter of space (a "MOV EAX,xx \ NOP" pair is the same size as the call), but rather that the DLL looks like it might be doing a bit more than everyone thinks, which manages to fix the problem. Steam was only mentioned because it conflicts with the community fix, it isn't causing any problems (if anything, it is the community fix which is causing a problem with Steam), and Steam isn't acting as DRM, but rather a sort of plugin.

[Hawk]

What are you trying to accomplish by running the cd installed version through the Steam environment?

If you launch a game through Steam, the majority of the time (OpenTTD and Flight Simulator X are the only things which have given me trouble in the past, for unrelated reasons), you have access to the Steam Overlay, which allows you to e.g. chat with friends. It appears that, in order to provide that functionality, it hooks DirectX, in a way which is apparently incompatable with the fix the community uses right now. For the edit on the first post, I only added the line prefaced with "EDIT:", and changed the title.
Since you don't use Steam, let me explain. While Steam (acting as a distribution service) has a heap of DRM (granted, nothing like e.g. Spore had), it also has community features, which also hooks into some games (e.g. for searching for other players). As a convience to players, you have the ability to launch non-Steam games through Steam, which hooks into the game in order to provide its Overlay and, by extension, Community features. (see 2nd ¶ of here)

OK! That explains it. Thanks!
Even when I had Steam installed, that community stuff and chat stuff never did interest me, but to each his own.

The funny thing is that some people never need a texture fix, I believe that graphics cards are the real issue.

Wasn't it believed in a thread somewhere here that the graphics issue started at anything above 256 MB VRAM?

Here's the thread.

viewtopic.php?f=82&t=3376

[RulerofRails]

Wow, that is a lot of information. Seems you know what you are doing. After reading Hawk's link (good link, thanks!), I probably wont bother trying to hack the d3d8.dll as I doubt it will help performance. I hope someone else can help you.

[OilCan]

It acts as a proxy to the system D3D8.dll file, passing the calls to the system, I believe with the only exception of GetAvailibleTextureMem. The vanilla community fix (without hex editing) makes that function always return 512MB (40000000h); I edited that file to see what happens with 3GB (C0000000h, my actual amount of VRAM) and 1.5GB (60000000h, half my actual VRAM, which is under the 7FFFFFFFh limit).
However, the IDirect3DDevice8 class (which GetAvailibleTextureMem is a function of) has said function accessed through its vtable. This allows a so-called "child" of e.g. the IDirect3DDevice8 class to do something different with its GetAvailibleTextureMem function;

I'm not even going to act like I know what this means. Yet, I am very impressed. And another reason I don't want to give up my old laptop. I think.

[TruePikachu]

The funny thing is that some people never need a texture fix, I believe that graphics cards are the real issue.

Wasn't it believed in a thread somewhere here that the graphics issue started at anything above 256 MB VRAM?

Here's the thread.

viewtopic.php?f=82&t=3376

I had come across that thread, but deemed it irrelevant at the time. I'll read over it all again, see if I get any inspiration for things to try.

In the meantime, I think I'll see what happens with the community fix DLL, maybe Direct3D itself is calling the fixed function in some way? I have no idea, especially since I've breakpointed the real D3D GetAvailibleTextureMem, but...hmmm...maybe I'll do a full decompile of the community fix, see if there isn't anything else which is fixing it, other than the difference of GetAvailibleTextureMem. Maybe I could also try to figure out why replacing the textures with the _A versions caused them to fail to load, might be related to the "unknown value" field in the PK4 (the first DWORD for each file descriptor). Might also help me find out exactly what's going on behind the scenes with texture loading as well.

An interesting oddity I had noticed while following one of my trains is that when an object is created (I'm speaking in the sense of the game itself, so the caboose at the end of a train that just finished loading), it has a high-res texture, which then gets switched to the low-res a short time later. Following the same train through numerous different stops showed that the pattern held; every time the caboose was added, it was high-res for but a long moment (1-3 seconds). I don't have the experience to try to find the function which does that through breakpoints and traces alone, but I might come across it while digging into the texture loading code. I already have a place to start looking (load address 0x5DD5E4, absolute address 0x1DD5E4 has a trio of strings, "%s_%c.tga" "%s_%c.jpg" "%s_%c.dds", which are very obviously for loading textures, if you know printf formats), so it might just be a matter of exploring.

[TruePikachu]

\o/ Mission successful! I got high-res working!

I had apparently missed a point where the call is made, which was well before I attached any debugger. I patch that function as well, and I have high-res textures when running within Steam. Screenshot

Anyway, the binary patch is a total of 12 bytes; at file offsets 0x11F974 and 0x11F99A, change "8B 08 50 FF 51 10" to:
* "B8 00 00 00 60 90" for 1.5GB
* "B8 00 00 00 40 90" for 1GB
* "B8 00 00 00 20 90" for 512MB (community fix amount)
* "B8 00 00 00 10 90" for 256MB
This works for at least my copy of the executable, I provided the old values. If you don't actually have those values in the two locations I gave (and _both_ need to be changed), it should be possible (if potentially dangerous) to search for the two sets of instructions.

I have not preformed comprehensive testing, so that is the next step. But things look good so far :D

[Hawk]

Looks like you're getting this issue worked out. That's good news.

[TruePikachu]

Further success, I decided to try to do a _real_ patch for the bug. Using the space between functions in the game, I was able to write up a new function which is called instead of GetAvailibleTextureMem in the two locations I previously patched. My function makes that same call to GetAvailibleTextureMem, but if that returns a value ≥2GB (so a negative value from the perspective of the game's code), my function will instead return 2GB-1, or the maximum positive signed value possible. While I haven't done a comprehensive test on this new patch, it should squish the bug once and for all if it is related to the GetAvailibleTextureMem function in the two locations I found it, and continue to work properly if you in fact don't have a huge amount of VRAM.

The code I wrote, for those who know x86 assembly, is:

Code: Select all

.text:0051F951 ; =============== S U B R O U T I N E =======================================
.text:0051F951
.text:0051F951
.text:0051F951 ; int __usercall BetterVRAMCheck<eax>(IDirect3DDevice8 *<eax>)
.text:0051F951 BetterVRAMCheck proc near               ; CODE XREF: WorldDriver::CheckTextureMemory2()+14↓p
.text:0051F951                                         ; WorldDriver::CheckTextureMemory()+A↓p
.text:0051F951
.text:0051F951 ; FUNCTION CHUNK AT .text:0051F982 SIZE 00000006 BYTES
.text:0051F951
.text:0051F951                 mov     ecx, [eax]
.text:0051F953                 push    eax             ; IDirect3DDevice8 *
.text:0051F954                 call    [ecx+vIDirect3DDevice8.GetAvailibleTextureMem]
.text:0051F957                 test    eax, eax
.text:0051F959                 jl      short bVRc_cont
.text:0051F95B                 retn
.text:0051F95B BetterVRAMCheck endp
.text:0051F95B
.text:0051F95B ; ---------------------------------------------------------------------------
......
.text:0051F982 ; ---------------------------------------------------------------------------
.text:0051F982 ; START OF FUNCTION CHUNK FOR BetterVRAMCheck
.text:0051F982
.text:0051F982 bVRc_cont:                              ; CODE XREF: BetterVRAMCheck+8↑j
.text:0051F982                 mov     eax, 7FFFFFFFh
.text:0051F987                 retn
.text:0051F987 ; END OF FUNCTION CHUNK FOR BetterVRAMCheck
.text:0051F987 ; ---------------------------------------------------------------------------

(that split is needed, because that's where one of the CheckTextureMemory() functions are)

The difference file is a bit longer than the old one:

Code: Select all

0x11F951: 90 90 90 90 90 90 90 90 90 90 90 -> 8B 08 50 FF 51 10 85 C0 7C 27 C3
0x11F974: 8B 08 50 FF 51 10 -> E8 D8 FF FF FF 90
0x11F982: 90 90 90 90 90 90 -> B8 FF FF FF 7F C3
0x11F99A: 8B 08 50 FF 51 10 -> E8 B2 FF FF FF 90

Patch is for the 1.05 nocd I have, will probably still work on a normal 1.05. Old values listed are from the old 1.05 (i.e. not with the other patch I wrote applied). In any case, here's one I prepared earlier. Please note that I haven't run this version through as much of a test as I ran the old patch I wrote. Also note that, for both of these patches, you don't need the custom d3d8.dll file. That is the main reason why I wrote these patches, to remove the necessity of that file. However, credit is still due to mpxd, for coming up with the original fix (the d3d8.dll file).

I've only tested this patch to the extent of loading up sandbox and waiting to see if the textures degrade, but I can almost guarentee that my old patch is 100% stable. I would think that this one also is, but you can't be too sure with code, I've learned in the past.

[RulerofRails]

That's great! Glad you have the ability to fix this. I just loaded up a large game which normally lags badly with the d3d8.dll "fix" and ran it for a bit with your modified exe. Worked perfectly and while I still get some lag I noticed a definite improvement!

I see you really know what you are doing. I am impressed! I don't know if you would be interested, but the format for the train and building models in RT3 is 3dp. This is a special format (which is actually mostly a 3ds file) made specifically for this game. It contains several different resolution images and needs some sort of algorithm for compiling. Using the program Itsa Timmy was using in the above link, it is possible to load the models from RT3 in other programs, but importing is impossible as the RT3 3dp format contains some other information that that otherwise excellent program isn't designed to handle.

That's about as much as I know. Needless to say, the limited skills of the various contributors on this site have restricted any brand new, "from scratch" engines and buildings. The changes that have been made have been done in hex editor. Gumboots really took this to a new level and with heaps of effort managed to make a few engines that were extensively modified to be just how he wanted them. Would you consider looking into this? If models could be edited and imported by the community the door would be open to much easier modding. We would be most grateful. I am mainly a player, but have read through the many struggles of others with editing these files manually. If you want more links to where this has been discussed/caused frustration for many members, just ask.

[Gumboots]

This is interesting stuff. I've been using mxpd's fix without any problems, but a better solution is always a good thing.

[TruePikachu]

That's great! Glad you have the ability to fix this. I just loaded up a large game which normally lags badly with the d3d8.dll "fix" and ran it for a bit with your modified exe. Worked perfectly and while I still get some lag I noticed a definite improvement!

I actually didn't think of preformance benefits either of these two patches could possibly have, compared to the d3d8.dll fix. That file acts as a proxy to the real d3d8.dll, so if a program wants to make use of a function from Direct3D, the call would go to there first, then that would call the actual function from Direct3D. I would think that it either could be or is more optimized than that, however, knowing how vtables are implemented, but the original, original version of that (not for RT3) was a proof-of-concept, and only a couple of changes were made to it, so who knows what the overall benefits are. My former patch, in theory, would have the strongest preformance benefit, but doesn't actually check how much memory one has; the later (and current) one checks the memory, but puts a reporting cap of just under 2GB to prevent the bug. And neither depend on another DLL to actually make the call.

I see you really know what you are doing. I am impressed! I don't know if you would be interested, but the format for the train and building models in RT3 is 3dp. This is a special format (which is actually mostly a 3ds file) made specifically for this game. It contains several different resolution images and needs some sort of algorithm for compiling. Using the program Itsa Timmy was using in the above link, it is possible to load the models from RT3 in other programs, but importing is impossible as the RT3 3dp format contains some other information that that otherwise excellent program isn't designed to handle.

That's about as much as I know. Needless to say, the limited skills of the various contributors on this site have restricted any brand new, "from scratch" engines and buildings. The changes that have been made have been done in hex editor. Gumboots really took this to a new level and with heaps of effort managed to make a few engines that were extensively modified to be just how he wanted them. Would you consider looking into this? If models could be edited and imported by the community the door would be open to much easier modding. We would be most grateful. I am mainly a player, but have read through the many struggles of others with editing these files manually. If you want more links to where this has been discussed/caused frustration for many members, just ask.

I actually think I came across some of the relevant code for that, when I was trying to figure out why just copying textured didn't work. I'll need to look for it again, but I know what I'm looking for. It might have just been a constructor following memory allocation, but that it the best place to start the search; it tells me exactly how big the in-memory structure the file is loaded to is, and frequently gives a hint of a location in the binary proper where the functions relating to it (like loading) are. Anyway, I'll take a look and see what I can figure out.

[RulerofRails]

Thanks so much for looking into this! This is exciting! Here are two links that may prove helpful:
By a programmer who worked on the game.
A post by Itsa Tommy with links on the characteristics of 3dp and 3ds.

[TruePikachu]

Quick update to this thread, no further work has been done on the patch. I noticed that, with the Coast to Coast map "East of Mississippi", when I have my newer patch in place, sometimes the entire map doesn't render. I do not know if this is a design feature, design flaw, something from my patch, or just my GFX card being weird. I can, however, state that I get ~60FPS on it compared to >100 on normal-sized maps, so it does tax the system a bit more. No texture degration though :)

[Hawk]

Large maps not rendering fully is common. You have to scroll over to the unrendered area to get it to show.