3dp File Converter

[Gumboots]

Ok, this is fascinating.

What I did was start with the loco body file for the 242 A1 (ie: the Chapelon A1 4-8-4, not the LNER/Gresley A1 4-6-2). The reasons I chose this file are that it's a default RT3 file that everyone will have, and that it is one of the cleanest of all the loco body files. IOW, it breaks down cleanly into individual components, and generally doesn't have odd bits strewn throughout the file in weird places. This makes it one of the easiest to work with, which is good.

So, got rid of the latest Blender and installed Ye Olde 2.49a. Opened 242_a1l_Body.3dp in 3dConverter and exported it as an .off. Imported that .off into Blender 2.49a, then exported that as .3ds. As already mentioned the file headers are different, which shunts bytes out of place, so I then made a test copy of the .3ds and cropped the file header to give the same starting alignment (including line number) as the original RT3 .3dp file. This header cropping means that the test ".3ds" is no longer a valid .3ds file, but this doesn't matter since I just wanted it for comparing the hex for the vertices.

Ok, so I already had the original .3dp bookmarked for all components. By comparing the "test not-a-.3ds" with the original .3dp I have been able to bookmark the .3ds for all components too. It's interesting stuff. Basically what happens is that if you have something like a box, where there are adjacent sides, then the .3ds format will duplicate the vertices that are shared between sides. This means that the hex for that component will be longer in .3ds than in RT3 .3dp, due to the duplication (or even triplication) of some vertices. However, the actual co-ordinates for the vertices do not change. They are still all located at the original points in space, which makes sense since you still have the same model, just in a different file format.

The other thing is that once you get to the end of the declarations for vertices, the way the polys are declared in .3ds bears absolutely no resemblance to the way they are declared in an RT3 .3dp file. In other words, although the hex from the .3ds for the vertices themselves may be of some use, as far as I can tell at the moment the gfx mapping in the .3ds won't be of any use.

Another point with regard to gfx mapping is that with some vertices being duplicated in the .3ds, this will throw out the numbering for them if pasted straight to a .3dp, which in turn will screw up the gfx mapping in the .3dp unless you go through it and re-number all the declarations for the gfx mapping (which would be a very ugly proposition).

Screenshots below show the original RT3 .3dp file, with all components bookmarked and highlighted, then the same for the "test.3ds" file (that has its header cropped to align the x,y and z co-ordinates). As you can see, the locations of the vertices are the same, but some are duplicated in the .3ds format.

body_3dp_hex.png (48.67 KiB) Viewed 6166 times

body_3ds_hex.png (47.44 KiB) Viewed 6166 times

One more weird thing about all this is that if you import the .3ds back into 3dConverter, it will display almost (sort of) normally. The extra vertices aren't a problem inside 3dConverter, however all of the faces that are supposed to be facing the camera will be transparent. This is due to them (apparently) having their normals reversed by the .3ds format, so unless you enable drawing of back faces they won't render as solid. This means that if (somehow, dunno how) you did manage to use the .3ds gfx mapping in the RT3 .3dp, all the shading would be arse backwards.

[Gumboots]

w00t!

I was just looking at the declarations for the polys in the "test not-a-.3ds" and they make sense to me. It's a different format to the RT3 .3dp but it's quite logical once it clicks.

The .3ds doesn't have any gfx mapping. It's probably possible to add this somehow, since .3ds would often be used for rendering from an associated skin image, but as exported from Blender without additional shenanigans there is no associated image to call mapping on.

This means that the declarations for the polys are just the numbers for the relevant vertices. For instance, a triangle ("poly" to its friends) drawn between points 00, 01 and 02 is just recorded in the .3ds file as that. To separate declarations for polys the .3ds file uses a similar trick to that used in RT3 .lco files, namely a double byte of all zeroes (00 00). To separate vertex numbers it uses a single byte of zeroes (00) just like an .lco file uses for ending names or whatever.

In the screenshot below, the first purple block (20 41 08 17 00 00 E0 02) is a header that acts as a divider between the last declaration for vertices and the first declaration for polys. The red block (01 00 00 00 02 00) says that the first poly is drawn between points 01 (followed by divider 00) and point 00 (followed by divider 00) and point 02 (followed by divider 00). The purple block after that is the 00 00 divider between the declarations for the first and second polys. The next six bytes, highlighted in green, is the second poly. Etc, etc.

body_3ds_polys_hex.png (35.56 KiB) Viewed 6165 times

Ok, so what does this all mean for RT3 modelling? I can already see how it may be useful, at least to some extent. It won't turn the whole thing into a no-brainer, but I can see ways in which it may reduce the workload. Any reduction in workload is better than none.

Also, given that Blender 2.49a apparently handles .3ds files differently to Bender 2.73a, it is quite possible that the .3ds file format has been changed over the years. If that is the case (seems likely, since it often happen for other file formats) then as RoR mentioned an even earlier version of Blender, which would presumably be coded for even older versions of .3ds, may give us the results we really want. I don't have time to play with that tonight, but will investigate it over the next few days.

[Gumboots]

A bit more progress. I tried playing around with exporting from CAD to Blender, then from Blender as .3ds, then back into 3dConverter and checking the generated hex code in the hex editor. This was just to see what happened.

Exporting from CAD as Wavefront (.obj) or as DXF mesh or as STL or whatever is no good. It ends up with an insanely complex mesh that has about 170,000 polys for the basic CAD model I made for the Schools class loco. Basically, those export formats multiply the vertices and polys by a factor of 10 or more. This is useless for RT3 stuff. That means it's no good starting with CAD apps, as far as I can tell.

So, what I tried then was starting with the basic cube model (in the standard .blend format) that Blender always defaults to on start up, and exporting that straight from Blender as .3ds. This is kinda cool. Unlike when you start with an imported .off and export that as .3ds, going straight from .blend to .3ds export does not multiply any vertices.

Despite the basic cube model having all vertices shared between several sides, the exported .3ds only contains co-ordinates for the actual 8 vertices. If it had been done as RT3 .3dp > .off > Blender > .3ds it would have had 16 or 24 or whatever vertices declared (as noted in previous posts).

Ok, so this is good. The basic cube model is 2 units long on each side, centred on the origin (0,0,0), and sure enough the hex says co-ordinates for each vertex are right where they should be: at either +1 or -1 on the x, y and z axes. I've also deciphered (to some extent) the hex for the polys. There are twelve of these (6 faces of a cube, each split into two triangles) and sure enough there are 12 triangles declared after the vertices.

Basic_Blender_cube_3ds_hex.png (30.48 KiB) Viewed 6155 times

The other good thing is that when going from RT3 .3dp > .off > Blender > .3ds then to viewing in 3dConverter, it was clear that the normals had been reversed, thereby screwing up the shading of the model's faces. This doesn't happen if you go straight from Blender's default format to .3ds. The .3ds exported that way will have all face and vertex normals pointing in the correct directions, so the shading of your model should be fine.

This makes sense, since IIRC it was established that the way the RT3 .3dp files were made originally was to start with a .3ds from AutoDesk and then run some custom algorithm (mainly on headers) to turn them into the weirdo/unusable RT3 .3dp format. What is required is to more or less, probably manually, duplicate PopTop's algorithm to do the conversion from .3ds to RT3 .3dp.

This is promising. I am cautiously optimistic that it will be possible to build a complex model from scratch in Blender and export it straight to .3ds and then, with a bit of cut and paste here and there in the hex, turn it into a functioning RT3 .3dp file. I already know it is obviously possible for simple models like bogies and trucks. I can see that clearly in my mind's eye, although for such simple models it is arguably just as easy to code them from scratch. In principle it is also possible for more complex models. The exact process is going to be more complex, and the resulting degree of difficulty higher, but it is definitely worth pursuing as the results could be very good.

[Gumboots]

Had a bit more of a look around, in between doing other things. I'm thinking that, if contemplating new models, there is really no point starting with .3dp. Might as well just build them in Blender, then find the best export format to get usable hex for RT3. So, on that basis there's not much point playing with ancient versions of Blender. Just grab the latest (2.73a) and use that. This has the advantage of better instructions and tutorials being available, and they sure are needed with Blender.

The only reason for using an older version (like 2.49a) would be if you actually wanted to import an existing RT3 model after converting it to .off, with the intention of modifying it and then exporting the modified version as .3ds. That's doable too, but is going to require more work on the hex to get it usable, due to all the multiplication of vertices that happens with that process. Just building stuff in Blender and exporting as .3ds gives a much cleaner result. It shouldn't, but it does. I figure once someone gets the hang of the basics of Blender (no need to know all sorts of advanced stuff) the amount of screwing around involved in adapting existing RT3 models has to be weighed up against the amount of screwing around involved in making new ones, with the latter looking like a better option for several reasons. Apart from anything else there would be no need to rely on 3dConverter for viewing models, because we could just load them straight up in Blender without any drama.

Anyway, it turns out that it is possible to have multiple versions of Blender installed, just like RT3, so if anyone finds it necessary they could alternate between 2.49a and 2.73a on the same box (see: http://www.techonce.net/threads/490625- ... of-blender).

Ok, so having thought all of that I figured the next thing to do was to go through every export option offered by Blender 2.73a and compare the hex output for all of them, just to make sure .3ds really did make the most sense. Turns out it does. All of the other export formats are not going to be any use to us for generating hex that is usable in-game. It's basically .3ds or nothing.

The next step for me is to run through some of the basic Blender tutorials so I at least have half an idea of how this monster of a thing is supposed to work. Once I figure out how to UV map the basic cube model (IOW, skin it with an image) I'll be able to export that as .3ds and check out the hex for the gfx mapping. This should tell me exactly what is required for conversion to RT3 .3dp, at which point I'll start trying to make actual game files for some basics (bogies, trucks, whatever) and see how it goes.

[RulerofRails]

At least you know what you are looking at. Good work!

I didn't have more time for this yet, but to me the new Blender seemed slightly more user friendly if that's possible. I wonder (haven't done a good search) if there are some free train models around that could be imported to Blender and then edited/hacked for RT3? Or would the code simply be messy? Best of luck with your efforts!

[Gumboots]

Yes, it does seem to be more user-friendly. The tutorials for it are also more user-friendly. All of the Blender "Noob to Pro" wiki pages are written for the latest version, and they seem to be better than the docs for older versions.

There probably are existing train (and building) models we could import and hack. In the brief bit of looking around I have done I've seen some pretty amazing Blender loco models, although they're far too high on poly count for use in RT3. As for other models, even if the format they are provided in gives messy code they may still be able to be used as 3d guides for constructing new models with clean code. Or, we could just work from blueprints. I've already managed to find drawings and dimensions for quite a range of locos from a range of countries. Blender will import blueprints as background images, which can then be used as guides for vertex placement. So yeah, if we can get a good routine sorted for the .3ds>.3dp conversion, it should give a lot more scope for a lot more people to play with RT3 modelling.

[Gumboots]

I've been working my way through the Blender Noob to Pro tutorials and they are very well written. I'm actually starting to get a workflow going, which is something that just wouldn't happen without the tutorials. If anyone else is thinking of trying Blender, read the tutorials. Blender is notorious for not being "user-friendly" in that it makes no sense at all if you don't go through the tutorials. I know we all like to just jump in and start doing stuff, but it really doesn't work with Blender. However, once you start working through the tutorials things rapidly start making a lot of sense.

Since Blender should allow us to easily generate lower levels of detail for locos (and buildings) further away from the camera, I've been looking through some of the default files to see how they handle this. It turns out that the majority of them use 5 levels of detail. The Northern 4-8-4 uses 6 levels, but the U1 only uses 4 levels. Given that many of the default skin images use 7 levels of detail (from A skin down to G), it's quite likely that the loco bodies could also be done with 7 levels if anyone wanted to. This could be good for performance if wanting to use higher vertex counts at close range (which I do).

Most of the default locos seem to aim for around 50 vertices at the lowest level of detail. I've made a list of some examples, along with a hypothetical example of a higher poly loco. By using 6 levels of detail and similar ratios to those used by other locos, it should be possible to have a (relatively) high poly first level, while still being very low on the lowest level. I may also test to see if loco body files will detect 7 levels of detail like the skin images do. This would be easy to test. All that would be necessary to to do something really obvious on the 7th level, such as making the loco 10 times as tall as usual. Once that kicks in, if it does, it will stand out at any range.

Vertex counts for Northern 484 instances (6 of) are 699 > 542 > 423 > 231 > 125 > 46
Ratios are 1.0 > 0.8 > 0.8 > 0.5 > 0.5 > 0.4

Vertex counts for H10 2-8-2 instances (5 of) are 827 > 573 > 341 > 181 > 52
Ratios are 1.0 > 0.7 > 0.6 > 0.5 > 0.3

Vertex counts for 242 A1 instances (5 of) are 578 > 366 > 191 > 102 > 80
Ratios are 1.0 > 0.6 > 0.5 > 0.5 > 0.8

Vertex counts for U1 instances (4 of) are 863 > 376 > 201 > 116
Ratios are 1.0 > 0.4 > 0.5 > 0.6

Hypothetical higher poly loco instances (6 of) 1200 > 960 > 672 > 403 > 202 > 61
Ratios are 1.00 > 0.8 > 0.7 > 0.6 > 0.5 > 0.3

[Gumboots]

Got some more progress on the whole Blender/graphics mapping/3ds schemozzle. I figured out how to map an image to a basic cube, then exported that to .3ds. It turns out that due to the way .3ds handles UV mapping, this results in the infamous duplicated vertices that were present when doing the old .3dp > .off > Blender > 3ds procedure. This gives a good indication of why the RT3 custom .3dp file format was originally coded, namely that it gives a leaner and more compact geometry than .3ds, which presumably makes RT3 .3dp better for performance.

Anyway, I have figured out the majority of the resulting .3ds hex code. The bits I still haven't figured out relate to file headers and "materials", which aren't an issue in terms of using some of the hex for RT3. IOW, I have the most of it sorted and a good idea how to do the conversion to usable .3dp code. The only catch at the moment is getting the .3ds to generate the vertex normals in a format we can use. This is an odd one, because loading the file in 3dConverter shows that the normals are all present and accounted for, and pointing in the right directions, but the .3ds hex doesn't show the rotten things AFAICT. I have a nasty suspicion that .3ds sorts them automatically via some shorthand voodoo, which may mean anyone who wants to do modelling will have to generate them some other way (I'll do another test with the same textured cube rotated 45 degrees on one axis, and see if that gives me something more recognisable for normals - ADDED NOTE: Just tried it. No show. Bleh).

All is not lost though, because I have an idea. Said idea is that when I was playing with other export formats I noticed, among other things, that some of them did give code for normals in human-readable format. This means that, if necessary, it would be possible to do two exports: the first in .3ds to get the majority of the code, and the second in another format to get the normals. Of course RT3 wants normals in arse-backwards Martian, but that's not necessarily a problem. All that would be required is to take the human-readable normals from the alternative format and run them through the hex editor's float conversion doohickey to generate the normals the way RT3 wants them.

At the moment the only real catch I can see is that for complex models it will probably make sense to export them as components rather than all in one go. This is just to reduce the strain on the brain, and to minimise the chance of errors, when converting the code to .3dp. If some genius figures out how to script the entire conversion that would be great, but that's above my pay grade at the moment. Even without a fully-scripted automatic conversion, doing it manually should still be a lot nicer and easier than doing all modelling with nothing but a hex editor.

Screenshots attached. The first shows how the Blender stuff works, including noting that the UV mapping is upside down compared to RT3 due to the origin settings in Blender (may be able to change that, but it's not a big deal) as well as showing how the .3ds vertices are mapped to the image.

Basic_cube_textured.jpg (194.42 KiB) Viewed 6663 times

The second shot shows the hex code for the .3ds, including bookmarking and some notes. Bookmarking is still a bit of a mess due to things being added on the fly, but I hope the general idea is clear enough.

Basic_cube_textured_3ds_export.jpg (450.21 KiB) Viewed 6663 times

[Gumboots]

Ok, went through the export formats again and it's easy to get the normals out of a .dae (Collada) export. Even better, since it's a deliberately open format they have very nicely bookmarked the whole thing for us. Screenshot attached.

Basic_cube_textured_45_degree_DAE_export.jpg (175.21 KiB) Viewed 6660 times

[Gumboots]

I've been thinking about this stuff some more. In principle, the .3ds to RT3 .3dp file conversion goes like this:

1/ In .3ds some vertices are duplicated if they are shared between adjacent faces**. These have to be removed for .3dp, but fortunately the duplicated vertices are grouped together in the hex. If the first vertex is shared between three different faces, you will see the first three vertices in the .3ds hex all showing the same x, y and z co-ordinates. That means anywhere in the hex that you see consecutive vertices with identical co-ordinates only the first one needs to be kept and the following ones can be deleted.

2/ Deleting excess vertices will mean subsequent ones will need to be renumbered accordingly when it comes to mapping the graphics. It would be possible to save some work and skip this step if the excess vertices were not deleted, but the trade-off is a higher vertex count for the model, so theoretically it would be worse for performance. It probably wouldn't be much worse, since poly count and gfx mapping would be identical in both cases. Only the vertex count changes. So, maybe not worry about the excess vertices and just keep the lot, depending on how many there are.

3/ The .3ds format uses file headers as a break between the poly declarations (ie: the x,y and z points for the poly corners) and the corresponding gfx mapping (UV co-ordinates). To get it into .3dp format, the superfluous headers have to be removed and the first UV declaration has to be grabbed and inserted after the first poly declaration. Same for the second, third, etc.

4/ Normals. These don't exist in human-readable (even via hex editor) format anywhere in .3ds. AFAICT they would have to be imported from a .dae file, then converted to Little Endian floats to suit .3dp, then inserted in the right location so the first lot of normals goes after the first UV declaration, same for second, etc.

5/ The main file header at the top would also need changing, including the correct vertex and poly counts, but that's trivial.

Ok, so this process is basically clear enough and can be described. That means it can be coded, in a similar way to how the original devs wrote an in-house script to do the conversion for them. Since Blender will quite happily export to .3ds and to .dae, someone could also write a Python script to automatically export from .blend straight to RT3's infamous .3dp format. This is no reason why this would not be possible. It clearly is possible. For someone who is familiar with writing Python add-on scripts for Blender, it is probably not even a particularly difficult job.

Unfortunately I currently know absolutely nothing about Python, or about writing add-ons for Blender. The important word here is "currently". The other important words are "someone who is familiar with writing Python add-on scripts for Blender". Such people may well abound on the various Blender forums scattered about the net. It's probably worth taking a bit of a look and asking some questions.

**I've just realised that this is not quite correct. What actually happens is that .3ds (or at least the .3ds export from Blender) splits a vertex into two or more vertices if, and only if, the UV mapping for that vertex requires two or more different locations on the skin image. This is why in the example above vertices 01 and 02 are duplicates of each other; as are 02 and 03 and 04; and 05 and 06 and 07; and 08 and 09; but 0A and 0B and 0C and 0D are not duplicated even though they share adjacent faces. Due to the way the seaming for the unwrapping is done, the UV mapping for the last four vertices only puts them in one location on the image.

This is a handy thing to know because it could simplify conversions. If as much as possible of the unwrapping is done so that any vertex is only mapped to one UV location, there may be no need to worry about duplicated vertices. A few here and there would make no appreciable difference to game performance.

Also, I did a quick check of the Blender tutorials again and it turns out that there is already a tutorial for exporting basic stuff: https://en.wikibooks.org/wiki/Blender_3 ... rt_scripts

A variation of this, with more stuff added, is basically what is required for fun with RT3 modelling. All that the RT3 .3dp file requires is all the vertex co-ordinates, the normals for the vertices, and the UV mapping for them. It is looking like it shouldn't be that hard to just export that lot straight from the original .blend file, without ever worrying about .3ds intermediate steps at all. All that is necessary is to get the exported data in the correct order for RT3 .3dp, and in Little Endian hex/floats/whatever.