Learning Unity

Introduction

Several months ago, I started learning Unity. I ran through some of the tutorials on-line and learned the basics of how Unity works. I know that tutorials will teach all the basics, but I also know that to really learn something, you need to build something real. For me, I usually choose a subject or project that interests me. That keeps me motivated to plow through the more difficult aspects of learning something.

To learn Unity, I decided to try and render a map from the original Descent I game. Why Descent? Why not! I know the levels very well (I sunk an enormous amount of hours into this game when I was young). I also know how the game is supposed to look and feel. I also have a working copy of the game (a copy that works with Windows 10) and I have the source code (which is available on-line).

Getting Started

To tackle a problem like this, I needed to limit the scope to something small at first. So I decided that I needed to decode part of the Descent data file. This file is known as the HOG file. The HOG file is nothing more than a collection of files all stuck together. Each file has a header that tells what the file name is and the offset inside the HOG file. That, is where I decided to start.

I found this old website that contains information in the HOG file: Descent-Network.com. I wrote a throw-away console application that reads all the file headers and prints the file names. As it turns out the descent.hog file contains 159 files. It looks like this:

After browsing the file types that are available, I decided that I needed to grab one RDL file and decode it. The RDL file contains the vertex information and the structure data for an entire level. To keep the scope small, I decided to decode one level and then plot all the vertex points in Unity using cubes for each vertex point. Armed with the on-line information about the RDL file, I started to decode the information in the descent.hog file.

The RDL header segment was easy. I decoded that information and obtained the offset to the Mine Data. I attempted to retrieve the vertex list and I ran into problems. The vertex count, times the bytes used by each set of vertices (12 bytes), exceeded the file length. I tried reversing the size of the file to determine the maximum possible vertices and couldn’t come up with a solution. So I dug around in the descent source code. Inside the MINE directory is a C source file named GAMEMINE.c. This file contains a function named load_mine_data_compiled_new. Inside that function, I discovered that there was a 1 byte version that preceded the vertex count. That means that I was off by one byte. As soon as I skipped over that byte of information, the vertex count became a reasonable size. For level 1, the vertex count is 994. The cube count was 285. I wasn’t sure what cubes were yet, but it seemed like a good number.

Next, I setup a loop to read the vertices into a list of structures. The data is stored as fixed point (you remember fixed point right?). The fixed point format is 16:16. That means that we’ll need read the data as a 32 bit integer and divide by 65536.0 to get our fixed point number into a floating point number. When I first wrote this, I read the numbers as unsigned 32 bit numbers. When I started troubleshooting polygons, I discovered that there were coordinates that were in the 65,000s. When I consulted the Descent level editor, I discovered that negative coordinates were possible and realized that I needed to read the fixed point numbers as signed integers.

As I alluded to, I went out and found a Descent I level editor that works in Windows 10. This is DLE XP. You can download the level editor by clicking here. The level editor was convenient for checking coordinates and matching the appearance of the level with what I rendered in Unity. When I rendered one cube per vertex in Unity I produced this output:

In the front is the starting cube and the curved shaft at the bottom is the exit tube. The descent level editor is a bit distorted, but you can see a similar match here:

The cube that is selected is cube number zero.

Cubes

The next task was to decode the mine structure. I always assumed that the mines were just constructed out of triangles that form polygons from the vertices. Following the vertex data in the HOG file are the cubes. As it turns out, the entire level is constructed from cubes. Cubes consist of six sides and eight vertices (I know, you already knew that). Vertices are shared between connected cubes. Cubes can have any of 1 to 6 sides. If a cube connects to another cube, then there is no side for the connecting square. Cubes can be distorted to form any desired shape. To create a complex room, many cubes can be combined and distorted into any shape.

Why cubes? The storage structure hints at the reason that the engineers chose this method. Each cube data structure stored contains a mask that tells which faces are connected to another cube. This means that it would be easy to create an algorithm to find a path from one cube to another. This is probably used by the robots to track you down after they’ve spotted you (like those moments when you blow past a robot and just keep going so you don’t get shot).

I wrote some dirty code to try and decode enough of the cube data to render all sides of each cube in the mine to see if it resembled the level editor view. This turned out to be more difficult than I imagined. First, the website describing the structure is incomplete. Fortunately, I had the source code to fall back on. The uvl section got me the first time around. There are 4 uvl’s for each side.

The next tricky part was the number of textures and uvls. I decided to use a similar “if” statement logic to the C code. The trick with that was the side number, which could be a -1 if the HOG file contains the number 255. I was skipping over this data because I didn’t really need it yet (I just wanted the data regarding the vertices that belonged to each cube). The problem is that the cube data is variable length. If the wrong amount of data is consumed, then the next cube data starts at the wrong offset in the HOG file and it contains garbage data. Therefore, I had to decode the 255 into -1 and include the texture and uvl data for those instances.

The next issue I ran into was my own fault. I spent hours going line by line in the C code and matching the number of bytes I was consuming. What I missed was the primary texture. I did not increment my offset counter by 2 when I read the primary texture. That caused by first cube to be off by 8 bytes (there are 4 walls in cube zero and each texture is 2 bytes).

How did I discover this? I knew from the level editor that cube 1 was open in the front and back faces. I reverse engineered the mask that should be present for cube 1 and I determined that it would be 0x30 (ignoring the energy center bit). When I searched the data starting from the first cube (cube 0), I know that the first byte is 0x30 (plus the energy center bit) and the next 0x30 occurs 163 bytes later. When I ran my code, I was ending at 155 bytes. That’s 8 bytes short. That told me that I needed to go back over the C code line by line, looking for something that was 8 bytes long. Or in this case, it was 4 walls with 2 bytes each.

To detect that I was reading the data correctly, I tested the children (or neighboring cube index) to see if it was within the range of total cubes. If it was less than zero or greater than the count, then I knew I had a problem with my algorithm. I also checked the vertices to make sure that the vertex index for each of the 8 vertices was greater than zero but less than the count. The code read cubes up until a point where there was a neighboring cube of -1. Apparently, there is a wall type that is not a wall, but there is no neighbor cube, so a -1 is stored and that is legal. So I changed my verification. Then my code read up to another point and I remembered reading in the documentation that the final mine exit face is set to a -2. So I adjusted my verification to allow a -2. Then all the cubes read correctly.

Next, I had to learn how to manipulate a Unity mesh in code. This was educational. I had to dig around until I found an article that showed how to render a single cube consisting of 12 triangles (2 for each side). I used that pattern as a dumb and dirty test. Ultimately, I want to change this so that vertices are shared instead of generating 4 per side of each cube. But I needed to get a result. Here’s what cube 0 looks like in unity:

Here’s cube 0 in the level editor (for comparison):

Cube 1, is right next to cube 0:

To make the cube appear, I arrange the triangles to point outward. Eventually, I’ll need to flip the triangles so they can be seen from the inside. Before I do that, I’m going to render all the triangles for every cube in one mesh so that the entire level will show up as one solid polygon. The level editor is smart enough to hide the top most polygons (like looking through the roof). I only intend to render all 6 sides of each cube to see the shape of the level.

After that task, I plan to read the cube mask to determine which sides need to be rendered and not render the sides that connect two cubes together.

Finally, I’m going to attempt to render the correct textures onto the cubes so they don’t appear as pink boxes.

The File Format

To help me visualize the format of the HOG file, I drew a map in Visio of what I have decoded so far. This map is not complete. I’ll add to this map as I decode more of the RDL file. But for now, here’s what I have:

I’ll get into more details in my next post. I’ll also post some code on GitHub that you can download.