Duke Nukem 3D: CODE LEGACY (PART 3 OF 4) >>
Build is an outstanding engine and the numerous games it powered brought very much deserved fame and fortune to both
parties: Ken Silverman and 3D Realms.
Ken Silverman completed his contract:
He delivered the binary of an astonishing 3D engine with well documented methods and assets formats.
As a testimony of his skills, 3D Realms credited him with "Ken '
I can do that' Silverman".
But Build development was focused on features and performances, not portability or readability. Having read the code I believe
open source developers stayed away from it because :
- It is discouragingly hard to read and extract knowledge from.
- It was not portable.
I have listed on this page a few of the difficulties I identified. I also released Chocolate Duke Nukem 3D a port that address those issues. Because I wanted people to remember what treasure of ingenuity it took to build a 3D engine back then. But also because I wanted people to understand how a teenager driven by passion contributed to one of the greatest game of all time.
Hard to understand
- Only three modules makes the entire codebase hard to mentally break down in sub-modules.
typedefandstructare not used internally (only used for public data:sectortype,walltypeandspritetype). Only arrays of primitive data types. (i.e: GRP Filesystem):static long numgroupfiles = 0; static long gnumfiles[MAXGROUPFILES]; static long groupfil[MAXGROUPFILES] = {-1,-1,-1,-1}; static long groupfilpos[MAXGROUPFILES]; static char *gfilelist[MAXGROUPFILES]; static long *gfileoffs[MAXGROUPFILES]; static char filegrp[MAXOPENFILES]; static long filepos[MAXOPENFILES]; static long filehan[MAXOPENFILES]- Comments are sparce. This makes math methods difficult to understand (i.e: inside):
inside (long x, long y, short sectnum){ walltype *wal; long i, x1, y1, x2, y2; unsigned long cnt; if ((sectnum < 0) || (sectnum >= numsectors)) return(-1); cnt = 0; wal = &wall[sector[sectnum].wallptr]; i = sector[sectnum].wallnum; do{ y1 = wal->y-y; y2 = wall[wal->point2].y-y; if ((y1^y2) < 0){ x1 = wal->x-x; x2 = wall[wal->point2].x-x; if ((x1^x2) >= 0) cnt ^= x1; else cnt ^= (x1*y2-x2*y1)^y2; } wal++; i--; } while (i); return(cnt>>31); } - The drawing routines were x86 hand optimized assembly. Those were ported back to C later but there are still extremely hard to read as well.
- Symbols names are sometimes controversial:
static long xb1[MAXWALLSB], yb1[MAXWALLSB], xb2[MAXWALLSB], yb2[MAXWALLSB]; static long rx1[MAXWALLSB], ry1[MAXWALLSB], rx2[MAXWALLSB], ry2[MAXWALLSB]; static short p2[MAXWALLSB], thesector[MAXWALLSB], thewall[MAXWALLSB];. - Magic numbers (especially when manipulating walls, floor and ceiling) are mysterious.
- The main Translation Units (game.c and engine.c) are so big that they slow down IDE. Sometimes to the point of crashing XCode.
Hard to port
- Assumed Little-Endian CPU.
- Used
longtype across the engine expecting them to always be 32 bits wide. - Assumed 32 bits addresses and manipulated them as integers.
- Features hand optimized x86 assembly with no C fallback.
- Expected 120 ticks per frame: The way DOS timer worked.
- No abstraction layer. Calls to filesystem/renderer were raw.
Lost ressources
Ken Silverman once kindly collaborated to the JonoF port and posted many explanation of his algorithms.
Unfortunately the forum had to be taken down due to spam. It seems all those precious information
are gone :( !
It is with disappointment that I've closed the forum on this site. Spam bots made it impossible to keep it under control, and although there is some valuable content within its posts, it's just not worth my or the other moderators' time to maintain it.
I suggest you head over to the Duke4.net forums if you want to remain involved in the community.
Add a comment
Comments (46)
It's a pitty that they could not just refurbish the graphics, add some new maps and release a new Duke. What they released was a "let's get things done" game and called it "Duke Nukem Forever".
Maybe there will be a "Duke Nukem Community Edition" with all the cool gameplay...
Thanks for your work!
same problem when running Starcraft or Diablo II, the easiest solution is to kill explorer.exe before run the game (use taskmanager or create a batch command :)
May I ask you to write an in depth article on how you actually *read* code? Where do you start, how much notes you make, maybe you document the code you read, and so on. In my experience, reading code is an essential skill for any programmer, but there seem to be absolutely no resources explaining how to do it properly. I've tried reading quake3 code (three times) and tried reading eAthena code (open-source server engine for MMORPG Ragnarok Online) but after couple days I've got lost in detailes and in the end had no clue about how thing works. You are on the other hand extremely skilful at the task and your insights are very valuable for an open community.
I can't read the article via Instapaper app, and the page is otherwise unreadable on mobile (and I don't have time to read at my desk)
I've always wondered, what software do you use to make your diagrams?
Excellent post btw, nice deep analysis of the source and render mechanics. I remember looking at ENGINE.C when it was first released and indeed did not feel encouraged to continue investigating :) But I respect that he got a post-DOOM engine working smoothly on such feeble hardware. It took some sacrifices, especially then. Compilers now can turn a ton of types, lambdas and whatnot into a tight loop but back then you really had to write high-level assembly to get stuff fast. Props to Ken and you!
As always a great read!
Once I saw it over at HackerNews while at work, I knew I had to make a short day :-)
One thing: Could you elaborate on why using cross products makes a difference to using a dot product if you have fixed-point/integer math?
Also, two small typos in the related source code snippet:
* If I am not color-blind, the one vector is blue and not green. ;-)
* The return-statement should compare against the 2D zero-vector not a scalar, right?
Cheers,
Daniel
http://web.archive.org/web/*/http://www.jonof.id.au/forum/
But why the hell would you use quicktime videos... :D
Chrome give me an alrt that is due to a plugin named : quick time
use html video tag or flash for video other solution are lame !
Thanks!
I wonder what tools you use to make your diagrams.
I appreciate to read your comments and notes on different games, its very nice of you to take the time and read and share it with all of us.
Have a good life
//David
Thanks.
I also enjoyed, very much, your article's page presentation.
Nicely done and pleasant to view.
I had no problem viewing any of the images or QT videos.
Depending on the editor, you have at most 24 lines of 80 columns on a VGA screen. The inside function you mention is 20 lines long - pretty close to that limit. The desire to keep logic sections within the size of the screen is much more important than comments for other people years down the line to enjoy.
Also, regarding globals: While I'm not sure about 486 specifically (I'm too young), a global array of a primitive type (1, 2, or 4 bytes per element) is very fast to access. I believe the address of the element could be calculated as part of the memory fetch instruction.
An array of structs, whether as a global or on the stack, would probably require 2 instructions to calculate the memory address, and a third for the actual fetch.
Aditionally, the size of the stack would have been tightly constrained.
Any plans to mention the active Source Port EDuke32?
http://www.eduke32.com/
OpenGL is very much improved, there is now true Room over Room capabilties and much more.
The team has an very active forum
http://forums.duke4.net/
Best
Taamalus
* Putting as much code into a single file as possible was a good way to help the primitive compilers do some optimisations & inlining
* Arrays of scalars instead of structs are simply much faster to iterate through given limited CPU caches
* The example math method is straightforward bitmask operations on the sign bits
* Ahh, hand-coded assembly. I'll admit it takes the right kind of masochist to read it :)
* Nothing magic about those numbers! Just bitmasks.
* IDEs have no excuse to be slow these days. DJGPP etc handled huge files just fine back then!
I think you mean 120Hz, not 120 ticks per frame (that would be far too much IRQ overhead). But the PIT native frequency was 1193180Hz. You set a divisor value from 0-65535 which determines the frequency of IRQ0. You can't get 120Hz exactly (more like 120.002Hz) so it's unusual to quantize rendering / physics to this as you get graphics "tearing". What did they use this timer for?
Thanks for sharing all this knowledge.
I get C:\Duke3D\ChocolateDuke3D.exe is not a valid Win32 application when tying to start.
When a get a bit more time I will try and get this complied locally, both on Win XP and Linux(Raspberry Pi)
The Videos are in the folder fd.fabiensanglard.net/duke3d/movies/
the 3 *.m4v-files
VLC plays them just fine
Maybe, if you're running out of code bases to review, you should turn to Aleph One. Its kind of "Chocolate Marathon" ( http://en.wikipedia.org/wiki/Marathon_2:_Durandal ).
Its surprising that most of my pet game engines of the past have similar algorithms (in fact, the "current sector" tracking is exactly the same) to Build.
Greetings from Rio ;-)
"The renderer does not rely on recursive function (the way Doom did when walking the BSP). Instead a stack and a loop is used (for sector flooding)" (from your raw notes)
There is no recursion based on machine stack, but rather on software stack. Maybe this is because (if my memory serves me well) Ken prototyped the engine in QuickBASIC and (from my faded memory again) it didn't have recursion until 4.0;
Or perhaps not...maybe was stack size thing from those old dark (and very fun) days?
QuickBasic was a fine "IDE" for some quick hacking. I grew making lousy 3D games in it myself. My biggest inspiration at the time was Ken, but I wasn't aware he used the very same tool I was;
Very interesting review as usual. I already took a look at code some years ago and discover most inner workings but some things remains mysterious, your review cleared them out.
I expected to see an explanation on how mirrors works (why they require an additional empty room with special texture). Still unclear to me.
One interesting trivia : the way updatesector() works caused some well know glitch/cheat during the game, that was used a lot in multiplayer (warp glitch) : if you make player run very fast in a place were sectors are very small (eg : stairs), updatesector get lost and start scanning all sectors. If there is two (or more) overlapping sectors where player is, game sometimes choose the wrong sector (depending their order), and it will teleport player there.
see video : http://www.youtube.com/watch?v=VUOhZIdCSiI