Second Reality Code Review: Part 1 (Introduction)

On July 23, 2013 the source code of Second Reality was released. Like many, I was eager to look at the internals of a demo that inspired me so much over the last 20 years.

I was expecting a monolithic mess of assembly but instead I found a surprisingly elaborated architecture, mixing several languages in an elegant way. The code is something like I had never seen before that perfectly represents two essential aspects of demomaking :

• Team work.
• Obfuscation.

The Demo

Before starting with the code, here is a link to a HD video capture (by Michael Huth) of the legendary demo. Nowadays, it is the only way to fully experiment the marvel without graphic glitches (even DOSBox cannot run it properly).

First contact with the code

The source code is hosted on GitHub. It is one git command away :

   git clone git@github.com:mtuomi/SecondReality.git

  

At first the content is confusing: 32 folders and a mysterious U2.EXE that won't run under DosBox.



The working title of the demo was "Unreal 2" (the first "Unreal" was Future Crew's previous demo, released for the first Assembly in 1992). Only later during development the name was changed to "Second Reality". This explains "U2.EXE" but not why it doesn't work...


Runnig CLOC provides interesting metrics:

    -------------------------------------------------------------------------------
    Language                     files          blank        comment           code
    -------------------------------------------------------------------------------
    Assembly                        99           3029           1947          33350
    C++                            121           1977            915          24551
    C/C++ Header                     8             86            240            654
    make                            17            159             25            294
    DOS Batch                       71              3              1            253
    -------------------------------------------------------------------------------
    SUM:                           316           5254           3128          59102
    -------------------------------------------------------------------------------


  

• The codebase is "only" 50% assembly.
• The codebase is almost twice as big as Doom engine.
• Seventeen makefiles are there. Why not only one ?

Running the demo

It is hard to figure it out but it is possible to run what was released via DosBox: You have to rename U2.EXE and run it form the right place. Once I learned about the internals of the code, it made a lot of sense:

        CD MAIN
        MOVE U2.EXE DATA/SECOND.EXE
        CD DATA
        SECOND.EXE

      

Et voila !

Architecture

Back in the 90s, the demo was mostly distributed via floppy disks. After unzipping, two big files would be installed : SECOND.EXE and REALITY.FC:

    .               <DIR>         01-08-2013 16:40
    ..              <DIR>         01-08-2013 16:40
    FCINFO10 TXT           48,462 04-10-1993 11:48
    FILE_ID  DIZ              378 04-10-1993 11:30
    README   1ST            4,222 04-10-1993 12:59
    REALITY  FC           992,188 07-10-1993 12:59 
    SECOND   EXE        1,451,093 07-10-1993 13:35
        5 Files(s)     2.496,343 Bytes.
        2 Dir(s)     262,111,744 Bytes free.

Coming from a game developer background I always expected the big picture to be :

• SECOND.EXE: The engine with all effects in the executable.
• REALITY.FC: The assets (music, sound effects, images) in a proprietary/encrypted format ala Doom WAD.

But after reading MAIN/PACK.C I discovered that I was completely wrong : The "Second Reality" engine is just a Loader and an Interrupt server (called DIS). Each scene (also called "PART") during the demo is a full self-contained DOS executable. Each parts are loaded by the Loader and started one after an other. The PARTs are stored in encrypted form at the end of SECOND.EXE :

• REALITY.FC contains the two musics played during the demo (with padding and a marker at the beginning for obfuscation).
• SECOND.EXE contains a loader and a Demo Interrupt Server (DIS).
• Appended after the end of SECOND.EXE, the 32 PARTs of the demo as DOS executables (encrypted).

This architecture offers many advantages :

• Improved collaborative process in the team: Each member of the team can work independently on its PART as long as it produces an executable with a _start symbol and remains within the memory budget (450K).
• Encrypting the EXEs at the end of SECOND.EXE and having them loaded at runtime made reverse engineering harder.
• Fast startup : The Loader and DIS are only 20K. DOS loads them super fast.
• Easy memory management: Once a PART is done, the loader overrides it with the next PART and free ALL the allocated memory.
• No asset manager/loader needed: As seen later each PART had its assets (mostly images) compiled within the code: Loading an EXE also loaded all the images it needed like a nice package.
• Any language could be used to program a PART: C, Assembly ... and Pascal can be found in the code.

Recommended readings

The three pillars to understand Second Reality source code are VGA, Assembly and PC architecture (PIC&PIT Programming). Here are incredibly helpful links :

• VGA Architecture: FreeVGA Project
• The Art of Assembly (especially chapter 17 about Interrupts and ISRs).
• 8254 PIT Programming
• Since the source if packed with "FAR" and "NEAR" pointers, it would be useful to read Intel's Basic Architecture Documentation, especially Chapter 4.3 about "Pointers Data Types".
• The ubiquitous Michael Abrash articles from Graphic Programming Black Book: Chapter 3: (Zen Timer and PIT), Chapter 24: VGA tour, Chapters 47,48 and 49: (Mode X).

Next

Second Reality Engine