Game Engine Black Book Postmortem

I am pleased to announce that the Game Engine Black Book about Wolfenstein 3D has shipped. It is 316 pages, full color and made of three parts describing the hardware of 1991, id Software tools, and the game engine internals. You can read a preview on Google Books and buy it here:

• Amazon.com (.fr, .de, .co.uk, .ca,...).
• Google Play Books for web browsers, Android devices, and iOS devices.

It took me three years to complete this project. It had its fair share of heavy winds and bumpy roads. I thought it could benefit some writers in the making to give a postmortem and share what I learned in the process.

First attempt

I started the project in July 2014 shortly after accepting a new full-time position in California. I picked Google Doc to do the writing and Gliffy for the drawing. I worked on it every evening and by August I thought I would have a 140 pages booklet within an other month. I updated the status on my website as "almost done". I uploaded the PDF on createspace.com, ordered a proof print and the result was horrible. The screenshots had lost all their pixels and my drawings were blurry. On the left is what I saw on my screen and on the right is what I saw once printed. Notice on the right how bilinear filtering blurred the pixels.


  

I knew about the 300 dpi rule and I confidently ignored it, assuming printer would use "nearest neighbor" magnification. It turns out some printer use nearest neighbor and some use bilinear filtering. createspace.com did not do what I was hoping for. The solution was to upscale assets and use vectorial format for drawings instead of rasterized format. I started converting the assets and the GUI did not like it. It could takes seconds to scroll to the next page. To use a web-based tool to edit a book featuring hundreds of screenshots in 2250x1400 like the Game Engine Black Book was a BAD idea. I ran into other limitations like no support for vectorial drawings and no possibility to pick the page color. I was clearly not using the right tool for the job.

Switching tools

I googled around to find out what technical authors used. Something that would be more adapted to my niche usecase. I ended up dropping WYSIWYG and go minimalistic with Sublime Text editor, pdfLatex to compile tex into PDF, Inkscape for vectorial drawings and GitHub for source control. With this setup I was able to create new command in LateX to use lightweight 75 DPI assets while writing and swap the command definition to use 300 DPI assets when compiling the book for printing. From typing to compiling the PDF, everything was fast.

Everything was fast as long as you knew what you were doing. I was used to jump into new programming languages and I assumed LateX would be no different. It turned out learning LaTeX on the go made the project advanced at the speed of an hangover sloth. It sucked my energy because I literally had to Google everything I was trying to do. From bolding font to creating a simple array, things that would have taken me seconds with Google Docs or HTML now took hours. I spent evenings struggling and decyphering compiler errors. I would run into weird issues when using poorly documented packages. For example package "listings" renders code listing on Safari with weird white lines artifacts. On Chrome it looks ok.

  

All this friction resulted in evenings pursuing what felt like side quests. I ended up losing all motivation. By October 2014 I had burned out. I stopped submitting commit to github and the Game Engine Black Book was dead.

Hibernation

The project stayed in limbo for two years after that. As you can see on the graph above, I picked it up again around Christmas 2015 but I got frustrated again pretty fast and as soon as I returned to work I stopped working on the Black Book.

Blessing in disguise

In October 2016 I injured a shoulder while training at the climbing gym. It was so bad I could not move my arm anymore. I started doing PT to recover and I found myself with a lot of time at home on my non climbing hands. I dusted off my github project "gebbwolf3d". I attempted a third time to get going and reached the same dead end.

Being stubborn, I tried a different approach. I recognized that what destroyed my motivation was the slow pace of progress. Going to bed after hours of struggle was disheartening. I came up with a plan. I monitored myself and if I started to feel frustrated with a drawing or a graph or a table, I would simply draw it on paper, take a picture and import it as a raster. It looked horrible but that was a placeholder which allowed me to move forward no matter how limited my LaTeX skills were. This adjustment turned out to be all I needed. At Google they say that "speed is a feature". Speed is paramount in a 3D engine. It turned out to be important for me too.

As I witnessed my shoulder sustain more and more weight at PT, more and more pages were added to the book. And also drawings, plenty plenty of drawings. By the end of January 2017 my shoulder had fully recovered and it was clear I was also going to be able to finish the book. Because createspace took one week to print and ship a proof, I used that time as resting spots. The seven proof prints are clearly visible on the graph as well as the final push.

Writing is 90%, proofreading the other 90%

Writing and drawing was difficult but proof reading was excruciating. To read the same thing over and over again lead to losing perspective and fatigue. I understand now why so many bad movies are released by Hollywood. When you work on something for too long you cannot tell anymore if it is good or if it is bad. I was lucky to have amazing people around me volunteering to proof read. I would have never published the book without them.

Take Away

It was great to make a dream come true and learn so much about technology from the past. To understand how programmers from the 90s strived like us. The spirit it takes today to reach new frontier is the same it took 20 years ago. It helped me to look at my work with a positive attitude and more confidence. And recognize that we do what we do "not because it is easy but because it is hard".

If I were to take away only one thing from this beautiful adventure it will be the support I received. To read detailed answers to my obscure questions about EMS from the original team and passionate enthusiasts. To see people volunteer their collection of 386 to run benchmarks for me. To witness someone carefully print a proof on an A4 and take the time to read it. To see a co-worker who often spend 9 hours a day at work take the time on his weekend to fix my horrible typos. To see my brother on the other side of the planet figure out VGA masks. These things moved me more than anything. Their kindness and generosity were humbling.

Recommended Readings