Development on Doom Classic
By John Carmack, Technical Director, Id Software
Way back in March when I released the source for Wolfenstein 3D Classic, I said that Doom Classic would be coming "real soon", and on April 27, I gave a progress report:
http://www.idsoftware.com/iphone-doom-classic-progress/
I spent a while getting the multiplayer functionality up, and I figured I only had to spend a couple days more to polish things up for release.
However, we were finishing up the big iPhone Doom Resurrection project with Escalation Studios, and we didn't want to have two Doom games released right on top of each other, so I put Doom Classic aside for a while. After Doom Resurrection had its time in the sun, I was prepared to put the rest of the work into Doom Classic, but we ran into another schedule conflict. As I related in my Wolf Classic notes http://www.idsoftware.com/wolfenstein-3d-classic-platinum/wolfdevelopment.htm , Wolfenstein RPG for the iPhone was actually done before Wolfenstein Classic, but EA had decided to sit on it until the release of the big PC / console Wolfenstein game in August.
I really thought I was going to go back and finish things up in September, but I got crushingly busy on other fronts. In an odd little bit of serendipity, after re-immersing myself in the original Doom for the iPhone, I am now working downstairs at Id with the Doom 4 team. I'm calling my time a 50/50 split between Rage and Doom 4, but the stress doesn't divide. September was also the month that Armadillo Aerospace flew the level 2 Lunar Lander Challenge:
http://www.armadilloaerospace.com/n.x/Armadillo/Home/News?news_id=368
Finally, in October I SWORE I would finish it, and we aimed for a Halloween release. We got it submitted in plenty of time, but we ran into a couple approval hiccups that caused it to run to the very last minute. The first was someone incorrectly thinking that the "Demos" button that played back recorded demos from the game, was somehow providing demo content for other commercial products, which is prohibited. The second issue was the use of an iPhone image in the multiplayer button, which we had to make a last minute patch for.
Release notes
Ok, the game is finally out (the GPL source code is being packaged up for release today). Based on some review comments, there are a couple clarifications to be made:
Multiplayer requires a WiFi connection that doesn't have UDP port 14666 blocked. I'm quite happy with the simple and fast multiplayer setup, but it seems like many access points just dump the packets in the trash. If the multiplayer button on the main menu doesn't start pulsing for additional players after the first player has hit it, you won't be able to connect. I have also seen a network where the button would pulse, but the player would never get added to the player list, which meant that somehow the DNS packets were getting through, but the app packets weren't. It works fine on a normal AirPort install... More on networking below.
I took out tilt-to-turn just to free up some interface screen space, because I didn't know anyone that liked that mode, and my query thread on Touch Arcade didn't turn up people that would miss it a lot.
Evidently there are a few people that do care a lot, so we will cram that back in on the next update. The functionality is still there without a user interface, so you can enable it by four-finger-tapping to bring up the keyboard and typing "tiltturn 4000" or some number like that, and it will stay set. Make sure you have tiltmove pulled down to 0. I never got around to putting in a real console, but you can change a few parameters like that, as well as enter all the original doom cheat codes like IDDQD, IDKFA, etc.
I think that the auto-centering control sticks in Doom Classic are a better control scheme than the fixed sticks from Wolf Classic. The advice for wolf was to adjust the stick positions so that your thumbs naturally fell in the center point, so I just made that automatic for Doom. Effective control always involved sliding your thumbs on the screen, rather than discretely tapping it, and this mode forces you to do that from the beginning.
Still, even if the new mode is some fraction "better", there are a lot of people who have logged a lot of hours in Wolfenstein Classic, and any change at all will be a negative initially. In the options->settings menu screen, there is a button labeled "Center sticks: ON" that can be toggled off to keep the sticks fixed in place like in Wolf.
A subtle difference is that the turning sensitivity is now graded so that a given small movement will result in a specific percentage increase in speed, no matter where in the movement range it is. With linear sensitivity, if you are 10 pixels off from the center and you move your thumb 10 pixels farther, then the speed exactly doubles. If you are 50 pixels off from the center, the same 10 pixel move only increases your turning rate by 20%. With ramped sensitivity, you would get a 20% (depending on the sensitivity scale) increase in speed in both cases, which tends to be better for most people. You can disable this by toggling the "Ramp turn: ON" option off.
In hindsight, I should have had a nice obvious button on the main options screen that said "Wolfenstein Style" and had the same options, but I have always had difficult motivating myself to do good backwards compatibility engineering. Even then, the movement speeds are different between the games, so it wouldn't have felt exactly the same.
It was a lot of fun to do this project, working on it essentially alone, as a contrast to the big teams on the major internal projects. I was still quite pleased with how the look and feel of the game holds up after so long, especially the "base style" levels. The "hell levels" show their age a lot more, where the designers were really reaching beyond what the technology could effectively provide.
Future iPhone work
We do read all the reviews in the App store, and we do plan on supporting Doom Classic with updates. Everything is still an experiment for us on the iPhone, and we are learning lessons with each product. At this point, we do not plan on making free lite versions of future products, since we didn't notice anything worth the effort with Wolfenstein, and other developers have reported similar findings.
We have two people at Id that are going to be dedicated to iPhone work. I doubt I will be able to personally open Xcode again for a few months, but I do plan on trying to work out a good touch interface for Quake Classic and the later 6DOF games. I also very much want to make at least a tech demo that can run media created with a version of our idTech 5 megatexture content creation pipeline. I'm not sure exactly what game I would like to do with it, so it might be a 500 mb free gee-whiz app...
Wolfenstein Classic Platinum was a break-in opportunity for the new internal iPhone developers. We were originally planning on making the Spear of Destiny levels available as in-app purchased content. Then we decided to make it a separate "Platinum Edition" application at a reasonable price. Finally, we decided that we would just make it a free update, but something has gone wrong during this process -- people who buy the app for the first time get everything working properly, but many people who upgrade the App from a previous purchase are seeing lots of things horribly broken. We are working with Apple to try to debug and fix this, but the workaround is to uninstall the app completely, then reload it. The exciting thing about Wolf Platinum is the support for downloadable levels, which is the beta test for future game capabilities. Using a URL to specify downloadable content for apps is a very clean way to interface to the game through a web page or email message.
The idMobile team is finishing up the last of the BREW versions of Doom 2 RPG, and work has started on an iPhone specific version, similar to the Wolfenstein RPG release. The real-time FPS games are never going to be enjoyable for a lot of people, and the turn based RPG games are pretty neat in many regards. If they are well received, we will probably bring over the Orcs&Elves games as well.
I want to work on a Rage themed game to coincide with Rage's release, but we don't have a firm direction or team chosen for it. I was very excited about doing a really-designed-for-the-iPhone first person shooter, but at this point I am positive that I don't have the time available for it.
Networking techie stuff
I doubt one customer in ten will actually play a network game of Doom Classic, but it was interesting working on it.
Way back in March when I was first starting the work, I didn't want the game to require 3.0 to run, and I generally try to work with the lowest level interfaces possible for performance critical systems, so I wasn't looking at GameKit for multiplayer. I was hoping that it was possible to use BSD sockets to allow both WiFi networking on 2.0 devices and WiFi or ad-hoc bluetooth on 3.0 devices. It turns out that it is possible, but it wasn't documented as such anywhere I could find.
I very much approve of Apple's strategy of layering Obj-C frameworks on top of Unix style C interfaces. Bonjour is a layer over DNS, and GameKit uses sockets internally. The only bit of obscure magic that goes on is that the bluetooth IP interface only comes into existence after you have asked DNS to resolve a service that was reported for it. Given this, there is no getting around using DNS for initial setup.
With WiFi, you could still use your own broadcast packets to do player finding and stay completely within the base sockets interfaces, and this might even make some sense, considering that there appear to be some WiFi access points that will report a DNS service's existence that your app can't actually talk to.
For every platform I have done networking on previously, you could pretty much just assume that you had the loopback interface and an Ethernet interface, and you could just use INADDR_ANY for pretty much everything. Multiple interfaces used to just be an issue for big servers, but the iPhone can have a lot of active interfaces -- loopback, WiFi Ethernet, Bluetooth Ethernet, and several point to point interfaces for the cellular data networks.
At first, I was excited about the possibility of multiplayer over 3G. I had been told by someone at Intel that they were seeing ping times of 180 ms on 3G devices, which could certainly be made to work for gaming.
Unfortunately, my tests, here in Dallas at least, show about twice that, which isn't worth fighting. I'm a bit curious whether they were mistaking one-way times, or if the infrastructure in California is really that much better. In any case, that made my implementation choice clear -- local link networking only.
A historical curiosity: the very first release of the original Doom game on the PC used broadcast IPX packets for LAN networking. This seemed logical, because broadcast packets for a network game of N players has a packet count of just N packets on the network each tic, since everyone hears each packet. The night after we released the game, I was woken up by a call from a college sysadmin yelling at me for crippling their entire network. I didn't have an unlisted number at the time. When I had decided to implement network gaming, I bought and read a few books, but I didn't have any practical experience, so I had thought that large networks were done like the books explained, with routers connecting independent segments. I had no idea that there were many networks with thousands of nodes connected solely by bridges that forwarded all broadcast packets over lower bit rate links. I quickly changed the networking to have each peer send addressed packets to the other peers. More traffic on the local segment, but no chance of doing horrible things to bridged networks.
WiFi is different from wired Ethernet in a few ways. WiFi clients don't actually talk directly to each other, they talk to the access point, which rebroadcasts the packet to the destination, so every packet sent between two WiFi devices is actually at least two packets over the air.
An ad-hoc WiFi network would have twice the available peer to peer bandwidth and half the packet drop rate that an access point based one does. Another point is that unlike wired Ethernet, the WiFi link level actually does packet retransmits if the destination doesn't acknowledge receipt. They won't be retransmitted forever, and the buffer spaces are limited, so it can't be relied upon the way you do TCP, but packet drops are more rare than you would expect. This also means that there are lots of tiny little ACK packets flying around, which contributes to reduced throughput. Broadcast packets are in-between -- the broadcast packet is sent from the source to the access point with acknowledgment and retransmits, but since the access point can't know who it is going to, it just fires it out blind a single time.
I experimentally brought the iPhone networking up initially using UDP broadcast packets, but the delivery was incredibly poor. Very few packets were dropped, but hundreds of milliseconds could sometimes go by with no deliveries, then a huge batch would be delivered all at once. I thought it might be a policy decision on our congested office access point, but it behaved the same way at my house on a quiet LAN, so I suspect there is an iPhone system software issue. If I had a bit more time, I would have done comparisons with a WiFi laptop. I had pretty much planned to use addressed packets anyway, but the broadcast behavior was interesting.
Doom PC was truly peer to peer, and each client transmitted to every other client, for N * (N-1) packets every tic. It also stalled until valid data had arrived from every other player, so adding more players hurts in two different ways -- more packets = more congestion = more likely to drop each individual packet. The plus side of an arrangement like this is that it is truly fair, no client has any advantage over any other, even if one or more players are connected by a lower quality link. Everyone gets the worst common denominator behavior.
I settled on a packet server approach for the iPhone, since someone had to be designated a "server" anyway for DNS discovery, and it has the minimum non-broadcast packet count of 2N packets every tic. Each client sends a command packet to the server each tic, the server combines all of them, then sends an addressed packet back to each client. The remaining question was what the server should do when it hasn't received an updated command from a client. When the server refused to send out a packet until it had received data from all clients, there was a lot more variability in the arrival rate. It could be masked by intentionally adding some latency on each client side, but I found that it plays better to just have the server repeat the last valid command when it hasn't gotten an update. This does mean that there is a slight performance advantage to being the server, because you will never drop an internal packet.
The client always stalls until it receives a server packet, there was no way I had the time to develop any latency reducing / drop mitigating prediction mechanisms. There are a couple full client / server, internet capable versions of Doom available on the PC, but I wanted to work from a more traditional codebase for this project.
So, I had the game playing well over WiFi, but communication over the Bluetooth interface was significantly worse. There was an entire frame of additional latency versus WiFi, and the user mode Bluetooth daemon was also sucking up 10% of the CPU. That would have been livable, but there were regular surges in the packet delivery rate that made it basically unplayable.
Surging implies a buffer somewhere backing up and then draining, and I had seen something similar but less damaging occasionally on WiFi as well, so I wondered if there might be some iPhone system communication going on. I spent a little while with WireShark trying to see if the occasional latency pileup was due to actual congestion, and what was in the packets, but I couldn't get my Macbook into promiscuous WiFi mode, and I didn't have the time to configure a completely new system.
In the end, I decided to just cut out the Bluetooth networking and leave it with WiFi. There was a geek-neatness to having a net game with one client on WiFi and another on Bluetooth, but I wasn't going to have time to wring it all out.