Oculus Rift: Motion Tracker development
- The hardware is expensive: The latest Sony HMZ-T1 is 799$.
- The hardware is not good enough: The motion sensor, screen latency and screen limited Field of View are unable to provide a feeling of immersion.
- The software is non-existent: No major studio has ever supported VR Headsets.
But things are about to change thanks to a device to be released this September via kickstarter funding: the Oculus rift.
Palmer Luckey the man behind the project has managed to put together a set of outstanding components that provide very low latency and a significantly increased field of view ; he also managed
to keep the price under 250$. This tackle the two first problems related to hardware.
As for the software, Luckey secured id Software to support the device officialy: Doom3 BFG Edition will be compatible out-of-the-box. Valve is also confirmed to be working on Portal compatibility:
EDIT : The kickstarter project opened on August 1st and reached its 250,000$ goal within an hour. The hype around this thing is beyond everything I have ever seen in the field. Looks amazing:
Motion Sensor: FSRK-USB-2
Unfortunately developers will get their hands on an Oculus RIFT only starting November 2012. But if you are eager to start developing you can ordered the motion tracker chipset that will be used: Hillcrest's FSRK-USB-2. It is a very tiny printed board that you can connect via micro-USB:
The software used to receive signal from the motion tracker is called libfreespace. It is built on top of libusb and comes with many examples that make it easy to integrate to any engine.
With a little bit of electrical tape and Windows 8 sunglasses it is possible to build a unglamourous but efficient prototype and experiment with headtracking:
The complete "deviceless" setup...
Integration to a 3D engine
Integrating libfreespace inputs to a 3D engine can be done in a few minutes with the following architecture:
The engine spawns a "pump" thread: An infinite loop that will endlessly requests the motion sensor orientation via
freespace_readMessage. The result is stored in a synchronized array. Upon rendering the engine just get the latest orientation know via the shared variable without waiting for communication via libfreespace: This brings latency down a lot since the time waiting for system calls to return from the USB bus is spent in the "pump" Thread and not in the "Engine" Thread.
You can also request a specific firmware from Hillcrest Labs that crank up frequency from 125Hz to 250Hz which will reduce latency further. You also need to enable Triple Buffering. Finally, the source code can be found on GitHub.
Once the motion tracking events are integrated to the engine, the video output also has to be significantly altered:
- The screen must two rendition from two point of view: One for the left eye and one for the right eye, side by side.
- Each eye image must also be distorted by the inverse amount operated by the physical lenses in the Oculus RIFT: This is called "pre-warping"
In the end the result has to look like this:
In practice this is done as follow:
- Render each eye one after an other to an offscreen surface (with OpenGL this would be a Frame Buffer Object).
- Use a special shader to perform the warping while moving from FBO to the framebuffer.
Add a comment
Devices such as Rift from time to time you're trying to get through. If you will be well made and affordable to everyone would be tempted to such a gadget.
Fabien I have a request I sent you the e-mail you asking for dEngine. I would be grateful for your response.
Thank you for this interesting article. I've used your Quake 2 code base to integrate a 512 Hz Xsens IMU. The raw data is processed in a quake2.exe thread to integrate the gyro measurements. An error-state Kalman filter calculates the gyro bias, so after a few seconds there is basically no drift.
Here is a small (bad-quality) video: http://youtu.be/DhUSa5nvjeI
About head tracking: as my hobby project when I was a student, I've done this: http://www.youtube.com/watch?v=jZFJKv9yuIo
I hope one day every single 3D game will work like it.