Recently Published: Taking a dive into Oculus’s hand tracking technology

Virtual Legality Blog

A patent application owned by Oculus for “Optical Hand Tracking in Virtual Reality Systems” was published earlier this month. I have provided a link to the patent application here.

First, here’s a quick brief on the mode of operation for the Oculus System. The Oculus Rift headset has a number of infrared LEDs placed on the exterior surface. Multiple cameras are aimed at the play area, looking for that LED output. By placing the cameras in different locations, the exact position of the LEDS (and thus the position of the headset) can be determined.

I assumed that any gloves created by Oculus would work on the same principle, where the gloves would have a number of LEDs woven into the fabric and outputting infrared light. While the glove does indeed have those kind of lights, the glove Oculus is working on appears to do quite a bit more.

The glove illuminates a portion of the skin of the finger with an illumination source (almost certainly an LED) and then images that skin. Any changes in the skin can be tracked and presumably extrapolated to identify finger movement.

The application is pretty vague on how the imaging device accomplishes this task; however, there are a couple of details we can use to infer the technical workings. Paragraph [0033] on page 9 of the linked PDF discusses a bending of the finger where the skin is illuminated, typically around the joint of the finger.

Further, Paragraph [0040] of the application discloses that the imaging device may output a 16 pixel by 16 pixel image. This is extremely small – an example is below:

Thus, it is likely that the imaging device is looking for change in the intensity of the light around the skin joint in order to estimate the bend of the finger. While there is likely much more that can be gleaned from a 16×16 pixel image, I would be surprised if intensity was not the primary variable. When the finger is fully extended, the imaging device should not pick up much light. When the finger is flexed, the light source would be pointed more or less at the imaging device, increasing the perceived intensity of the illumination.

If this technology works, it is a clever and lightweight solution to detecting the flexing of fingers and other portions of a hand. With Oculus acquiring a hand tracking company while also recently showcasing a tracking glove (no internal cameras seen), it is clear that some kind of glove periphery will be on the market in the future.