VR technology is rapidly expanding and advancing, enabling more realistic experiences with each new iteration. Haptic technology has been all the rage, but excitement over better haptics is now giving way to the need for better visual renderings.
While haptic technology is touch-oriented, hand tracking technology significantly improves visual presentation. Considered a “level up”, audiences are excited to test it. Demos of systems that allow users to experience the upgraded visuals have been well-received, and Facebook’s Oculus Quest is adding hand tracking to their latest system in 2020.
Current haptic technology centers on replicating what it feels like to touch an object, and allowing users to experience that feeling in the virtual world. However, even the most realistic tactile sensation can be irrelevant if users can’t get past the issue of laggy visuals, which can ruin the VR experience.
Nothing yanks a user back out of an authentic virtual reality experience like jitter — or “judder”, as some are now calling the jerky motions that can occur when a user is moving their hands. Furthermore, a lack of consistency in visuals is the prime cause of “simulator sickness.” Improving immersion and resolving latency issues is critical — and haptics isn’t delivering a solution, leading many to trade haptics for hand tracking.
Request a demo today.
Where haptics win out — and where they fall short
Haptic “touch” controllers for VR were developed as a way to make the virtual environment feel more realistic. Vibrations can be sent to the fingers from the controller, providing an extra layer of experience.
Advances continue to be seen in haptics, particularly the ability to “force-feed” sensation as shown in this video.
Different haptic glove-based systems are in development, (including three already patented VR glove ideas by Facebook). Haptic technology is being continually improved, with gloves, hats, jackets, and even full bodysuits on the table for developers to compete over.
These haptic upgrades intend to take the “feeling” from mere touch sensation and vibration to feedback that makes the user feel pressure. This could prevent a VR participant from punching through a wall by providing resistance. Forced feedback haptics of this kind generate push and pull sensations, making interactions with virtual objects more realistic.
However, using haptic technology in handheld controllers limits the rest of the VR experience. With haptics, the fingers still have to be positioned in a certain way to hold the controllers. Even when gloves are in everyday use, haptics won’t be sufficiently able to provide the precision visuals required for true user immersion.
No matter how realistic the haptic-driven sensation is, without higher quality visuals, a disconnect will still plague many VR users. None of the haptic innovations currently in development seem ready to resolve the problem of VR sickness, or capable of preventing latency issues.
Why hand tracking systems are superior to haptics
What makes hand tracking superior to haptic? Hand tracking systems may not have the warm fuzzies of haptics; controller and glove-free tracking aren’t designed for touchy-feely experiences. However, when it comes to visual consistency and rapid refresh rates, hand tracking is poised to be a game-changer.
The Oculus blog describes the latest Quest hand tracking VR technology as a way to “unlock new mechanics for VR developers and creators alike.” By installing hand tracking on Quest, Facebook hopes to encourage greater expression in VR experiences. People who aren’t familiar with conventional gaming controllers can also benefit since they won’t need to learn how to manage a particular controller system.
Being able to immediately achieve immersion without having to locate and implement gloves or controllers is another plus for hand tracking. This increases potential cross-industry collaboration in education, entertainment, and corporate use.
Hand tracking technology will launch on Quest, the standalone, tetherless VR headset, in early 2020. It will be available only as an experimental feature for consumers at first (as well as an SDK for developers.) It’s hoped that VR devs will jump into creating and deploying new experiences based on the controller-free, hand-tracking software options, and quickly rolling out for Quest users who opt-in to hand tracking.
The original research project at Facebook Reality Labs has been developed into a deep learning tool that can understand the position of fingers based on feedback from the Quest monochrome cameras. There is no need for extra processors, additional sensors, or active depth-sensing cameras. The hand tracking technology is AI-powered and approximates hand shape, creating a set of 3D points to precisely track hand and finger movement in VR.
This provides a fully articulated hand tracking system without the need for additional equipment, including hand controllers, instrumented gloves, or haptic devices. Processing operates on-device, with system optimization for various recognized interactions.
Deep neural networks predict hand locations and landmarks, such as joints. A 26 degree-of-freedom pose is reconstructed, and the resulting 3D model (which includes surface geometry and configuration) is used to create smooth interactions.
The jitter problem currently troubling haptic controllers can be alleviated by developers using the 3D models to drive user interfaces. The tracking architecture produces accurate, low-jitter estimates of hand poses, and the neural network framework encourages real-time hand tracking on a mobile processor.
Display of hand motion is consistent over a wide range of environments, and resources dedicated to user applications aren’t needed to maintain the quality of visuals. This core technology could very well be a gamechanger for VR.
Potential uses for hand tracking technology in VR
Two scenarios that have been demoed so far include fantasy and real-world simulations. In the first, users enjoyed a trip into a wizarding realm, where they could poke a dragon’s belly, scoop eyeballs from a treasure chest, and watch their hands turn into tentacles complete with suckers.
The second scenario was considerably more pedestrian, but certainly more useful; Farmers Insurance promoted a VR experience that portrayed an insurance claims adjuster walkthrough. Users had to control the simulation to accurately identify and tag leaks and other issues in a kitchen overrun with water damage.
Other potential uses include social interactions, immersive training scenarios, and more. While haptics can provide exciting sensational interactions, the improved visual quality coupled with the elimination of cybersickness appears to be VR’s current trajectory – especially with the increasing potential for VR use in the business world.
“So, can we have it?”
As a leading provider of VR Training software, PIXO VR is routinely asked about the availability of haptic gloves for a more immersive experience in our simulations. We’re also frequently approached and pitched by various haptics innovators on their latest products and prototypes. The (virtual) reality is, while we’ve seen some interesting developments in haptics in recent years, we’re yet to find gloves or other haptic solutions that we’re able to recommend to our customers.
The “perfect” haptic glove may very well be in development right now in some company’s R&D lab, but for the time being, we just can’t get behind one. This is one reason why we’re both excited and interested to see what Oculus and others can offer with regard to hand tracking, which may provide a workable alternative. One thing’s for sure: whenever new hand tracking and/or haptic gloves are ready to hit the market, we’ll have the scoop right here on the PIXO VR blog.