FingerTrak is a new type of tracking system for hands and fingers. It is in the form a thermal camera bracelet which reads the wearer’s wrist to track fingers.

Both virtual reality and augmented reality have made tremendous strides in comfort and visual fidelity and researchers are continuing to develop new input solutions that should give users a more natural feel than the conventional controllers.

FingerTrak Thermal Tracking Wristband Prototype
FingerTrak

Last week, researchers announced a new wristband-based 3D hand sensing solution that relies on thermal cameras to track the hand movements in 3D by abstracting 20 finger joint positions from the contours on the wrist of the wearer. By using thermal imaging instead of optical tracking, the FingerTrak devices aim to revolutionize both hand and finger tracking for Augmented Reality, Virtual Reality and other purposes.

The current virtual reality input options are limited to hand tracking and virtual reality controllers. Headsets like Oculus Quest and HoloLens 2 use optical tracking and hardware add-ons such as LeapMotion.

At the moment, the trend in VR controllers tilts towards handy input devices which are attached at the back of the wearer’s hands and which allow users to have more or less sophisticated finger tracking through sensors. The Valve Index controllers have been the flagship controllers in this area. The Oculus Touch controllers also tend to be handy but you still need some Oculus Quest accessories to attach the controllers to your hand.

While VR hand tracking has shown lots of potential, it still faces some problems and limitations.

It has shown great potential in the way that it has made it possible for users to control some menus, apps and games. This is best displayed in the Oculus Quest hand tracking functionality. In this case, both the hands and fingers are optically tracked with the use of cameras, a novel VR tracking solution which has great potential for the future.

However, problems still bound. It can’t, for example, display overlapping hands and the tracking will only work within a very narrow range. Besides, this tracking is not very accurate.

The FingerTrak Thermal Imaging Tracking Solution

FingerTrak is a new type of tracking system that has been developed by the researchers from University of Wisconsin and Cornell University.

It provides finger tracking by using thermal imaging cameras to track images in 3D. These are more precise than the optical tracking cameras.

The FingerTrak tracking solution was developed by researchers at the SciFi Lab at the Cornell University with the help of researchers from the University of Wisconsin and Madison. FingerTrak leverages deep neural networks in stitching together the input emanating from some three or four miniature thermal cameras that are mounted around the wearer’s wrist to collective capture the whole hand pose.

The backbone and the regression networks in the deep neural networks use the silhouettes that have been generated by the cameras to estimate the fingertip and joint positions. So far, the results are not 100% perfect but already, the technology can be deployed in certain forms of Virtual Reality and Augmented Reality input.

FingerTrak could also be put into other potential applications such as human-robot interactions and in control, sign language translation, mobile health use-cases such as in the early detection of degenerative diseases like Alzheimer’s and Parkinson’s.

According to the researchers, the wrist contours alone in the FingerTrak are sufficient to “accurately predict the entire hand posture” which will allow for the entire sensing system to be put on the wrist instead of having to rely on rings, gloves and various other techniques which have been used in other haptic implementations so far.

A demo video exists which shows the FingerTrak hand motion tracking translate into movements for a bionic hand while also allowing a computer to determine when a user is performing activities such as drinking coffee, writing or using a smartphone.

It is still not apparent whether FingerTrak’s thermal imaging is capable of rapidly tracking specific gestures like what is being written by someone. According to the researchers, FingerTrak has an average angular error rate that varies from 6.46 to 8.06 degrees at the time of testing, depending on the background that it is tested against. This means that at best, it might supplement rather than completely replace the higher accuracy finger tracking solutions currently in existence, at least in certain use-cases.

The use of wrist-mounted cameras could also help augment the existing inside-out tracking cameras already in use in other virtual reality headsets such as the Oculus Quest which monitor hand and finger tracking positions right from the head. The FingerTrak prototype already looks very small and could get even smaller with additional engineering.

At the moment, it leverages very low-resolution thermal cameras of just 32 by 24 pixels. These cameras are tiny, pea-sized, and they are also thinner than Apple’s thinnest Watch but it still needs to be mounted on multiple areas on the wristband. The FingerTrak prototype also uses a tethered Raspberry Pi board. However, this hardware can be alternately controlled via another miniaturized solution like a smartwatch. The later versions could be connected with a smartphone or directly with virtual reality headset or Augmented Reality glasses, possibly even wirelessly.

In spite of its novel functionality, FingerTrak is still in very early research. Any implementation for the consumer market is still a long way off. Questions still abound on the extent to which this device is a viable controller replacement for VR and AR use-cases. What will happen, for example, when the user’s two hands are too close together or even overlap? How will this system keep the fingers and hands apart? Can its precision be improved considerably? The researchers still have to grapple with a number of questions in order to make this product viable and marketable.

The device will be presented in mid-September this year at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing. You can click here to read the research paper on this groundbreaking hand tracking innovation.

Source: VentureBeat

https://virtualrealitytimes.com/wp-content/uploads/2020/07/FingerTrak-Thermal-Tracking-Wristband-Prototype-600x337.jpghttps://virtualrealitytimes.com/wp-content/uploads/2020/07/FingerTrak-Thermal-Tracking-Wristband-Prototype-150x90.jpgSam OchanjiTechnologyFingerTrak is a new type of tracking system for hands and fingers. It is in the form a thermal camera bracelet which reads the wearer's wrist to track fingers. Both virtual reality and augmented reality have made tremendous strides in comfort and visual fidelity and researchers are continuing to develop...VR, Oculus Rift, and Metaverse News - Cryptocurrency, Adult, Sex, Porn, XXX