Apple Patents Finger Tracking for Smart Glasses
Apple’s first tech glasses are expected to be launched in 2022 and 2023. The tech giant has now filed a patent with the U.S. Patent and Trademark Office (USPTO) for a headset finger accessory which will be used in controlling content through 3D air gestures, force sensing and much more.
The patent is titled “Computer Systems With Finger Devices” and it shows a finger-worn device built with a touch sensor which can be used in controlling objects on a Virtual Reality or Mixed Reality screen.
The device also has a haptic feedback that enables users to feel. The patent application talks of this technology being used in the future together with a head-mounted augmented reality (AR) device. This could be the mixed reality headset that Apple is reportedly building and which is set for release in 2022 with Apple’s long-awaited Glasses also slated for release in the following year (2023). The device could feature sensors such as force sensors, inertial measurement units, ultrasonic sensors, optical sensors as well as other components.
The sketches accompanying the patent application show a finger device whose size would be roughly that of a fingernail. Another image in the patent shows how a user could change the volume on a HomePod smart speaker simply by turning a virtual volume control with the finger-worn device.
It appears that Apple is concentrating on hands and fingers for Apple’s upcoming tech glasses slated to be released in 2022 and 2023. Since the start of 2019, Apple has patented five technologies on hand and finger tracking for virtual reality headsets, Mixed Reality glasses and Augmented Reality glasses.
The latest Apple patent published by the USPTO secures the technology for three-dimensional gesture recognition in the air as well as a finger attachment as a ring or cap that could provide haptic feedback to match digital interaction. This allows the user to know whether a digital element is touched or selected, for example.
The finger attachment working in concert with the tracking cameras in the headsets/smart glasses or magnetic sensors should be able to detect finger movements in the air. It can also be used in a pressure-sensitive fashion. It is a tiny device. From illustrations on the patent application, it appears to be the size of a fingernail.
The past one year has been a busy one for Apple with numerous patents related to finger devices rolling out from innovation labs. These have been designed to be compatible with Apple’s future mixed reality head-mounted display (HMD). The patent published by the USPTO this week is Apple’s fifth patent application relating to finger tracking devices which could be used in the future in manipulating content on an HMD’s display. These will replace the use of specialized haptic gloves which with the current technology, are still clumsy to use and wear.
The Apple Patent
The finger devices shown in the latest Apple patent can detect hand motions like the 3D in-air hand gestures. They can also offer haptic responses which will make the gameplay in immersive games more realistic. The design of the finger devices will be made of fabric materials.
The latest Apple patent for finger tracking is of a system which may have one or more finger-mounted electronic devices. These could be finger devices that have U-shaped housings that have been configured to be mounted on the fingers of the user when gathering the sensor input or providing the haptic output.
Each of the finger-mounted devices might have a body. The body will form part of the support structure for the components in the finger-mounted devices like the force sensors, accelerometers as well as various other sensors. The body could also be used for haptic output devices.
When using them, a user may wear the finger-mounted devices on their fingertips when they are interacting with the external objects.
The finger-mounted electronic device may have a body with a first and second side body members which leave the finger pad exposed with the upper body member that extends between the first and second side body members.
Some or the entire body may be covered with leather or fabric. This fabric could wrap around the first as well as the second body members and could extend to the upper body member. This fabric may encase the electronic components in the finger-mounted device. The touch sensor in the device may have electrodes made from a conductive material that will be either on the fabric or the fabric’s conductive strands.
An infrared reflective ink may be used to form the visual markers on the fabric to provide an infrared tracking system. The fabric could have light-transmissive parts overlapping the optical components. Additionally, the fabric used may also have breathability features like bumps, pits and through-holes.
This fabric may attach to the body via wrapping techniques, adhesive attachment techniques, intertwining, molding attachment techniques or other suitable methods.
Apple’s Augmented Reality Glasses are Inconceivable without Finger Tracking
Should Apple launch its first tech glasses in the next two or three years, finger tracking should be expected. Our own hands are, no doubt, the most intuitive input devices for XR and because our hands are always there, they make for the perfect input device.
It’s the hands, the ability to use hands as an input device on a smartphone touchscreen, that led to the smartphone revolution over the past decade or so. Apple shaped this further with multitouch for iPhone. The company is therefore endeavoring to repeat this success in XR. Apple has apparently already developed a ‘Reality Operating System’ that is designed for the XR era that can be safely operated with hands.
Apple CEO Tim Cook has confirmed once again that the tech giant will be putting a lot of effort into augmented reality. During a tech event in Ireland, Tim described augmented reality as “the next big thing”.
Facebook Already Offering Hand and Finger Tracking for XR
Facebook is already providing finger tracking for its Oculus Quest VR headset. Quest’s finger-tracking functionality was launched in December 2019 without an additional control device on the finger. The haptic feedback as described in the Apple patents is missing in Facebook’s finger tracking functionality.
In April 2018, Apple is said to have tried purchasing Leap Motion, a company that specializes in finger tracking using camera. Apple was reportedly interested in Leap Motion’s staff and its approximately 100 patents relating to Computer Vision. Leap Motion eventually merged with the haptics company Ultrahaptics.
Source: Patently Apple
https://virtualrealitytimes.com/2020/01/26/apple-patents-finger-tracking-for-smart-glasses/https://virtualrealitytimes.com/wp-content/uploads/2019/07/Apple-AR-Glasses-600x382.jpghttps://virtualrealitytimes.com/wp-content/uploads/2019/07/Apple-AR-Glasses-150x90.jpgInventionsTechnologyApple’s first tech glasses are expected to be launched in 2022 and 2023. The tech giant has now filed a patent with the U.S. Patent and Trademark Office (USPTO) for a headset finger accessory which will be used in controlling content through 3D air gestures, force sensing and much...Sam OchanjiSam Ochanji[email protected]EditorVirtual Reality Times - Metaverse & VR