iOS 13 Developer File Leak Confirms the Existence of the Apple XR Glasses
Although it is not official yet, an iOS 13 documentation now confirms the existence of Apple’s augmented reality glasses with the codename ‘Garta’.
It now seems apparent that Apple is working on releasing iPhone-dependent (ARKit-compatible) augmented reality glasses that will come with a handheld controller. The discovery was made by the iOS developers Guilherme Rambo and Steve Troughton-Smith in newly released iOS 13 and XCode 11 versions.
The iOS 13 GM also comes with a readme file (!) for how employees can run Stereo AR apps on an iPhone when you don't have access to Apple's headset 😳 pic.twitter.com/SeZEHW8p0S
— Steve Troughton-Smith (@stroughtonsmith) September 10, 2019
Troughton has stated that these glasses will be more of passive display with the computing work being done by an ARKit-compatible iPhone.
The picture of Apple’s AR efforts from iOS 13 is very different to what one might expect. It points to the headset being a much more passive display accessory for iPhone than a device with an OS of its own. The iPhone seems to do everything; ARKit is the compositor
— Steve Troughton-Smith (@stroughtonsmith) September 11, 2019
The release has provided a clear evidence of an augmented reality Apple system software for AR apps in the stereo mode known as Starboard, a highly relevant piece of software which is a prelude to the release of compatible hardware. The apps can subsequently be viewed through a dual camera or in a pair of augmented reality glasses. The Starboard app will also enable users to switch between the hand-held and the head-worn modes.
StarBoard frameworks on iOS 13 now. StarBoard is Apple’s system shell for stereo augmented reality apps (headset). Guess secrecy is out? pic.twitter.com/XTnlqQgpHh
— Steve Troughton-Smith (@stroughtonsmith) September 10, 2019
The Starboard currently looks like iOS’s CarPlay feature which leverages the iPhone to produce graphics for a vehicle’s external touch display. While the CarPlay feature generates a single image for the 2D dashboard display, the Starboard feature mainly outputs two images for a 3D stereoscopic headset which will be paired with a new touch controller for augmented reality navigation.
The code suggests that the user’s hands will grasp something similar to an Oculus Go controller or a Siri Remote thereby combining the touch surface and the motion controls with the Home button and a trigger.
Apple’s code includes references to Holokit, Franc, Luck and Garta which seem to refer to the company’s new augmented reality software and hardware that is currently being tested internally. On the other hand, it seems that Apple is likely to have developers create both objects and environments within its current ARKit software environment.
Troughton-Smith has speculated that Apple could be priming a third-party accessory program for introducing compatible glasses onto the market alongside its own offering as a “halo” product.
The route Apple has taken appears to be the same as that of the Qualcomm XR Smart Viewer initiative that relies on the Snapdragon 855-powered smartphones in processing both visuals and data for lightweight and screen-laden glasses. The Qualcomm initiative was unveiled in February this year, soon after Nreal demo’d its Nreal Light augmented reality glasses at the CES which run on the previous generation Snapdragon 845 processors.
The Nreal Light glasses are priced at $500 and have a 52-degree field of view along with a bright and colorful 1080p high-resolution display which makes the digital objects seem like they are floating in the user’s field of view. The Nreal glasses also have twin cameras and a SLAM technology allow the glasses’ inside-out tracking system to display the objects in the real-world environments and accurately adjusts their position as well as perspective as the user moves around them. More improvements in the display resolution and graphics chips will make it possible to render even more complex objects to the point of photorealism.
The major challenges for Apple is how to provide users with accessory that will have the form factor along with the right pricing that will resonate with the mainstream market, beyond just the enterprise users. Both Microsoft and Magic Leap have so far released augmented reality/mixed reality headsets that cost thousands of dollars and which require significant computing power. The Microsoft AR device is anchored on user’s heads while the Magic Leap glasses are tethered to the waist. However, none of these platforms have so far proved popular with the mainstream end users.
It is not yet clear whether the Apple device will stream data to the headset wirelessly while still ensuring sufficient battery power to allow the device to be usable for several hours at a time. Apple’s new iPhone 11 and the iPhone 11 Pro both have a Wi-Fi 6 as well as an ultra-wideband wireless chip known as U1. This feature could make it possible for Microsoft to stream very high-bandwidth wireless transmissions from an iPhone to the headset instead of having to rely on a cable.
There will also be the issue of the operating software though Apple’s decision to leverage iPhones as the initial foundation for its augmented reality headset could be a clever workaround. Apple could also tap into developers to code the iOS apps which will initially run on iPhones before spinning these off into apps capable of running on more powerful future augmented reality headsets or over 5G from the cloud servers. It was also rumored that Apple could be developing a custom variant of an iOS known as “rOS” for its wearable augmented reality devices which could be launched in the future if the technology permits, should 5-nanometer chip manufacturing shrink it’s A-series or S-series processors to the frame sizes of the glasses.
The reveal date for Apple’s augmented reality glasses is still not apparent. A report emerged in July stating that Apple had killed off the project followed by another which claimed that the tech giant was mobilizing its internal resources to finish the AR project in readiness for a 2020 launch. The conflicting reports may have been an indication that Apple had killed off its initial ambitious standalone device for a more practical and tethered one that it could ship soon.