Apple iOS: Developers are Showcasing New Augmented Reality Features
Apple’s iOS 13 update is already out here and Apple is also releasing the version 3 of its augmented reality software ARKit. Many developers are beginning to show off the magical augmented reality effects that they can be created with it.
The key features of the ARKit 3 are the human motion detection feature and the masking effect which makes a digital object disappear behind a real person or vice versa thereby causing digital objects to either obscure or bounce off a real object.
— プログラミング少女公式@YouTube🎥動画も配信中⚡️ (@programming_sj) October 1, 2019
This allows developers to create highly realistic augmented reality effects. Smartphone AR effects are rapidly transforming smartphone cameras into a kind of special effects studio. They began as technically more convincing face masks but they are quickly moving into effects that cover the whole body.
— ふしっきー (@fusikky) September 30, 2019
There is a Black Mirror episode known as ‘White Christmas’ where users wearing some augmented reality implants are able to visually and acoustically block other people thereby creating some kind of real-life but virtually enabled block button in real life settings.
That “human blocker” has also been partially implemented in ARKit 3. The only missing piece that would transform this dystopia into reality is the identification of the person via a facial scan along with AR glasses with a wider field of view. The same technology coupled with full-body costumes for entertainment could also be applicable in the Marvel series.
— CHASUKE (@chasuke__) September 30, 2019
ARKit is capable of capturing three persons and then separating them from the background so it is possible to have green screen effects without necessarily having a green screen. There are already some effects which are possible on ARKit 3 that illustrate this.
— Dilmer Valecillos (@Dilmerv) September 18, 2019
A great example is through the new Meisai AR Filter app that can be freely downloaded from the Apple app store. The app was developed by the Japanese studio Kitasenju Design and can produce some stupendous visual displays.
— Tomoki Itamiya 板宮 朋基 (@t_itamiya) September 30, 2019
The Meisai app uses Motion Capture and People Occlusion features in the ARKit 3 to cover the hands of the users with real-time digital patterns and animations. Some of these effects deliver an almost psychedelic experience.
The Meisai app is also able to segment backgrounds and replace them with virtual backdrops to provide users with a green screen setup. It can also create trippy scenes for viewers by displaying the digital clones of the users’ appendages as seen below.
The is app is freely available on the App Store but it can only run on ARKit 3-comptible iPhones and iPads.
Beyond just providing some fun and good time, the app also demonstrates how advanced Apple’s ARKit is when paired with the most recent iPhone and iPad hardware. It certainly outperforms Facebook’s Spark AR and Google’s ARCore.
The improvements in hand and finger recognition allows users to precisely draw their fingers in the air. The body capture and concealment functionality however works only iPhones and iPads that are built with the latest A12 mobile processors such as the iPhone XS, iPhone XS Max, iPhone XR and 11 as well as the 12.9-inch iPad Pro.