Australia’s Opera Queensland worked with Google Creative Lab KI in an augmented reality and AI-powered demo project that brings the stage experience of the opera to the four walls of your home using augmented reality and artificial intelligence.

With the prototype AR app, opera fans no longer have to line up or even travel to the theater to enjoy an opera production. The project will allow opera enthusiasts to turn their homes into a theater stage and experience the opera via augmented reality.

Opera enthusiasts will now enjoy performances from the comfort of their homes
Opera enthusiasts will now enjoy performances from the comfort of their homes

The AR experimental app, dubbed Project AR-ia, was created with a version of The Magic Flute that’s performed by the Australian opera singers Emma Pearson, Brenton Spiteri and Wade Kernot and the performance was accompanied by the Brisbane’s Chamber Orchestra, Camerata.

Opera Queensland hopes to use the technology to not only demystify opera and make it more accessible, thereby bringing it to newer audiences. The Project AR-ia experimental augmented reality app shows how the opera art can adapt with the times and be enjoyed by audiences anywhere and at any time.

Performer Brenton Spiteri
Performer Brenton Spiteri as captured on Project AR-ia

The app engages audiences in various ways and revolutionizes how we experience the music form. While attending the opera has entailed dressing up and travelling with loved ones to the theater, future opera enthusiasts will be able to simply watch the performances from the comfort of their homes with AR avatars performing for them.

According to the Google project lead Jonathan Richards, Project AR-ia offers a glimpse of how some of the traditional entertainment might look like in a few years’ time. Google also hopes that by bringing the audio-visual art into AR via its ARCore platform, the tech would inspire a new generation of storytellers to explore the possibilities of working with augmented reality.

Opera Queensland approached Google in early 2018 in a bid to explore the creation of diverse opera experiences through new technology. The opera wanted to give all enthusiasts a front-row seat in the opera from the comfort of their homes. To realize this, the performances of the opera singers have been completely digitized in 3D as a first step. The technology leverages Google’s AI-based volumetric recording system known as The Relightables which was unveiled recently.

The images of the opera actors were captured using 90 high-resolution cameras and numerous depth sensors from multiple perspectives. An algorithm subsequently created an animated 3D model of the person captured from the many individual images.

The digital person or avatar created is then projected into the user’s real environment through Google’s smartphone AR software. This is how the user is able to get a private screening of Mozart’s Magic Flute in their living rooms.

The technical challenge faced by the developer was that of the high data outlay. According to the developer, just 20 seconds of the spatial film material consumed more than a terabyte of data which had to be computed and compressed even further.

Even after the data compression, one second of the volumetric opera still required about five megabytes. Three opera singers that are animated at 30 frames per second required about 450 megabytes of data per second. This is equivalent to about 4.5 gigabytes for every ten seconds. This roughly corresponds to the data volume of a complete high-resolution 2D film.

This massive data throughput required for the streaming of the opera must played a role in the developers not releasing the prototype augmented reality opera. The project itself began as a storytelling experiment for a new kind of experience exploring how to make art accessible to a greater number of people. The work on the AR formats is set to continue as the demo app is at the forefront of creative technology and will require significant investment and development over a duration of time that will help propel it beyond the prototype stage.

https://virtualrealitytimes.com/wp-content/uploads/2019/11/Performer-Brenton-Spiteri-600x337.pnghttps://virtualrealitytimes.com/wp-content/uploads/2019/11/Performer-Brenton-Spiteri-150x90.pngSam OchanjiGamingMusicTechnologyAustralia’s Opera Queensland worked with Google Creative Lab KI in an augmented reality and AI-powered demo project that brings the stage experience of the opera to the four walls of your home using augmented reality and artificial intelligence. With the prototype AR app, opera fans no longer have to line...VR, Oculus Rift, and Metaverse News - Cryptocurrency, Adult, Sex, Porn, XXX