Adobe Sets its Sights on Augmented Reality and Mixed Reality Software
Designers will by the end of this year receive three new Adobe apps all of which represent the company’s next vision for a new era of creativity. Unlike its past apps, the new apps are focused on blending the real life and physical elements with digital elements.
Adobe CTO Abhay Parasnis says that these new generation apps emphasize creative experiences that will be “out of the glass” (without virtual reality glasses or headsets). These new apps will also be interconnected via mobile and desktop experiences thereby enabling users to switch experiences between a multiplicity of devices.
Adobe says that all the apps that it is developing-be they Fresco, Aero or Photoshop on iPad-will see the company push forward toward a cloud native user experience thereby simplifying collaboration in projects.
This is best exemplified by Adobe’s Project Glasswing, a mixed reality display exploration that is currently under development. This experimental display prototype explores the possibility of bringing all of Adobe’s design apps and tools into the physical world through either Photoshop, After Effects or XD layers that appear in the transparent space in front of real 3D objects, without requiring smartphones or special glasses.
Project Glasswing is a touch-based experiment that will enable Adobe to showcase new experiences which will blend the digital content and interactions with the real and physical objects thereby creating new possibilities for unlocking a 3D ecosystem around creative storytelling in the future.
The goal for Adobe is to develop apps that “feel the same” within the paradigm of the cloud documents and in the UI. This will create a user experience where the files “feel local” even if the work is being performed in the cloud, instantly syncing and saving all the changes made in the actual work.
— Howard Pinsky (@Pinsky) June 19, 2019
Adobe’s Fresco, for example, mimics how oil paints and watercolors behave on paper and has been developed after eight years of research that leveraged the Adobe Sensei artificial intelligence platform. The first version of this app needed a high-end graphical processing unit (GPU) to run the engines. Much of the research time concentrated on ensuring that the realistic strokes wouldn’t be a drain on the performance and the battery life.
The Adobe research team worked hard to cut down the latency and ensure that the paints and brushes mixed naturally in real-time. This was a major breakthrough in the development of the application.
While Fresco was developed to bring the analog painting experience into the digital painting environment, another of Adobe’s projects, the Project Aero aims to realize just the opposite of this.
Project Aero is an augmented reality authoring tool that empowers designers to create immersive experiences by using the Adobe tools that they are already familiar with without having to learn coding skills. What Aero aims to achieve is take digital creations from Adobe applications such as Illustrator, Dimension, InDesign and Photoshop and seamlessly overlay these into the physical 3D spaces around us.
Aero was showcased during Apple’s developer conference which was held in 2018. During the conference, Adobe had announced that the tool would work easily with the augmented reality file format that was developed by Apple and Pixar. This unified file format is known as USDZ and is currently restricted only to iOS so it allows the files to easily be exported into iPhones or regular apps such as Safari and iMessage without the need to download a separate AR viewer.
The Aero app also enables artists to simply use drag-and-drop modules to create responsive augmented reality experiences and animations which can automatically factor in variables such as lighting and physics, a task that would ordinarily need familiarity with programs such as ARKit and Unity.
Aero also lets artists fine-tune their augmented reality creations through the use of 3D textures and materials derived from Substance, an Allegorithmic texturing software that was recently acquired by Adobe. There are a few artists who already have a closed beta access to Aero. However, general users can sign up for early access here.
Despite Apple’s strong push towards AR, the software giant still understands the limits of augmented reality which forces users to peer through screens and requires that the devices look at the real-world environment.
Adobe also appreciates that both the hardware and software part has to be fine-tuned for augmented reality and immersive experiences to go mainstream and justify any significant investment. Adobe is hoping to realize that through Project Glasswing, the transparent LCD display which could find wide applications in areas such as hotels, museums or retail environments.
The magic with the Adobe solution is its capability to blend both the physical and the digital environments without the need for mixed reality hardware. That could potentially be a game-changer in the race to go mainstream, particularly in the design or creative applications where the glasses and headsets are more of hindrances than useful aids. Adobe is thinking of it as an alternative to augmented reality.
Adobe’s Project Glasswing is not necessarily a hardware product. Rather, it provides a means of showcasing how Adobe’s software solutions can be applied in real-life. Adobe is planning to go into partnership with various other companies that will push the hardware forward because the software giant isn’t interested in building the displays.