Microsoft Showcases New HoloLens Hologram Capable of Language Translations
Microsoft keynote showcases language translation and holoportation, giving event-goers a window into the future.
It is now possible to have an almost life-like and hyper-realistic holographic replica of yourself but Microsoft has taken it a notch higher by having these holographic replicas speak in multiple languages.
Microsoft’s Japanese-speaking hologram was one of the key highlights during the Inspire 2019 conference. Microsoft demo’d the latest innovation via its HoloLens 2 Mixed Reality headset in its keynote speech at the Microsoft Inspire partner conference which was held in Las Vegas. Azure Corporate Vice President Julia White used Microsoft’s HoloLens 2 headset to generate a hologram that presented her keynote speech in Japanese in her own voice.
The Windows Mixed Reality headset captured and rendered the hologram, scaling it up to a life-sized hologram that resembled White. The hologram demonstration also leveraged Azure services in translating White’s keynote speech into Japanese while still maintaining her unique voice’s inflections and tone. Apart from the body and voice capture, the project also relied on the Azure AI as well as the HoloLens 2 technology in transforming the real human presenter into a 3D hologram that was capable of delivering a presentation in any language.
The keynote begins with White wearing the Microsoft HoloLens 2 mixed reality headset and then sticking out her open hand for it to be seen by the mixed reality headset. At this point, a miniaturized green model appeared out of nowhere.
White demonstrated how the tiny hologram was capable of tracking and following her hand movements. The presenter also issued the simple command “Japanese keynote. Render keynote” which prompted the tiny green model to disintegrate into a cloud of particles. These subsequently realigned to form an almost photorealistic holographic rendition of Julia White.
This new Hologram would subsequently present the rest of the keynote in Japanese with almost lifelike gestures and facial expressions. Microsoft utilized its new Neural Text-To-Speech (Neural TTS) technology that taps into the power of artificial intelligence to automatically generate personal voice signatures which are based on simple voice samples. Apart from the Neural TTS, the demonstration also utilized the Azure speech-to-text and the Azure Speech Translation. The English-to-Japanese translation was rendered in an artificial voice which had been synthesized to sound like White’s voice actually speaking Japanese. The text-to-speech service is currently available in over 45 languages for standard voices. The service can also be used in creating neural voices with the custom neural voices synthesizing various units of speech to generate the computer voice.
Microsoft’s pairing of mixed reality with the Neural TTS technology could potentially transform how clients, businesses as well as various other professionals in diverse industries deliver live presentations like talks, conferences and lectures. The technology could help professionals create engaging and personable presentations in different languages simultaneously across the world.
However, this technology is still in the early stages just like the HoloLens 2 Mixed Reality headset so the procedure is still extremely costly. Microsoft shot Julie White’s hologram in conditions which are near-perfect inside Microsoft’s dedicated mixed reality studio which is equipped with various high-tech and high-grade equipment setup like professional lighting and camera equipment designed for mixed reality capture.
The HoloLens hologram demonstration paints an interesting picture on possible future mixed reality applications in areas such as live talks, Q&As, lectures and various other kinds of live presentations.
This is not the first time that Microsoft is fiddling with holoportation. The tech software giant has created a similar project in the past. Holoportation takes a complete 3D model of a person and places them inside a mixed reality environment. The technology can be used to teleport people to conferences and events, making them appear as if they are actually there even though they might be thousands of miles away.
Check out the Microsoft Inspire website to watch the HoloLens Hologram demonstration. The holoportation and translation demonstration can be watched from the 32nd minute on the video. It’s quite impressive that the demonstration has been accomplished with technologies that are already in existence so it is not something that we might have to wait for decades to experience.