The Oculus Quest VR headset uses directed speakers, which are a convenient one-size-fits-all solution for projecting sound into an environment, but leak sound to bystanders — while in-ear monitors may be better for changing overall sound levels while wearing AR glasses in public. “What form factor to use to solve a problem ultimately depends on the application,” says FRL Research audio lead Ravish Mehra.
As Facebook acknowledges, the lab’s “perceptual superpowers” pitch is very similar to the function of existing hearing aids, which also amplify sound and reduce background noise. (One experimental system even uses brain implants to focus on specific voices.
And any spatial audio improvements could soon be applied to VR headsets, including a rumored Oculus Quest update around this month’s Facebook Connect conference. “The work that we are doing and the work that we presented today definitely has applications for our VR line of devices,” says Mehra.
AI audio analysis presents even weirder and more alarming possibilities, like the ability to flag specific voices or conversation keywords in a crowded room. (It could also theoretically perform more innocuous, helpful tasks like real-time translation. ) And, of course, the glasses would be recording your own conversations as well.
Among other things, the system could ask for permission from someone else’s glasses before amplifying a conversation, or it could have a limited range. “Rather than thinking of this as a magic flashlight we can point at anybody and hear what they’re saying, think about it more as people being able to participate in the conversation they’re in anyway,” he told reporters.
Chief scientist Michael Abrash and his team at FRL Research (formerly Facebook Reality Labs) released details today about what the team calls “perceptual superpowers” — AR systems that figure out what you’re trying to hear, then amplify it and dampen background noise.
Conversely, if you’re on a phone or video call, improved spatial sound could project participants’ voices or other audio to specific parts of the room, increasing the sense that you’re really with somebody else — or “audio presence,” in FRL Research’s terms.