Research Highlights

FIND THE DIRECTION OF SOUNDS IN 3D IN REAL-TIME

Figure 1Using the world’s smallest microphone array, researchers at the Advanced Digital Sciences Center (ADSC) have demonstrated the ability to determine the direction of origin of a sound source in three dimensions, in real time. Whereas state-of-the-art techniques use large microphone arrays for good accuracy and have difficulty achieving real-time performance, ADSC’s approach determines audio direction to within a 2°×2° spherical surface area in real-time with high accuracy, which is a leap forward in the state of the art in acoustic localization.

This work is part of ADSC’s broader program to provide low-cost, high-quality, real-time telepresence for activities such as virtual meetings. Realistic teleimmersion must respect the fact that human hearing is extremely sensitive to the perceived direction of arrival of sounds. Differences in the expected and perceived directions of arrival, so that the voice of the colleague on one’s left in a virtual meeting seems to come from the right, will break the illusion of being in the same room. Realistic teleimmersion requires clean capture of acoustic sources, accurate estimation of their direction of arrival, and appropriate reconstruction of their virtual direction for each user. While these problems have been extensively studied with arrays of physically separated microphones, their large size limits the practical uses of these systems. State-of-the-art algorithms for small microphone arrays have limited ability to perform speech enhancement, 3D audio direction finding, and reconstruction.

ADSC has demonstrated 2D and 3D direction finding for the miniature microphone array pictured above, using an ordinary laptop, a commodity graphics card, and a general-purpose graphics processing unit. The array uses three directional and one omni-directional microphone. In addition to running in real-time, the new hardware and software are simultaneously smaller in size, lower power, less expensive, more accurate, and more robust than conventional approaches based on a much larger microphone array. ADSC’s algorithms leverage the increased acoustic information obtainable from a miniature directional array, and employ a wide variety of signal processing techniques not feasible in traditional, physically separated microphone arrays. While conventional approaches rely on the time difference of each microphone pair to determine the direction of a sound, ADSC’s approach employs the amplitude difference between the microphones.

figure 2To further improve the user’s sense of teleimmersion, ADSC has developed a novel 3D audio capture and reconstruction algorithm for stereo headphones, based on the miniature microphone array. In subjective testing, a wide variety of users reported good localization accuracy.

View a detailed PDF of this research.