This paper reviews pioneering works in microphone array processing and multichannel speech enhancement, highlighting historical achievements, technological evolution, commercialization aspects, and key challenges. It provides valuable insights into the progression and future direction of these areas. The paper examines foundational developments in microphone array design and optimization, showcasing innovations that improved sound acquisition and enhanced speech intelligibility in noisy and reverberant environments. It then introduces recent advancements and cutting-edge research in the field, particularly the integration of deep learning techniques such as all-neural beamformers. The paper also explores critical applications, discussing their evolution and current state-of-the-art technologies that significantly impact user experience. Finally, the paper outlines future research directions, identifying challenges and potential solutions that could drive further innovation in these fields. By providing a comprehensive overview and forward-looking perspective, this paper aims to inspire ongoing research and contribute to the sustained growth and development of microphone arrays and multichannel speech enhancement.

An adaptive second-order differential microphone design is proposed here that is constructed from a weighted sum of omnidirectional microphones. Theoretically, only three microphones are required to form a second-order array. The three microphone signals are combined to form three unique fixed second-order beams. Any second-order differential beampattern can be realized using a weighted sum of these three ldquobuilding-blockrdquo beam outputs. If certain simple constraints are placed on the weighting of the three fixed beams, the two null locations that define the final second-order beampattern can be constrained to defined angular regions.

With the proliferation of inexpensive digital signal processors and high-quality audio codecs, microphone arrays and associated signal processing algorithms are becoming more attractive as a solution to improve audio communication quality. For room audio conferencing, a circular array using modal beamforming is potentially attractive since it allows a single or multiple beams to be steered to any angle in the plane of the array while maintaining a desired beampattern. One potential problem with circular arrays is that they do not allow the designer to have control of the spatial response of the array in directions that are normal to the array. In this paper we propose an augmented circular microphone array that allows one to have some control of the vertical spatial response of the array. A second-order system was built and measured.

This paper describes a beamforming microphone array consisting of pressure microphones that are mounted on the surface of a rigid sphere. The beamformer is based on a spherical harmonic decomposition of the soundfield. We show that this allows a simple and computationally effective, yet flexible beamformer structure. The look-direction can be steered to any direction in 3-D space without changing the beampattern. In general the number of sensors and their location is quite arbitrary as long as they hold a certain orthogonality constraint that we derived. For a practical example we chose a spherical array with 32 elements. The microphones are located at the center of the faces of a truncated icosahedron. The radius of the sphere is 5 cm. With this setup we can achieve a Directivity Index of 12 dB and higher. The operating frequency range is from 100 Hz to 5 kHz.

A linear-correction least-squares estimation procedure is proposed for the source localization problem under an additive measurement error model. The method, which can be easily implemented in a real-time system with moderate computational complexity, yields an efficient source location estimator without assuming a priori knowledge of noise distribution. Alternative existing estimators, including likelihood-based, spherical intersection, spherical interpolation, and quadratic-correction least-squares estimators, are reviewed and comparisons of their complexity, estimation consistency and efficiency against the Cramer-Rao lower bound are made. Numerical studies demonstrate that the proposed estimator performs better under many practical situations.

A multi-input one-step least-squares (OSLS) algorithm for passive source localization is proposed. It is shown that the OSLS algorithm is mathematically equivalent to the so-called spherical interpolation (SI) method but with less computational complexity. The OSLS/SI method uses spherical equations (instead of hyperbolic equations) and solves them in a least-squares sense. Based on the adaptive eigenvalue decomposition time delay estimation method previously proposed by the same authors and the OSLS source localization algorithm, a real-time passive source localization system for video camera steering is presented. The system demonstrates many desirable features such as accuracy, portability, and robustness.

It is well known that the effectiveness of loudspeaker-based 3D audio systems is critically dependent on the listener being in a known position, the so-called "sweet spot". In this paper we propose a new system that provides increased robustness to perturbations such as head movement, reverberation, and different head shapes. The system is based on using a particular geometry that offers a combination of symmetric and asymmetric loudspeaker positions. This combination increases the bandwidth over which the proposed system is robust, when compared with existing 3D audio systems.

To locate an acoustic source in a room, the relative delay between microphone pairs must be determined efficiently and accurately. However, most traditional time delay estimation (TDE) algorithms fail in reverberant environments. A new approach is proposed that takes into account the reverberation of the room. A real time PC-based TDE system running under Microsoft/sup TM/ Windows system was developed with three TDE techniques: classical cross-correlation, phase transform, and a new algorithm that is proposed in this paper. The system provides an interactive platform that allows users to compare performance of these algorithms.

Acoustic crosstalk cancellation is a signal processing technique whereby two (or possibly more) loudspeakers are used to deliver desired signals exactly at the listener's ears. Such a system is useful for three-dimensional (3-D) audio applications, and removes the requirement for the listener to wear headphones. However, crosstalk cancelers are notoriously nonrobust to slight movements in head position. Recently, a system consisting of two closely spaced loudspeakers (referred to as the "stereo-dipole") has been proposed to improve the robustness of the crosstalk canceler. In this letter we present a robustness analysis of the two-channel crosstalk canceler, and derive optimum loudspeaker positions.

Crosstalk cancellation is a signal processing technique whereby two (or more) loudspeakers are used to deliver desired signals exactly at the listener's ears. Such a system is useful for 3D audio applications, and removes the requirement for the listener to wear headphones. However, crosstalk cancelers are notoriously non-robust to slight movements in head position, and there currently exists no clear method for determining the best loudspeaker placement in a given situation. We propose a robustness measure to evaluate the performance of crosstalk cancelers as a function of loudspeaker spacing. Based on this analysis we conclude that certain loudspeaker spacings give far better robustness performance, and provide a simple empirically-derived equation for determining the optimum loudspeaker spacing in a given situation.

Virtual environment research has focused on interactive image generation and has largely ignored acoustic modeling for spatialization of sound.Yet, realistic auditory cues can complement and enhance visual cues to aid navigation, comprehension, and sense of presence in virtual environments.A primary challenge in acoustic modeling is computation of reverberation paths from sound sources fast enough for real-time auralization.We have developed a system that uses precomputed spatial subdivision and "beam tree" data structures to enable real-time acoustic modeling and auralization in interactive virtual environments.The spatial subdivision is a partition of 3D space into convex polyhedral regions (cells) represented as a cell adjacency graph.A beam tracing algorithm recursively traces pyramidal beams through the spatial subdivision to construct a beam tree data structure representing the regions of space reachable by each potential sequence of transmission and specular reflection events at cell boundaries.From these precomputed data structures, we can generate high-order specular reflection and transmission paths at interactive rates to spatialize fixed sound sources in real-time as the user moves through a virtual environment.Unlike previous acoustic modeling work, our beam tracing method: 1) supports evaluation of reverberation paths at interactive rates, 2) scales to compute highorder reflections and large environments, and 3) extends naturally to compute paths of diffraction and diffuse reflection efficiently.We are using this system to develop interactive applications in which a user experiences a virtual environment immersively via simultaneous auralization and visualization.

A new first-order differential microphone array with an infinitely steerable and variable beampattern is described. The microphone consists of 6 small pressure microphones flush-mounted on the surface of a 3/4" diameter rigid nylon sphere. The microphones are located on the surface at points where included octahedron vertices contact the spherical surface. By appropriately combining the three Cartesian orthogonal pairs with simple scalar weightings, a general first-order differential microphone beam (or beams) can be realized and directed to any angle in 4/spl pi/ steradian space. A working real-time version has been created and measured results from this microphone are shown. This microphone should be useful for surround sound recording/playback applications and to virtual reality audio applications.

In this paper we describe an algorithm for optimizing a microphone array with the objective of receiving a desired speech signal from a known direction, in the presence of broad band noise sources with unknown locations and possibly slowly varying characteristics. Each microphone feeds a tapped-delay-line filter, and the outputs of the filters are summed to give the final output. The tap weights of the filters are adjusted adaptively to minimize the output power subject to a constraint on the allowed distortion of the desired speech signal. Since the quality of a speech signal is quite insensitive to phase distortion, the constraint is on the magnitude transfer function from the source to the output. It is this choice of constraint that differentiates our approach from other methods of adapting arrays.