In this study, we investigate integrating eye tracking with auditory attention decoding (AAD) using portable EEG devices, specifically a mobile EEG cap and cEEGrid, in a preliminary analysis with a single participant. A novel audiovisual dataset was collected using a mobile EEG system designed to simulate real-life listening environments. Our study has two main objectives: (1) to use eye tracking data to automatically infer the labels of attended and unattended speech streams, and (2) to train an AAD model using these estimated labels, evaluating its performance through speech reconstruction accuracy. The results demonstrate the feasibility of using eye tracking data to estimate attended speech labels, which were then used to train speech reconstruction models. We validated our models with varying amounts of training data and a second dataset from the same participant to assess generalization. Additionally, we examined the impact of mislabeling on AAD accuracy. These findings provide preliminary evidence that eye tracking can be used to infer speech labels, offering a potential pathway for brain-controlled hearing aids, where true labels are unknown.

A wearable microphone array platform is used to localize stationary sound sources and amplify the sound in the desired directions using several beamforming methods. The platform is equipped with inertial sensors and a magnetometer allowing predictions of source locations during orientation changes and compensation for the displacement in the array configuration. The platform is modular, open and 3D printed to allow for easy reconfiguration of the array and for reuse in other applications, e.g., mobile robotics. The software components are based on open source. A new method for source localization and signal reconstruction using Taylor expansion of the signals is proposed. This and various standard and non-standard Direction of Arrival (DOA) methods are evaluated in simulation and experiments with the platform to track and reconstruct multiple and single sources. Results show that sound sources can be localized and tracked robustly and accurately while rotating the platform and that the proposed method outperforms standard methods at reconstructing the signals.

This paper presents three iterative methods for orientation estimation. The first two are based on iterated Extended Kalman filter (IEKF) formulations with different state representations. The first is using the well-known unit quaternion as state (q-IEKF) while the other is using orientation deviation which we call IMEKF. The third method is based on nonlinear least squares (NLS) estimation of the angular velocity which is used to parametrise the orientation. The results are obtained using Monte Carlo simulations and the comparison is done with the non-iterative EKF and multiplicative EKF (MEKF) as baseline. The result clearly shows that the IMEKF and the NLS-based method are superior to q-IEKF and all three outperform the non-iterative methods.