Digital phenotyping (DP) is a multidisciplinary field of science that quantifies the individual level phenotype through active and passive data. Although DP is a multidisciplinary field, there lacks a technical and a systematic approach to representing DP. This work proposes the development of digital phenotype profile (DPP) to represent a user’s physical and behavioural health baseline through systematic investigations with an emphasis on robustness and explainability. To achieve this, a Statistical, Information Theory, and Data-driven (SID) pipeline will develop the foundation of the DPP. SID evaluates the non-linearity of the signal to offer inference for domain-specific feature extraction, evaluates the information theory to rank the DPP parameters, and imputes missing data for robust analysis, respectively. SID was applied to a 24-hr Multi-Level dataset and was able to represent individual DPPs. The respective DPPs were visualized and clusters of awake and asleep were used for individual specific modelling.

This paper introduces a first of its kind dataset for audio scene analysis (ASA) and presents a baseline approach for audio source counting. SARdBScene is developed to promote research for audio source counting, as a relatively new ASA task, and present a comprehensive dataset that covers a variety of scenarios and audio-based tasks. It contains 80 hours of audio scene mixtures depicting four distinct environments with detailed annotations that make it a unique collection of curated data in the audio analysis landscape. Our baseline approach using ResNet establishes state-of-the-art results of 77.3% and 85.7% accuracy for audio source counting up to 12 sources and speaker counting up to 4 speakers, respectively.

Audio scene analysis (ASA) is a challenging and multifaceted task in audio signal processing that uncovers information about the nature of an audio recording. Regardless of the analysis goal, a number of audio sources are observed in any audio scene. However, this consideration is usually not explored or given considerable thought in research. This work aims to demonstrate the utility of audio source counting with a novel solution consisting of a multimodal system for ASA. Both speaker counting and sound event counting techniques use deep neural networks (DNN) to predict the number of sources. We are able to present competitive results for audio source counting by achieving prediction accuracy of 46.03% and 89.57% with a margin of error of$\pm 1$for speaker counting, which outperforms state-of-the-art systems for similar tasks. For sound event counting we achieve 50.55% and 86.59% prediction accuracy and accuracy with a margin of error of$\pm 1$, respectively, that establishes a clear baseline. Our system also demonstrates real-time aspects with an overall processing time of$\sim 0.4614$s per audio recording.