Abstract An ultralightweight self-supervised stereo matching network, called US $$^3$$ 3 Net, which requires only 12K parameters, is designed for efficient and accurate depth estimation using resource-constrained devices. US $$^3$$ 3 Net incorporates two key innovations: a low-complexity feature extraction module and a soft occlusion detection approach for performance improvement. First, we design a low-complexity feature extraction module to reduce the computational burden while preserving structural details necessary for stereo matching. By refining the encoder backbone and aggregation module, our design ensures a better balance between model complexity and accuracy. Second, to address occlusion-related errors in disparity estimation, we propose a novel occlusion detection method, called Depth-Aware Geometric Soft Occlusion (DAGSO), to adaptively define the occlusion confidence scores based on depth information. DAGSO can effectively mitigate false occlusions in distant regions and can improve the accuracy of disparity estimation. Experimental results using KITTI datasets demonstrate that US $$^3$$ 3 Net achieves state-of-the-art performance in terms of model complexity and depth estimation accuracy. It outperforms previous self-supervised stereo matching methods and monocular depth estimation methods in metrics such as AbsRel, SqRel, RMSE, and RMSElog at a reduction of parameter size by 47% compared with ES $$^3$$ 3 Net (23K). This makes US $$^3$$ 3 Net a practical solution for real-time depth estimation on edge devices such as drones and autonomous systems. Code is available at: https://github.com/g830319ag/US3Net.

In this study, we proposed a new end-to-end convolutional neural network, called MS-SincResNet, for music genre classification. MS-SincResNet appends 1D multi-scale SincNet (MS-SincNet) to 2D ResNet as the first convolutional layer in an attempt to jointly learn 1D kernels and 2D kernels during the training stage. First, an input music signal is divided into a number of fixed-duration (3 seconds in this study) music clips, and the raw waveform of each music clip is fed into 1D MS-SincNet filter learning module to obtain three-channel 2D representations. The learned representations carry rich timbral, harmonic, and percussive characteristics comparing with spectrograms, harmonic spectrograms, percussive spectrograms and Mel-spectrograms. ResNet is then used to extract discriminative embeddings from these 2D representations. The spatial pyramid pooling (SPP) module is further used to enhance the feature discriminability, in terms of both time and frequency aspects, to obtain the classification label of each music clip. Finally, the voting strategy is applied to summarize the classification results from all 3-second music clips. In our experimental results, we demonstrate that the proposed MS-SincResNet outperforms the baseline SincNet and many well-known hand-crafted features. Considering individual 2D representation, MS-SincResNet also yields competitive results with the state-of-the-art methods on the GTZAN dataset and the ISMIR2004 dataset. The code is available at https://github.com/PeiChunChang/MS-SincResNet.

Investigating the identity, distribution, and evolution of bird species is important for both biodiversity assessment and environmental conservation. The discrete wavelet transform (DWT) has been widely exploited to extract time-frequency features for acoustic signal analysis. Traditional approaches usually compute statistical measures (e.g., maximum, mean, standard deviation) of the DWT coefficients in each subband independently to yield the feature descriptor, without considering the intersubband correlation. A new acoustic descriptor, called the local wavelet acoustic pattern (LWAP), is proposed to characterize the correlation of the DWT coefficients in different subbands for birdsong recognition. First, we divide a variable-length birdsong segment into a number of fixed-duration texture windows. For each texture window, several LWAP descriptors are extracted. The vector of locally aggregated descriptors (VLAD) is then used to aggregate the set of LWAP descriptors into a single VLAD vector. Finally, principal component analysis (PCA) plus linear discriminant analysis (LDA) are employed to reduce the feature dimensionality for classification purposes. Experiments on two birdsong datasets show that the proposed LWAP descriptor outperforms other local descriptors, including linear predictive coding cepstral coefficients, Mel-frequency cepstral coefficients, perceptual linear prediction cepstral coefficients, chroma features, and prosody features. Furthermore, the proposed LWAP descriptor, followed by VLAD encoding, PCA plus LDA feature extraction, and a simple distance-based classifier, yields promising results that are competitive with those obtained by the state-of-the-art convolutional neural networks.

Traditional birdsong recognition approaches used acoustic features based on the acoustic model of speech production or the perceptual model of the human auditory system to identify the associated bird species. In this paper, a new feature descriptor that uses image shape features is proposed to identify bird species based on the recognition of fixed-duration birdsong segments where their corresponding spectrograms are viewed as gray-level images. The MPEG-7 angular radial transform (ART) descriptor, which can compactly and efficiently describe the gray-level variations within an image region in both angular and radial directions, will be employed to extract the shape features from the spectrogram image. To effectively capture both frequency and temporal variations within a birdsong segment using ART, a sector expansion algorithm is proposed to transform its spectrogram image into a corresponding sector image such that the frequency and temporal axes of the spectrogram image will align with the radial and angular directions of the ART basis functions, respectively. For the classification of 28 bird species using Gaussian mixture models (GMM), the best classification accuracy is 86.30% and 94.62% for 3-second and 5-second birdsong segments using the proposed ART descriptor, which is better than traditional descriptors such as LPCC, MFCC, and TDMFCC.

In this paper, we will propose a 3D model retrieval approach using 2D cepstral features. First, six projection planes representing the elevation (depth) value are generated. Then, 2D cepstral features are extracted from each projection plane for searching similar 3D models. Experiments conducted on the Princeton Shape Benchmark (PSB) database have shown that the proposed 2D cepstral features outperforms other state-of-the-art descriptors in terms of the DCG score.

In this paper, we will propose an automatic music genre classification approach based on long-term modulation spectral analysis of spectral (OSC and MPEG-7 NASE) as well as cepstral (MFCC) features. Modulation spectral analysis of every feature value will generate a corresponding modulation spectrum and all the modulation spectra can be collected to form a modulation spectrogram which exhibits the time-varying or rhythmic information of music signals. Each modulation spectrum is then decomposed into several logarithmically-spaced modulation subbands. The modulation spectral contrast (MSC) and modulation spectral valley (MSV) are then computed from each modulation subband. Effective and compact features are generated from statistical aggregations of the MSCs and MSVs of all modulation subbands. An information fusion approach which integrates both feature level fusion method and decision level combination method is employed to improve the classification accuracy. Experiments conducted on two different music datasets have shown that our proposed approach can achieve higher classification accuracy than other approaches with the same experimental setup.

In this paper, an automatic music genre classification approach which integrates the features derived from static and transitional information of cepstral (MFCC) and spectral (OSC) features will be proposed. MFCC and OSC capture the characteristics of one audio frame. Therefore, the transitional information, including delta-MFCC, delta-OSC, delta-delta-MFCC, and delta-delta-OSC, are then extracted and combined with MFCC and OSC to improve the classification accuracy. Two information fusion techniques, including feature level fusion and decision level fusion, are developed to combine the extracted feature vectors. Experiments conducted on the music database employed in the ISMIR2004 Audio Description Contest have shown that the proposed approach can achieve a classification accuracy of 84.23%, which is better than the winner of the ISMIR2004 Music Genre Classification Contest.

This paper presents a method for automatic classification of birds into different species based on the audio recordings of their sounds. Each individual syllable segmented from continuous recordings is regarded as the basic recognition unit. To represent the temporal variations as well as sharp transitions within a syllable, a feature set derived from static and dynamic two-dimensional Mel-frequency cepstral coefficients are calculated for the classification of each syllable. Since a bird might generate several types of sounds with variant characteristics, a number of representative prototype vectors are used to model different syllables of identical bird species. For each bird species, a model selection method is developed to determine the optimal mode between Gaussian mixture models (GMM) and vector quantization (VQ) when the amount of training data is different for each species. In addition, a component number selection algorithm is employed to find the most appropriate number of components of GMM or the cluster number of VQ for each species. The mean vectors of GMM or the cluster centroids of VQ will form the prototype vectors of a certain bird species. In the experiments, the best classification accuracy is 84.06% for the classification of 28 bird species.

In this paper, we proposed a novel feature, called octave-based modulation spectral contrast (OMSC), for music genre classification. OMSC is extracted from long-term modulation spectrum analysis to represent the time-varying behavior of music signals. Experimental results have shown that OMSC outperforms MFCC and OSC. If OMSC is integrated with MFCC and OSC, the classification accuracy is 84.03% for seven music genre classification.

This paper proposed a fast vector quantization encoding algorithm called difference pyramid search (DPS). According to the formation of the difference pyramid and partial distortion elimination, a rejection test inequality is derived to progressively reject a lot of nonclosest code words as early as possible. Experimental results show that the proposed DPS algorithm outperforms other pyramid-based fast search algorithms, including mean pyramid search, L2-norm pyramid search, and mean-variance pyramid search.

In this correspondence, a fast approach to motion estimation is presented. The algorithm uses the block sum pyramid to eliminate unnecessary search positions. It first constructs the sum pyramid structure of a block. Successive elimination is then performed hierarchically from the top level to the bottom level of the pyramid. Many search positions can be skipped from being considered as the best motion vector and, thus, the search complexity can be reduced. The algorithm can achieve the same estimation accuracy as the full search block matching algorithm with much less computation time.

One of the most serious problems for vector quantization, especially for high dimensional vectors, is the high computational complexity of searching for the closest codeword in the codebook design and encoding phases. Although quantizing high dimensional vectors rather than low dimensional vectors results in better performance, the computation time needed for vector quantization grows exponentially with the vector dimension. This makes high dimensional vectors unsuitable for vector quantization. To overcome this problem, a fast search algorithm, under the assumption that the distortion is measured by the squared Euclidean distance, is proposed. Using the mean pyramids of codewords, the algorithm ran reject many codewords that are impossible matches and hence save a great deal of computation time. The algorithm is efficient for high dimensional codeword searches. Experimental results confirm the effectiveness of the proposed method.>