Large language models (LLMs) have been shown to exhibit *emergent abilities* in some downstream tasks, where model performance stagnates at first and then improves sharply and unpredictably with scale beyond a threshold. In this work, we investigate the phenomenon by grouping questions based on difficulty level and provide a possible explanation for emergent abilities. Specifically, we observe U-shaped scaling for hard questions and inverted-U scaling followed by steady improvement for easy questions. The two scaling patterns initially offset each other, causing stagnant overall performance. The performance starts to soar when the scaling pattern of easy questions reverts from inverse to standard scaling, leading to emergent abilities. Based on this finding, we propose a simple yet effective pipeline, called *Slice-and-Sandwich*, to predict the emergence threshold and model performance beyond the threshold. Our code is publicly available at https://github.com/tony10101105/ExpEmergence.

3D visual grounding aims to identify the target object within a 3D point cloud scene referred to by a natural language description. Previous works usually require significant data relating to point color and their descriptions to exploit the corresponding complicated verbo-visual relations. In our work, we introduce Vigor, a novel Data-Efficient 3D Visual Grounding framework via Order-aware Referring. Vigor leverages LLM to produce a desirable referential order from the input description for 3D visual grounding. With the proposed stacked object-referring blocks, the predicted anchor objects in the above order allow one to locate the target object progressively with-out supervision on the identities of anchor objects or exact relations between anchor/target objects. We also present an order-aware warm-up training strategy, which augments referential orders for pre-training the visual grounding framework, allowing us to better capture the complex verbo-visual relations and benefit the desirable data-efficient learning scheme. Experimental results on the NR3D and ScanRefer datasets demonstrate our superiority in low-resource scenarios. In particular, Vigor surpasses current state-of-the-art frameworks by 9.3% and 7.6% grounding accuracy under 1% data and 10% data settings on the NR3D dataset, respectively.

Neuron-level interpretations aim to explain network behaviors and properties by investigating neurons responsive to specific perceptual or structural input patterns. Although there is emerging work in the vision and language domains, none is explored for acoustic models. To bridge the gap, we introduce AND, the first Audio Network Dissection framework that automatically establishes natural language explanations of acoustic neurons based on highly responsive audio. AND features the use of LLMs to summarize mutual acoustic features and identities among audio. Extensive experiments are conducted to verify AND’s precise and informative descriptions. In addition, we highlight two acoustic model behaviors with analysis by AND. First, models discriminate audio with a combination of basic acoustic features rather than high-level abstract concepts. Second, training strategies affect neuron behaviors. Supervised training guides neurons to gradually narrow their attention, while self-supervised learning encourages neurons to be polysemantic for exploring high-level features. Finally, we demonstrate a potential use of AND in audio model unlearning by conducting concept-specific pruning based on the descriptions.

Deep neural networks (DNNs) are recently found to be vulnerable to JPEG compression artifacts, which distort the feature representations of DNNs leading to serious accuracy degradation. Most existing training methods which aim to address this problem add compressed images to the training data to enhance the robustness of DNNs. However, their improvements are limited since these methods usually regard the compressed images as new training samples instead of distorted samples. The feature distortions between the raw images and the compressed images are not investigated. In this work, we propose a new training method, called Feature Consistency Training, that is designed to minimize the feature distortions caused by JPEG artifacts. At each training iteration, we simultaneously input a raw image and its compressed version with a randomly sampled quality into a DNN model and extract the features from the internal layers. By adding feature consistency constraint to the objective function, the feature distortions in the representation space are minimized in order to learn robust filters. Besides, we present a residual mapping block which takes the quality factor of the compressed image as an additional information to further reduce the feature distortion. Extensive experiments demonstrate that our method outperforms several existed training methods on JPEG compressed images. Furthermore, DNN models trained by our method are found to be more robust to unseen distortions.

Head pose estimation has attracted immense research interest recently, as its inherent information significantly improves the performance of face-related applications such as face alignment and face recognition. In this paper, we conduct an in-depth study of head pose estimation and present a multiregression loss function, an L2 regression loss combined with an ordinal regression loss, to train a convolutional neural network (CNN) that is dedicated to estimating head poses from RGB images without depth information. The ordinal regression loss is utilized to address the nonstationary property observed as the facial features change with respect to different head pose angles and learn robust features. The L2 regression loss leverages these features to provide precise angle predictions for input images. To avoid the ambiguity problem in the commonly used Euler angle representation, we further formulate the head pose estimation problem in quaternions. Our quaternion-based multiregression loss method achieves state-of-the-art performance on the AFLW2000, AFLW test set, and AFW datasets and is closing the gap with methods that utilize depth information on the BIWI dataset.

Diabetic retinopathy is the primary cause of blindness in the working-age population of the developed world. Diagnosing the disease heavily relies on imaging studies, which is a time consuming and a manual process performed by trained clinicians. Enhancing the accuracy and speed of the detection process can potentially have a significant impact on population health via early diagnosis and intervention. Motivated by this, we propose a recognition pipeline based on deep convolutional neural networks. In our pipeline, we design lightweight networks called SI2DRNet-vl along with six methods to further boost the detection performance. Without any fine-tuning, our recognition pipeline outperforms state of the art on the Messidor dataset along with 5.26x fewer in total parameters and 2.48x fewer in total floating operations.

Finding the locations and identities of faces in videos is a very important task in numerous applications. In this paper, we propose a correlation-based face detection approach to improve the performance of face recognition tasks for videos. We apply correlation measures to pairs of response maps which are generated from automatically selected neurons in deep convolutional neural network (CNN) models to detect faces in each video frame. The embeddings extracted from faces cropped by our proposed approach are more consistent across each video sequence and more suitable for face recognition and clustering tasks. Experimental results from the YouTube Faces (YTF) dataset demonstrate that our proposed approach is more robust and achieves better recognition accuracy compared to state-of-the-art face detection approaches.

In this paper, we propose Confidence Network (ConfNet) which not only makes predictions on input images but also generates a confidence score that estimates the probability of correctness of each prediction. Furthermore, Confidence Loss is proposed to make ConfNet automatically learn confidence scores in the training phase. The experiments on two public datasets show that the confidence scores generated by ConfNet are highly correlated with the model accuracy and outperforms two related methods. When stacking two ConfNets in a cascade structure, 3.8x computational cost can be saved compared to the single state-of-the-art model with only 0.1 % increase of error rate.

This paper investigates the type-2 T-S fuzzy multiple feedback-loop guaranteed cost controller design for uncertain singular systems with transmission and state delays. Based on the type-2 T-S fuzzy theory, this work constructs a mathematical model for a class of nonlinear uncertain singular time-delay systems. Using the parallel distributed compensation concept with Lyapunov theory and linear matrix inequality technique, a robust guaranteed cost fuzzy controller is designed with the multiple feedback-loop structure to stabilize the considered system with guaranteed cost performance. Finally, an optimization procedure is presented to reduce the upper bound of guaranteed cost. According to the simulation results, when parametric uncertainties and transmission and state delays exist simultaneously, it indicates that the proposed guaranteed cost controller demonstrates better design flexibility. Also, the closed-loop system is robustly stable and the upper bound of guaranteed cost is decreased.

In this paper, we present an unsupervised hierarchical image segmentation algorithm based on a split-and-merge scheme. In the split phase, we propose an efficient partition algorithm, called Just-Noticeable-Difference Bayesian Sequential Partitioning (JND-BSP), to partition image pixels into a few regions, within which the color variations are perceived to be smoothly changing without apparent color differences. In the merge phase, we propose a simple but effective merging criterion to sequentially construct a hierarchical structure that represents the relative similarity among these partitioned regions. Instead of generating a segmentation result with a fixed number of segments, the new algorithm produces an entire hierarchical representation of the given image in a single run. This hierarchical representation is informative and can be very useful for subsequent processing, like object recognition and scene analysis. To demonstrate the effectiveness and efficiency of our method, we compare our new segmentation algorithm with several existing algorithms. Experiment results show that our new algorithm can not only offers a more flexible way to segment images but also provides segmented results close to human's visual perception.

Spectral matting is a useful technique for image matting problem. A crucial issue of spectral matting is to determine the number of matting components which has large impacts on the matting performance. In this paper, we propose an improved framework based on spectral matting in order to solve this limitation. Iterative K-means clustering with the assistance of the modularity measure is adopted to obtain the hard segmentation that can be used as the initial guess of soft matting components. The number of matting components can be determined automatically because the improved framework will search possible image components by iteratively dividing image subgraphs.

We present an optimization architecture for a cross- layer design in wireless video transmission and investigate the multicarrier effect on wireless communication. To transmit video frames constrained by delay bound and target frame error rate (FER), we formulate an optimization problem to minimize the required resource units for a single user by adapting payload length, M-ary modulation, block size, and code rate. Instead of using exhaustive approach, we propose a minimal required resource units (MRRU) algorithm with feedback of mean mutual information per bit (MMIB) to resolve this problem by binomial distribution function approximation. Finally, the performance comparison between MRRU algorithm and maximum throughput approach is given, and the effect on multicarrier system in a correlated Rayleigh fading channel is shown.