Fatigue and distraction are the most common long-term poor state and short-term abnormal behavior of drivers, significantly increasing the driving risk of vehicles equipped with the advanced driver assistance system (ADAS). To provide a more reliable decision-making basis for ADAS and improve driving safety, this paper proposes a driving risk assessment framework considering the driver’s long-term poor state and short-term abnormal behavior. Firstly, based on the self-built fatigue dataset and transfer learning method, an adaptive fatigue detection model with strong generalization capability is established to enable multi-view driver fatigue detection. Then, the idea of multi-clustering and adding offset parameters is introduced into the classical contrast loss function, and the D-InfoNCE loss function is designed to realize the accurate identification of the driver’s specific distraction behavior under open set detection. Subsequently, a driving risk assessment system is developed to quantify driving risk based on the vehicle driving risk factors when fatigued or distracted driving occurs. Finally, the proposed driving risk assessment system is validated by the datasets and driver-in-the-loop test bench. The results show that the proposed framework can accurately detect the driver’s fatigue state and distraction behavior and give ADAS the corresponding driving risk levels to enhance driving safety.

Shan Wu, Chunlei Xin, Hongyu Lin, Xianpei Han, Cao Liu, Jiansong Chen, Fan Yang, Guanglu Wan, Le Sun. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Multi-label unknown intent detection is a challenging task where each utterance may contain not only multiple known but also unknown intents. To tackle this challenge, pioneers proposed to predict the intent number of the utterance first, then compare it with the results of known intent matching to decide whether the utterance contains unknown intent(s). Though they have made remarkable progress on this task, their method still suffers from two important issues: 1) It is inadequate to extract multiple intents using only utterance encoding; 2) Optimizing two sub-tasks (intent number prediction and known intent matching) independently leads to inconsistent predictions. In this paper, we propose to incorporate segment augmentation rather than only use utterance encoding to better detect multiple intents. We also design a prediction consistency module to bridge the gap between the two sub-tasks. Empirical results on MultiWOZ2.3 show that our method achieves state-of-the-art performance and improves the best baseline significantly.

Temporal modeling still remains as a challenge for action recognition. Most existing temporal models focus on learning local variation between neighbor frames. There exists obvious deviations between local and global variations, such as subtle and notable motion variations. In this paper, we propose a global temporal difference module for action recognition, which consists of two sub-modules,i.e., a global aggregation module and a global difference module. These two sub-modules cooperate following the idea of using prior knowledge from the global view (i.e., global motion variation) to guide local learning at each moment. In the global aggregation module, the global prior knowledge is learned by aggregating the visual feature sequence of video into a global vector. In the global difference module, we prepare the difference vector sequence of video by subtracting each local vector from the global vector. Our method performs as a contextual guidance with a global view. The sequential dependency between these difference vectors is exploited with a channel-wise self-attention operation. Finally, the difference vectors at each timestamp are further used to enhance the semantics of the original local features. The enhanced features endow the action recognition has less deviation to understand the variation in the video globally. We instantiate the global temporal difference module into the ResNet block to form a global temporal difference network (GTDNet). Exhaustive experiments are conducted and our method achieves competitive performance at small FLOPs on Something-Something V1 & V2 and Kinetics-400.

Data-driven methods have achieved notable performance on intent detection, which is a task to comprehend user queries. Nonetheless, they are controversial for over-confident predictions. In some scenarios, users do not only care about the accuracy but also the confidence of model. Unfortunately, mainstream neural networks are poorly calibrated, with a large gap between accuracy and confidence. To handle this problem defined as confidence calibration, we propose a model using the hyperspherical space and rebalanced accuracy-uncertainty loss. Specifically, we project the label vector onto hyperspherical space uniformly to generate a dense label representation matrix, which mitigates over-confident predictions due to overfitting sparse one-hot label matrix. Besides, we rebalance samples of different accuracy and uncertainty to better guide model training. Experiments on the open datasets verify that our model outperforms the existing calibration methods and achieves a significant improvement on the calibration metric.

Dialogue topic segmentation is a challenging task in which dialogues are split into segments with pre-defined topics. Existing works on topic segmentation adopt a two-stage paradigm, including text segmentation and segment labeling. However, such methods tend to focus on the local context in segmentation, and the inter-segment dependency is not well captured. Besides, the ambiguity and labeling noise in dialogue segment bounds bring further challenges to existing models. In this work, we propose the Parallel Extraction Network with Neighbor Smoothing (PEN-NS) to address the above issues. Specifically, we propose the parallel extraction network to perform segment extractions, optimizing the bipartite matching cost of segments to capture inter-segment dependency. Furthermore, we propose neighbor smoothing to handle the segment-bound noise and ambiguity. Experiments on a dialogue-based and a document-based topic segmentation dataset show that PEN-NS outperforms state-the-of-art models significantly.

Shan Wu, Bo Chen, Chunlei Xin, Xianpei Han, Le Sun, Weipeng Zhang, Jiansong Chen, Fan Yang, Xunliang Cai. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Pre-trained language models have achieved noticeable performance on the intent detection task. However, due to assigning an identical weight to each sample, they suffer from the overfitting of simple samples and the failure to learn complex samples well. To handle this problem, we propose a density-based dynamic curriculum learning model. Our model defines the sample's difficulty level according to their eigenvectors' density. In this way, we exploit the overall distribution of all samples' eigenvectors simultaneously. Then we apply a dynamic curriculum learning strategy, which pays distinct attention to samples of various difficulty levels and alters the proportion of samples during the training process. Through the above operation, simple samples are well-trained, and complex samples are enhanced. Experiments on three open datasets verify that the proposed density-based algorithm can distinguish simple and complex samples significantly. Besides, our model obtains obvious improvement over the strong baselines.

Dialogue state tracking (DST), which estimates user goals given a dialogue context, is an essential component of task-oriented dialogue systems.Conventional DST models are usually trained offline, which requires a fixed dataset prepared in advance.This paradigm is often impractical in real-world applications since online dialogue systems usually involve continually emerging new data and domains.Therefore, this paper explores Domain-Lifelong Learning for Dialogue State Tracking (DLL-DST), which aims to continually train a DST model on new data to learn incessantly emerging new domains while avoiding catastrophically forgetting old learned domains.To this end, we propose a novel domainlifelong learning method, called Knowledge Preservation Networks (KPN), which consists of multi-prototype enhanced retrospection and multi-strategy knowledge distillation, to solve the problems of expression diversity and combinatorial explosion in the DLL-DST task.Experimental results show that KPN effectively alleviates catastrophic forgetting and outperforms previous state-of-the-art lifelong learning methods by 4.25% and 8.27% of whole joint goal accuracy on the MultiWOZ benchmark and the SGD benchmark, respectively.

Given a long text, the summarization system aims to obtain a shorter highlight while keeping important information on the original text. For customer service, the summaries of most dialogues between an agent and a user focus on several fixed key points, such as user's question, user's purpose, the agent's solution, and so on. Traditional extractive methods are difficult to extract all predefined key points exactly. Furthermore, there is a lack of large-scale and high-quality extractive summarization datasets containing key points. In order to solve the above challenges, we propose a Distant Supervision based Machine Reading Comprehension model for extractive Summarization (DSMRC-S). DSMRC-S transforms the summarization task into the machine reading comprehension problem, to fetch key points from the original text exactly according to the predefined questions. In addition, a distant supervision method is proposed to alleviate the lack of eligible extractive summarization datasets. We conduct experiments on a large-scale summarization dataset collected in customer service scenarios, and the results show that the proposed DSMRC-S outperforms the strong baseline methods by 4 points on ROUGE-L.

A driver-vehicle-road (DVR) model based on kinematic vehicle model is proposed in this paper. In this DVR model, the kinematics vehicle-road model is adopted, and the driver model considering the human driver's characteristics is also included. Thus the behaviors of human driver's preview and neuromuscular delay can be considered in design of path planner and controller by using this DVR model. The repulsive force field based on the artificial potential field (APF) and the circle decomposition of vehicle shape are used to describe the constraints of obstacle avoidance and the road departure avoidance. Based on the proposed DVR model, a trajectory planer using model predictive control (MPC) is designed with consideration of collision and lane-departure avoidance, driver's intention, and vehicle occupant comfort. Simulation results show that with the proposed planner, the vehicle can successfully avoid static/moving obstacles and return to the original lane without lane departure. Simulation results indicate that the proposed kinematic vehicle model based DVR model can be used to design the path planner in normal driving and some typical driving scenarios. And the proposed path planner can provide the vehicle driven by different human drivers with individually safe trajectories in typical scenarios of obstacle avoidance.

Intent detection and slot filling are two important tasks in Spoken Language Understanding. The Condition Random Fields (CRF) was introduced for the tasks pretty much the same fashion to deep neural networks. Recently, attention based encoder-decoder models have shown promising results for joint intent detection and slot filling tasks in spoken language understanding and dialog systems. However, the two tasks are often trained separately. In this paper, we propose ACJIS, a novel Attentive Cross approach for Joint Intent detection and Slot filling. We introduce a cross attention approach to enhance the modeling power on capturing the meaning of word at both tagging level and word level. In order to utilize the information from the two tasks, we leverage multi-task learning to train the model. Our model generates state-of-the-art results on the bench-mark ATIS task. The proposed model also achieves significant gains over the attention based RNN modeling approach for intent detection and slot filling respectively.

With the fast development of high-speed network and digital video recording technologies, broadcast video has been playing a more and more important role in our daily life. In this paper, we propose a novel news story segmentation scheme which can segment broadcast video into story units with multi-modal information fusion (MMIF) strategy. Compared with traditional methods, the proposed scheme extracts a wealth of semantic-level features including anchor person, topic caption, face, silence, acoustic change, audio keywords and textual content. Parallel to this, we make use of a multi-modal information fusion strategy for news story boundary characterization by joining these visual, audio and textual cues. Encouraging experimental results on News Vision dataset demonstrate the effectiveness of the proposed scheme.

Near-duplicate image retrieval (NDIR) is an important topic for many applications such as multimedia content management, copyright infringement identification et al. In this work we propose a novel NDIR framework based on visual phrase. Compared with previous researches, this paper first introduces a spatial visual phrase (SVP) model enabling to capture relative geometry information between visual words. Then, it proposes an image-specific strategy to select descriptive SVPs. The strategy can not only handle the phrase sparseness problem which occurs in traditional selection strategy but also allow to select visual phrases according to the characteristic of each image. Experiments are carried out over Ukbench dataset and TRECVID dataset respectively, and encouraging experimental results demonstrate that both the SVP model and the selection strategy significantly improve the overall performance.

Efficient detection of commercial is an important topic for many applications such as commercial monitoring, market investigation. This paper reports an unsupervised technique of discovering commercial by mining repeated sequence in audio stream. Compared with previous work, we focus on solving practical problems by introducing three principles of commercial: repetition principle, independence principle and equivalence principle. Based on these principles, we detect the commercials by first mining maximal repeated sequences (MRS) and then post-processing the MRS pairs based on independence principle and equivalence principle for final result. In addition, a coarse-to-fine scheme is adopted in the acoustic matching stage to save computational cost. Extensive experiments both on simulated data and real broadcast data demonstrate the effectiveness of our method.

Previous work demonstrated that the improvements in spectral efficiency and link reliability can be obtained in wireless relay networks employing terminals with MIMO capability. In order to take the advantage of MIMO relay networks while mitigate the complexity of decoding procedure, we propose a scheme that applies ZF relaying at the relay terminals and utilizes MMSE V-BLAST receiver at the destination terminal for two- hop half-duplex coherent MIMO relay networks. Considering the channel estimation is often imperfect due to the noise and interference, this paper extends the V-BLAST receiver design to the imperfect channel state information (CSI), assuming the channel estimation errors are independent complex Gaussian with zero mean and known second-order statistics. A robust MMSE nulling operator and its corresponding symbol detection ordering criterion is proposed for the MMSE V-BLAST receiver, which aims at providing, in each layer, the minimum average mean square error (MSE), over the random channel errors. The proposed ordering rule achieves global average MSE optimization which results in better performance than the previously proposed robust ZF V-BLAST. It has been shown that the proposed design exploits the characteristics of MIMO relay networks to provide good performance with respect to imperfect CSI while keeping a low complexity of the decoding procedure.

Previous work demonstrated that the improvements in spectral efficiency and link reliability can be obtained in wireless relay networks employing terminals with MIMO capability. We propose to apply the non layered and layered ZF (V-BLAST) receiver to the relay and destination terminals in coherent MIMO relay networks under two-hop half-duplex relaying. Considering the channel estimation is often imperfect due to the noise and interference, this paper assumes that no perfect channel state information (CSI) is available for both backward and forward channels. The V-BLAST receiver is extended to the imperfect CSI under the assumption that the channel estimation errors are independent complex Gaussian random variables with zero mean and known second-order statistics. A robust symbol detection ordering criterion is proposed for applying ZF V-BLAST to the MIMO relay networks. This new ordering rule aims at providing, in each layer, the maximum average signal to interference- plus-noise ratio (SINR), over the random channel errors. Our simulations show that aided by the new ordering criterion, the suboptimal ZF operators, directly resulted from the imperfect CSI, is capable of alleviating the effect of CSI errors and improving the robustness of V-BLAST system.

In practice, channel estimation is often imperfect due to the noise and interference. This paper extends the V-BLAST system to the imperfect channel state information by dividing the channel matrix into two parts. Upon an assumption that the channel estimation error vectors are independent complex Gaussian with zero mean and known second-order statistics, a symbol detection ordering criteria for ZF V-BLAST system is proposed which aims at providing, in each layer, the maximum average signal to interference-plus-noise ratio (SINR), over the random channel errors. This robust V-BLAST ordering rule takes the imperfect CSI into account, while maintaining the simple implementation of the V-BLAST structure. It is shown that the new ordering criterion is capable of achieving global performance optimization and outperforms the standard ZF V-BLAST when estimation errors exist.

In practical V-BLAST systems, channel estimation is often imperfect due to noise and interference. This paper analyzes the impacts of imperfect channel estimation on MMSE V-BLAST system under the assumption that the channel estimation errors are unknown deterministic and norm-bounded. Two novel V-BLAST ordering algorithms are therefore presented with the idea of worst-case post-processing performance optimization. The proposed algorithms organize the order of symbol processing according to their worst-case post-processing performance while maintain the simple implementation of the V-BLAST structure. The simulation results show our approaches are robust to the channel estimation errors.