Deep learning-based compressed sensing (CS) technology attracts widespread attention owing to its remarkable reconstruction with only a few sampling measurements and low computational complexity. However, the existing video compressive sampling approaches cannot fully exploit the inherent interframe and intraframe correlations and sparsity of video sequences. To address this limitation, a novel sampling and reconstruction method for video CS (called WRDD) is proposed, which exploits the advantages of wavelet residual sampling and dual-domain fusion optimization. Specifically, in order to capture high-frequency details and achieve efficient and high-quality measurements, we propose a wavelet residual (WR) sampling strategy for the nonkeyframe sampling, which is achieved by the wavelet residuals between nonkeyframes and keyframes. Furthermore, a dual-domain (DD) fusion strategy is proposed, which fully combine intraframe and interframe to improve the reconstruction quality of nonkeyframes both in the pixel domain and multilevel feature domains. Extensive experiments demonstrate that our WRDD surpasses the state-of-the-art video and image CS methods in both subjective and objective evaluations. Besides, it exhibits outstanding antinoise capability and computational efficiency.

Multi-task learning in advanced driver assistance systems aims to endow models with the capacity to jointly handle multiple related tasks, such as object detection, depth estimation, and more. However, existing multi-task learning models largely rely on the extensive number of labelled data. In practice, the process of annotating data for multi-task training proves to be exceedingly costly, yet not always accurate. This study introduces an innovative setting named online multi-task inference learning that updates the multi-task model during inference. And we propose a Strong-Help-Weak (SHW) framework which aims to enhance weaker (or more challenging) tasks by leveraging guidance from closely related stronger (or easier) tasks. Specifically, we first build two benchmarks based on KITTI and BDD with four tasks (object detection, object depth estimation, lane line segmentation, and driving area segmentation). Then, we propose two novel modules inspired by two priors: 1) Detection-guided Depth Inference Learning (DetDis) module that leverages the inverse relationship between object size and distance to refine the predicted object distance; and 2) Area-guided Lane Line Inference Learning (AreaLane) module that utilises inclusion relationship between driving area and lane line to infer more accurate lane line. Both modules are efficient and can provide more reliable supervision for the corresponding weaker tasks (object distance estimation and lane line segmentation), respectively. Extensive experiments on the two benchmarks show that our SHW can obtain consistent improvements on the weaker tasks during the inference stage with low computational costs.

Distracted driver activity recognition plays a critical role in risk aversion-particularly beneficial in intelligent transportation systems. However, most existing methods make use of only the video from a single view and the difficulty-inconsistent issue is neglected. Different from them, in this work, we propose a novel MultI-camera Feature Integration (MIFI) approach for 3D distracted driver activity recognition by jointly modeling the data from different camera views and explicitly re-weighting examples based on their degree of difficulty. Our contributions are two-fold: (1) We propose a simple but effective multi-camera feature integration framework and provide three types of feature fusion techniques. (2) To address the difficulty-inconsistent problem in distracted driver activity recognition, a periodic learning method, named example re-weighting that can jointly learn the easy and hard samples, is presented. The experimental results on the 3MDAD dataset demonstrate that the proposed MIFI can consistently boost performance compared to single-view models.

Blockchain’s immutable and traceable records and independence from third-party involvement make it an irreplaceable tool in applications involving multiple stakeholders. However, securely sharing off-chain files among stakeholders while ensuring non-repudiation is a significant challenge. This is because blockchain cannot monitor off-chain behavior, and stakeholders may refuse to acknowledge records on the chain. In this study, we propose an efficient solution for secure file sharing among stakeholders through on-chain and off-chain collaboration for blockchain applications with additional off-chain storage modules. Specifically, we design an adapted blockchain structure and propose a consensus process integrated with the sharing process to manage off-chain behavior and prevent delivery repudiation. We also incorporate a ciphertext policy into the sharing process to ensure the integrity and confidentiality of the shared file. Additionally, we propose a watermarking protocol in conjunction with blockchain records to hold unauthorized disclosure behavior accountable. Our scheme extends the management scope of blockchain to off-chain and achieves 32x, 19x, and 1.48x higher throughput than Bitcoin, Ethereum, and Fabric, respectively.

Abstract Edge‐device‐based object detection is crucial in many real‐world applications, such as self‐driving cars, ADAS, driver behavior analysis. Although deep learning (DL) has become the de‐facto approach for object detection, the limited computing resources of embedded devices and the large model size of current DL‐based methods increase the difficulty of real‐time object detection on edge devices. To overcome these difficulties, in this work a novel YOLOv4‐dense model is proposed to detect objects in an accurate, fast manner, which is built on top of the YOLOv4 framework but with substantial improvements. More specifically, lots of CSP layers are pruned since it will decrease inference speed. And to address the losing small objects problem, a dense block is introduced. In addition, a lightweight two‐stream YOLO head is also designed to further reduce the computational complexity of the model. Experimental results on NVIDIA JETSON TX2 embedded platform demonstrate that YOLOv4‐dense can achieve a higher accuracy, faster speed with smaller model size. For instance, on the KITTI dataset, YOLOv4‐dense obtains 84.3% mAP and 22.6 FPS with only 20.3 M parameters, surpassing the state‐of‐the‐art models with comparable parameter budget such as YOLOv3‐tiny, YOLOv4‐tiny, PP‐YOLO‐tiny by a large margin.

According to the surveys of the World Health Organization, distracted driving is one of main causes of road traffic accidents. To improve road traffic safety, real-time detection of drivers’ driving behavior is very important for the development of highly reliable Advanced Driver Assistance System (ADAS). At present, the deep learning architecture based on a Convolutional Neural Network (CNN) has disadvantages such as large number of parameters and weak global feature extraction ability. Therefore, this paper proposes an innovative driver distraction detection model based on the fusion of a transformer and a CNN, referred to as FPT, which is the first exploration in the field of driver distraction detection. First, we introduce the latest Twins transformer as a benchmark. Then, we design residual embedding to replace block embedding, which can further integrate the convolutional neural network with Transformer and improve the feature extraction ability. In addition, the Multilayer Perceptron (MLP) module with a large parameter occupancy rate in the original transformer structure is replaced with a lightweight group convolution module to reduce computational complexity. Finally, a cross-entropy loss function for label smoothing is designed to guide network learning with significantly differentiated features. Comparison results on two large-scale driver distraction detection datasets show that the proposed FPT offers a better compromise between computational cost and performance compared to the state-of-the-art CNN and Transformer architectures.

Distracted driver classification (DDC) plays an important role in ensuring driving safety. Although many datasets are introduced to support the study of DDC, most of them are small in data size and are short of diversity in environmental variations. This largely limits the development of DDC since many practical problems such as the cross-modality setting cannot be fully studied. In this paper, we introduce 100-Driver, a large-scale, diverse posture-based distracted diver dataset, with more than 470K images taken by 4 cameras observing 100 drivers over 79 hours from 5 vehicles. 100-Driver involves different types of variations that closely meet real-world applications, including changes in the vehicle, person, camera view, lighting, and modality. We provide a detailed analysis of 100-Driver and present 4 settings for investigating practical problems of DDC, including the traditional setting without domain shift and 3 challenging settings (i.e., cross-modality, cross-view, and cross-vehicle) with domain shifts. We conduct comprehensive experiments on these 4 settings with state-the-of-art techniques and show several insights to the future study of DDC. Our 100-Driver will be publicly available offering new opportunities to advance the development of DDC. The 100-driver dataset, source code, and evaluation protocols are available athttps://100-driver.github.io.

Automatic car damage detection has attracted significant attention in the car insurance business. However, due to the lack of high-quality and publicly available datasets, we can hardly learn a feasible model for car damage detection. To this end, we contribute with Car Damage Detection (CarDD), the first public large-scale dataset designed for vision-based car damage detection and segmentation. Our CarDD contains 4,000 high-resolution car damage images with over 9,000 well-annotated instances of six damage categories. We detail the image collection, selection, and annotation processes, and present a statistical dataset analysis. Furthermore, we conduct extensive experiments on CarDD with state-of-the-art deep methods for different tasks and provide comprehensive analyses to highlight the specialty of car damage detection. CarDD dataset and the source code are available athttps://cardd-ustc.github.io.

Traffic accidents caused by distracted drivers account for a large proportion of traffic accidents each year, and monitoring the driving state of drivers to avoid traffic accidents caused by distracted driving has become a very important research direction. At present, the field of driver distraction detection mainly adopts supervised learning methods, which have problems such as poor generalization ability, large labeling cost, and weak artificial intelligence. This paper is oriented toward driver distraction fine-grained detection and innovatively proposes a new unsupervised deep learning algorithm, which is referred to as UDL, to achieve a more human-like level of intelligence. First, we build a new unsupervised deep learning algorithm; furthermore, we integrate the multilayer perceptron (MLP) architecture to build a new backbone and projection head to strengthen feature extraction capabilities; and finally, a new loss function based on contrast learning and a stop-gradient strategy is designed to guide the model to learn more robust features. The comparison results on large-scale driver distraction detection datasets show that our UDL method can accurately detect driver distraction without labels and exhibits excellent generalization performance with a linear evaluation accuracy of 97.38%; In addition, after fine-tuning with fewer labels, our UDL method can achieve superior performance close to state-of-the-art supervised learning methods, achieving 99.07% accuracy after fine-tuning using only 50% of the labeled data, which greatly reduces the cost and limitations of manual annotation.

For deployment on an embedded processor for distracted driver classification, the model should satisfy the demand for both high accuracy, real-time inference, and limited storage resources. Conventional deep CNN models such as VGG, ResNet, DenseNet, often aim for high accuracy, making their model heavy for an embedded system with limited memory space and computing resources. In contrast, lightweight models are greatly compressed but at a significant sacrifice of accuracy. To bridge this gap, we propose an instance-specific multi-teacher knowledge distillation model (IsMt-KD) to learn more accurate, speedy, and lightweight CNNs for distracted driver posture classification. Specifically, in multi-teacher knowledge distillation, most of the current approaches either randomly select a teacher model and apply the prediction of such teacher model as the soft-label or allocate an equal weight to every teacher model and average all the predictions of the teachers as the soft label. In this paper, we observe that, when facing the same instance, the outputs of different teachers vary greatly, in which some teachers can predict it right whereas the others may give pretty high probabilities to the irrelevant classes. Thus, it is inappropriate to set fixed weights or the same weights for teachers. To this end, a simple yet effective instance-specific teacher grading module is designed to dynamically assign weights to teacher models based on individual instances. In this way, we can dynamically distill the knowledge from multiple teachers by considering both instance-specific high-level and instance-specific intermediate-level information. Our extensive experimental results on AUC and StateFarm datasets, and our implementation on edge hardware platforms including HUAWEI MediaPad c5 and Nvidia Jetson TX2, verify the effectiveness and feasibility of our approach.

This paper considers a novel problem, named One-View Learning (OVL), in human retrieval a.k.a. person re-identification (re-ID). Unlike fully-supervised learning, OVL only requires pretty cheap annotation cost: labeled training images are only provided from one camera view (source view/domain), while the annotations of training images from other camera views (target views/domains) are not available. OVL is a problem of multi-target open set domain adaptation that is difficult for existing domain adaptation methods to handle. This is because 1) unlabeled samples are drawn from multiple target views in different distributions, and 2) the target views may contain samples of “unknown identity” that are not shared by the source view. To address this problem, this work introduces a novel one-view learning framework for person re-ID. This is achieved by adversarial multi-view learning (AMVL) and adversarial unknown rejection learning (AURL). The former learns a multi-view discriminator by adversarial learning to align the feature distributions between all views. The later is designed to reject unknown samples from target views through adversarial learning with two unknown identity classifiers. Extensive experiments on three large-scale datasets demonstrate the advantage of the proposed method over state-of-the-art domain adaptation and semi-supervised methods.

A large number of studies have shown that most vehicle collisions are caused by drivers' abnormal operations. To ensure the safety of all people on the road network as much as possible, it is crucial to be able to predict the drivers' driving safety risks in real time. In this paper, we propose a novel cost-sensitive L1/L2-nonnegativity-constrained deep autoencoder network for driving safety risk prediction. Unfortunately, with existing research methods, the size of the sliding time window is too large, the feature extraction is relatively subjective, and class imbalances occur, which leads to low identification accuracy, long prediction times, and poor applicability. We first propose using a three-layer L1/L2-nonnegativity-constrained autoencoder to adaptively search the optimal size of the sliding window and then construct a deep L1/L2-nonnegativity-constrained autoencoder network to automatically extract the hidden features of the driving behaviors. Finally, we build a new L1/L2-nonnegativityconstrained focal loss classifier to predict the driving behaviors under different safety risk levels. The results from the public 100-Car naturalistic driving study dataset indicate that our method can effectively find the optimal window size, reduce the data volume and reconstruction error, and extract more distinctive features. Furthermore, this method effectively curbs the class imbalance, improves the driving safety risk prediction performance, reduces overfitting, shortens the prediction time, and improves the timeliness.

The use of pen as an effective communication interface to computer becomes an active research area in recent years. However, the lack of complete solutions for interchange between platforms has hampered its application as no efficient device can record the entire procedure of human handwriting behavior. In this paper, we design a force Tablet (F-Tablet) for human-computer-interaction (HCI), which can acquire the kinematics and kinetics information of human handwriting, including strokes of pen-up and pen-down, pen nib trajectory and three-axis forces of pen tip directly and simultaneously. Any stylus- and pen-like device can be used to write on it. The core part of the system, named as F-Tablet, is introduced and its ink was defined according to InkML format. An improved DTW (dynamic time warping) algorithm is also put forward to verify the online signatures based on the digital handwriting forces vector ink. The iterative experiment is introduced to decide weights for writing forces in different direction and the classification threshold

External force at the end-effector is often required for the force control of robotic manipulator. In some cases, the force information measured by wrist force sensor consists of the external force and the undesired inertial force arising from the acceleration of the end-effector. Up to now, it is still difficult to extract the external force exactly for the insufficiency of acceleration information. A novel six-axis accelerometer in the type of dual annular membranes structure is presented in this paper. It can simultaneously measure all three linear acceleration components ax, ay, az and three angular acceleration components αx, αy, αz, which can be used to extract the external forces. The sensor structure and its sensing principle are described. The rated strains and interference strains obtained from Finite Element Method (FEM) simulations indicate that the accelerometer has a low level cross-sensitivity. The characteristic experiments reveal that the experimental sensitivities are in good correspondence with the results of the FEM simulations. It also shows that this accelerometer has a good linearity and minor interference errors as well as principle errors.

The mechanism of the sensory-to-motor transformation as well as motor-to-sensory transformation of human beings has attracted much attention in recent years. As no efficient device can record hand intrinsic behavior, it is difficult to get its neuro-physiological models of sensory-motor coordination. In this paper, we offer a system to acquire the kinematics and kinetics information of human hand movement through handwriting. Joined with human being, an eye-brain-hand to computer (EBH-C) interaction system is presented. In this human-in-the-loop-testing system, human beings acquire image, voice or text from computer by eyes or ears then write them down. The core part of the system, named as F-Tablet/spl trade/ is able to acquire the trajectory and three-axis forces of pen-tip directly and simultaneously. With the help of this system, we designed an experiment to evaluate the handwriting movements and forces controlling ability of different ages. Some experiment results were present. Aided with conventional analysis of electroencephalography (EEG) and magnetoencephalography (MEG), the whole procedures of information transmitting, acquired by eyes and ears, processed by brain, outputted and actuated by hand, can be recorded.