Reliable identification of gunshot events is crucial for reducing gun violence and enhancing public safety. However, current gunshot detection and recognition methods are still affected by complex shooting scenarios, various nongunshot events, diverse firearm types, and scarce gunshot datasets. To address these issues, based on triaxial acceleration of guns, a novel general deep transfer learning approach is proposed for gunshot detection and recognition, which combines a temporal deep learning model with transfer learning and automated machine learning (AutoML) to improve the accuracy, reliability and generalization performance. First, a new gunshot recognition model named as MobileNetTime is proposed for the two-class gunshot event detection, three-class coarse firearm recognition, and 15-class fine firearm recognition, which utilizes 1-D convolution and inverted residual modules to autonomously extract higher-level features from the time series acceleration data. Second, considering the impact of nongunshot events, the AutoML is employed for model fine tuning, to transfer the pretrained MobileNetTime from the handgun to various firearm types. In addition, we propose a low-power versatile gunshot recognition system framework employing a triaxial accelerometer for both of wrist-worn and gun-embedded scenarios, which adopts a two-stage wake-up mechanism that selectively monitors gunshot events using temporal and spectral energy features. The experimental results on the two gunshot datasets DGUWA and GRD show that the proposed model can achieve up to 100% accuracy on the DGUWA dataset and 98.98% accuracy on the GRD dataset for the two-class gunshot detection. Moreover, the proposed deep transfer learning approach achieves a 98.98% accuracy for 16-class firearm classification, which is 6.21% higher than the model without transfer learning.

Gun violence and misuse pose great threat to the public safety. Real-time monitoring of gun usage and gunshot events are very promising for effective gun control. However, most available monitoring systems are installed in a fixed location instead of the guns, which greatly limits the flexibility and coverage. In this study, we propose a wireless gun monitoring and gunshot recognition system based on a low-cost triaxial acceleration sensor, which can monitor the gun in real time and accurately recognize gunshot events. Addressing the limited resources of the embedded systems, we further propose an efficient gunshot recognition algorithm EfficientNetTime that combines the lightweight neural network and knowledge distillation, so as to enable the deployment on embedded devices. First, a novel lightweight deep learning model is proposed as the basic model, which combines the advantages of 1-D convolution and depthwise separable convolution to effectively characterize the gunshot signal while decreasing the computing cost of convolution. Second, using the knowledge distillation, EfficientNetTime is used as the teacher model to generate a compressed student model that maintains accuracy and greatly reducing model size. Finally, the EfficientNetTime student model can be deployed on resource-limited embedded systems. The proposed method can automatically extract features for end-to-end recognition and is robust to temporal transformations of input signals. Using a publicly available gunshot data set, the proposed EfficientNetTime model is verified and compared against the state-of-the-art models. Experimental results demonstrate that the EfficientNetTime model surpasses other gunshot recognition methods in terms of the accuracy and model size.

In general, vehicle images have varying resolutions due to vehicles’ movements and different camera settings. However, most existing vehicle reidentification models are single-resolution deep networks trained with preuniformly resizing vehicle images, which underestimate adverse effects of varying resolutions and lead to unsatisfactory performance. A straightforward solution for dealing with varying resolutions is to train multiple vehicle reidentification models. Each model is independently trained with images of a specific resolution. However, this straightforward solution requires significant overhead and ignores intrinsic associations among different resolution images. For that, an efficient multiresolution network (EMRN) is proposed for vehicle reidentification in this article. First, EMRN embeds a newly designed multiresolution feature dimension uniform module (MR-FDUM) behind a traditional backbone network (i.e., ResNet-50). As a result, the whole model can extract fixed dimensional features from different resolution images so that it can be trained with one loss function of fixed dimensional parameters rather than training multiple models. Second, a multiresolution image randomly feeding strategy is designed to train EMRN, making each minibatch data of a random resolution during the training process. Consequently, EMRN can implicitly learn collaborative multiresolution features via only a unitary deep network. The experiments on three large-scale data sets, i.e., VeRi776, VehicleID, and VRIC, demonstrate that EMRN is superior to state-of-the-art vehicle reidentification methods.

Liquid crystal display (LCD) defects detection on module level is increasingly important for flat-panel displays (FPD) industry to increase the production capacity via machine vision technology. However, it is an overwhelmingly challenging issue due to various difficulties. This article discloses a practical automatic optical inspection (AOI) system consisting of hardware structure and software algorithm to detect module-level defects. The AOI system is the core component to build a distributed integrated inspection system with the help of the Internet of Things (IoT). Starting from the analysis of the challenges encountered in module-level defects inspection, a delicate photograph scheme is proposed to reveal different kinds of defects. In order to robustly work on the module-level defects detection with complex situations, a novel framework based on YOLOV3 detection unit is proposed in this article, including the preprocessing module, detection module, defects definition module, and interferences elimination module. To the best of our knowledge, this is the first work that designs a practical AOI system for module-level defects detection. In order to demonstrate the effectiveness of the proposed method, extensive experiments have been conducted on the manufacturing lines. The evaluation of the detection performance of the AOI system in comparison with a manual scheme indicates that the proposed system is practical for module-level defects detection. Currently, the proposed system has been deployed in a real-world LCD manufacturing line from a major player in the world.

Existing vehicle re-identification methods commonly use spatial pooling operations to aggregate feature maps extracted via off-the-shelf backbone networks, such as visual geometry group network (VGGNet), Google network (GoogLeNet) and residual network (ResNet). They ignore exploring the spatial significance of feature maps, eventually degrading the vehicle re-identification performance. In this paper, firstly, an innovative spatial graph network (SGN) is proposed to elaborately explore the spatial significance of feature maps. The SGN stacks multiple spatial graphs (SGs). Each SG assigns feature map’s elements as nodes and utilizes spatial neighborhood relationships to determine edges among nodes. During the SGN’s propagation, each node and its spatial neighbors on an SG are aggregated to the next SG. On the next SG, each aggregated node is re-weighted with a learnable parameter to find the significance at the corresponding location. Secondly, a novel pyramidal graph network (PGN) is designed to comprehensively explore the spatial significance of feature maps at multiple scales. The PGN organizes multiple SGNs in a pyramidal manner and makes each SGN handles feature maps of a specific scale. Finally, a hybrid pyramidal graph network (HPGN) is developed by embedding the PGN behind a ResNet-50 based backbone network. Extensive experiments on three large scale vehicle databases (i.e., VeRi776, VehicleID, and VeRi-Wild) demonstrate that the proposed HPGN is superior to state-of-the-art vehicle re-identification approaches in terms of accuracy, parameter cost, and computation cost. In addition, experiments show that the proposed PGN is universal to various backbone networks.

Fog computing is promising to solve the challenge caused by an extremely large amount of data on cloud computing. In this study, an efficient binary convolutional neural network with numerous skip connections (BNSC-Net) is proposed for fog computing to enable real-time smart industrial applications. This network features decomposition convolution kernels and concatenated feature maps. Moreover, the network performance is further improved through expanding the update interval of the straight-through estimator. To verify the performance, BNSC-Net is tested on two broadly used public data sets: 1) ImageNet and 2) CIFAR-10. An ablation study is first conducted to verify the effectiveness of the proposed improved operations, and results demonstrate that BNSC-Net can obviously increase the classification accuracy for both data sets. ImageNet-based classification results indicate that BNSC-Net can achieve 59.9% TOP-1 accuracy that is 2.6% higher than the state-of-the-art binary neural networks, such as projection convolutional neural networks (PCNNs). Finally, a subset with ten classes is selected from ImageNet to simulate the data collected in the smart industry with limited categories, based on which BNSC-Net also demonstrates an impressive classification performance with friendly memory and calculation requirements. Particularly, the receiver operating characteristic curves of BNSC-Net surpass that of the state-of-the-art algorithm DeepIns. Therefore, the proposed BNSC-Net is effective and efficient for building deep learning-enabled industrial applications on fog nodes.

In online customer service applications, multiple chatbots that are specialized in various topics are typically developed separately and are then merged with other human agents to a single platform, presenting to the users with a unified interface.Ideally the conversation can be transparently transferred between different sources of customer support so that domain-specific questions can be answered timely and this is what we coined as a Bot-Agent symbiosis.Conversation transition is a major challenge in such online customer service and our work formalises the challenge as two core problems, namely, when to transfer and which bot or agent to transfer to and introduces a deep neural networks based approach that addresses these problems.Inspired by the net promoter score (NPS), our research reveals how the problems can be effectively solved by providing user feedback and developing deep neural networks that predict the conversation category distribution and the NPS of the dialogues.Experiments on realistic data generated from an online service support platform demonstrate that the proposed approach outperforms state-of-the-art methods and shows promising perspective for transparent conversation transition.

Object reidentification with the goal of matching pedestrian or vehicle images captured from different camera viewpoints is of considerable significance to public security. Quadruple directional deep learning features (QD-DLFs) can comprehensively describe object images. However, the correlation among QD-DLFs is an unavoidable problem, since QD-DLFs are learned with quadruple independent directional deep networks (QIDDNs) driven with the same training data, and each network holds the same basic deep feature learning architecture (BDFLA). The correlation among QD-DLFs is harmful to the complementarity of QD-DLFs, restricting the object reidentification performance. For that, we propose joint quadruple decorrelation directional deep networks (JQD3Ns) to reduce the correlation among the learned QD-DLFs. In order to jointly train JQD3Ns, besides the softmax loss functions, a parameter correlation cost function is proposed to indirectly reduce the correlation among QD-DLFs by enlarging the dissimilarity among the parameters of JQD3Ns. Extensive experiments on three publicly available large-scale data sets demonstrate that the proposed JQD3Ns approach is superior to multiple state-of-the-art object reidentification methods.

In recent years, deep learning (DL) has been successfully and widely applied in the person reidentification (Re-ID). However, the DL-based person Re-ID methods face a bottleneck that the scales of most existing person Re-ID databases are not large enough for training very deep models. To address this problem, a body symmetry and part-locality-guided direct nonparametric deep feature enhancement (DNDFE) method is proposed in this article. Based on the observation that the body symmetry and part locality are two important appearance properties inherited in the upright walking persons, the proposed method designs two nonparametric layers, namely, the body symmetry average pooling and local normalization layers, to construct a DNDFE module to well explore the body symmetry and part locality properties. The proposed DNDFE module could be directly embedded between the traditional deep feature learning module and similarity learning module to enhance the DL features so as to improve the person Re-ID performance. The experimental results have shown that the proposed DNDFE method is superior to multiple state-of-the-art person Re-ID methods in terms of accuracy and efficiency.

In order to resist the adverse effect of viewpoint variations, we design quadruple directional deep learning networks to extract quadruple directional deep learning features (QD-DLF) of vehicle images for improving vehicle re-identification performance. The quadruple directional deep learning networks are of similar overall architecture, including the same basic deep learning architecture but different directional feature pooling layers. Specifically, the same basic deep learning architecture that is a shortly and densely connected convolutional neural network is utilized to extract the basic feature maps of an input square vehicle image in the first stage. Then, the quadruple directional deep learning networks utilize different directional pooling layers, i.e., horizontal average pooling layer, vertical average pooling layer, diagonal average pooling layer, and anti-diagonal average pooling layer, to compress the basic feature maps into horizontal, vertical, diagonal, and anti-diagonal directional feature maps, respectively. Finally, these directional feature maps are spatially normalized and concatenated together as a quadruple directional deep learning feature for vehicle re-identification. The extensive experiments on both VeRi and VehicleID databases show that the proposed QD-DLF approach outperforms multiple state-of-the-art vehicle re-identification methods.

Nowadays more and more urban residents are aware of the importance of the air quality to their health, especially who are living in the large cities that are seriously threatened by air pollution. Meanwhile, being limited by the spare sense nodes, the air quality information is very coarse in resolution, which brings urgent demands for high-resolution air quality data acquisition. In this paper, we refer the real-time and fine-gained air quality data in city-scale by employing the crowdsource automobiles as well as their built-in sensors, which significantly improves the sensing system's feasibility and practicability. The main idea of this paper is motivated by that the air component concentration within a vehicle is very similar to that of its nearby environment when the vehicle's windows are open, given the fact that the air will exchange between the inside and outside of the vehicle though the opening window. Therefore, this paper first develops an intelligent algorithm to detect vehicular air exchange state, then extracts the concentration of pollutant in the condition that the concentration trend is convergent after opening the windows, finally, the sensed convergent value is denoted as the equivalent air quality level of the surrounding environment. Based on our Internet of Things cloud platform, real-time air quality data streams from all over the city are collected and analyzed in our data center, and then a fine-gained city level air quality map can be exhibited elaborately. In order to demonstrate the effective- ness of the proposed method, experiments crowdsourcing 500 floating vehicles are conducted in Beijing city for three months to ubiquitously sample the air quality data. Evaluations of the algorithm's performance in comparison with the ground truth indicate the proposed system is practical for collecting air quality data in urban environments.