Internet of Things (IoT) devices have become increasingly prevalent, and while convenient they pose security risks and must be monitored to keep networks safe. The problem of identifying IoT devices by fingerprinting their network traffic has been studied, with various approaches emerging. While achieving good results, many solutions require complex feature extraction, considerable computational overhead, and extensive domain knowledge in both networking and machine learning to select relevant features and supporting ML algorithms. In addition, many current studies work to identify heterogeneous devices in an artificially sterile (lab) environment. To improve the process we introduce FlexHash, a system that uses a combination of locality-sensitive hashing and machine learning to identify specific devices based on a generic view of their network traffic characteristics. We successfully identify both heterogeneous and homogeneous (identical) devices in an environment with both live network noise and noise generated from other (unknown) IoT devices. FlexHash is able to consume unprocessed network traffic in the form of .pcap files and produce feature vectors capable of highly accurate device identification and anomaly detection. To enhance the strength of this approach we develop our own n-gram based hashing method allowing for various parameters in the algorithm to be tuned, making it possible to accurately differentiate individual devices from among a field of identical peers as well as device genre and heterogeneous devices with a single packet of network traffic.

Recent growth in the utilization of IoT has offered convenience and utility, but has also increased security risk. Many devices lack the capacity to support adequate encryption or other common means of protection, and are often designed for easy connection out-of-the-box exposing vulnerabilities related to default configurations. Managing IoT devices in a network can be difficult as MAC addresses are easily spoofed, creating a need for techniques to properly identify and monitor membership. Many of the proposed solutions for IoT device identification require complex feature extraction and engineering. In addition, little work has been done to identify individual devices from among identical peers. We propose a novel hashing algorithm, FlexHash, and show that we are able to identify identical devices with a very high degree of accuracy using only a single packet of network traffic. By applying hybrid locality sensitive hashing in combination with machine learning our approach achieves accuracy scores as high as 98% for identical devices and 99% for identifying device genre.

Recent advances in AI indicate that the future of cybersecurity workforce development lies in professionals working in Human-AI teams to defend online resources from opposing Human-AI teams of malicious attackers. However, there is little research on how human biases and attitudes affects the performance of human-AI teams in cybersecurity. To help explore this new research area, we describe a simulation game that helps students (future professionals) understand the concept of firewalls while enabling us to probe attitudes towards cybersecurity and AI, as well as trust and cooperation in HumanAI teams. Early study prototyping results indicate that students prefer an AI-teammate over a human in this simulation game setting. In addition, students seem to engage well with the game play, pointing towards this research platform’s suitability for exploring trust and cooperation in human-AI teams for game-based cybersecurity training, and to support our prior results on differing perspectives on cybersecrity risk.

Caricatures, with their exaggerated features, offer an efficient means for individuals to recognize each other compared to veridical (real) images. However, matching veridical images to caricatures remains a challenging task in machine learning. This difficulty stems from poor quality caricature datasets lacking clear labels, inadequate labeling in widely used veridical image datasets like CelebA, and a shift away from attribute-based representations due to the rise of neural networks. These issues significantly impact the accuracy of face verification tasks between caricatures and veridical images. To address these challenges, this paper introduces a classification protocol for prominent facial feature recognition and a verification protocol for matching celebrity veridical images to their caricatures. We utilize CarVer, a recently curated dataset comprising both veridical and caricature images accompanied by detailed prominent attribute labels. Our approach aims to develop a set of prominent facial attributes that can effectively represent both real and caricatured images, enabling improved face verification across these modalities. This research has potential applications across various industries where robust cross-modal face recognition is crucial.

Research in psychology and neuroscience point to the potential of caricatures to improve human face recognition in non-ideal settings. Face verification using caricatures is the first step in using caricatures in this way, which involves verifying pairs of faces where one is veridical (realistic) and the other is a caricature. Existing biometric research in the area of caricature face verification is recent and limited by a lack of well-constructed, large-scale datasets. We show that the largest available dataset, WebCaricature, falls short of acceptable standards of image quality and representation necessary for face verification. We present a new caricature dataset, CarVer, and introduce a face verification evaluation standard to replace WebCaricature’s. We find that the CarVer dataset (F1=0.82) outperforms WebCaricature (F1=0.76) using WebCaricature’s Bounding Box alignment method. Additionally, our new alignment methods, Enlarged Bounding Box (F1=0.82) and Revised Bounding Box (F1=0.80) outperform WebCaricature’s Bounding Box (F1=0.72) and Eye Location (F1=0.74) alignment methods.

CelebA is the most common and largest scale dataset used to evaluate methods for facial attribute prediction, an important benchmark in imbalanced classification and face analysis. However, we argue that the evaluation metrics and baseline models currently used to compare the performance of different methods are insufficient for determining which approaches are best at classifying highly imbalanced attributes. We are able to obtain results comparable to current state-of-the-art using a ResNet-18 model trained with binary cross-entropy, a substantially less sophisticated approach than related work. We also show that we can obtain near-state-of-the-art results on accuracy using a model trained with just 10% of CelebA, and on balanced accuracy simply by maximizing recall for imbalanced attributes at the expense of all other metrics. To deal with these issues, we suggest several improvements to model evaluation including better metrics, stronger baselines, and increased awareness of the limitations of the dataset.

One significant challenge in the field of supervised deep learning is the lack of large-scale labeled datasets for many problems. In this paper, we propose Consensus Spectral Clustering (CSC), which leverages the strengths of convolutional autoencoders and spectral clustering to provide pseudo labels for image data. This data can be used as weakly-labeled data for training and evaluating classifiers which require supervision. The primary weaknesses of previous works lies in their inability to isolate the object of interest in an image and cluster similar images together. We address these issues by denoising input images to remove pixels which do not contain data pertinent to the target. Additionally, we introduce a voting method for label selection to improve the clustering results. Our extensive experimentation on several benchmark datasets demonstrates that the proposed CSC method achieves competitive performance with state-of-the-art methods.

Research in visual anomaly detection draws much interest due to its applications in surveillance. Common datasets for evaluation are constructed using a stationary camera overlooking a region of interest. Previous research has shown promising results in detecting spatial as well as temporal anomalies in these settings. The advent of self-driving cars provides an opportunity to apply visual anomaly detection in a more dynamic application yet no dataset exists in this type of environment. This paper presents a new anomaly detection dataset the Highway Traffic Anomaly (HTA) dataset- for the problem of detecting anomalous traffic patterns from dash cam videos of vehicles on highways. We evaluate state-of-the-art deep learning anomaly detection models and propose novel variations to these methods. Our results show that state-of-the-art models built for settings with a stationary camera do not translate well to a more dynamic environment. The proposed variations to these SoTA methods show promising results on the new HTA dataset.

In this work we present a new methodology on learning-based path planning for autonomous exploration of subterranean environments using aerial robots. Utilizing a recently proposed graph-based path planner as a "training expert" and following an approach relying on the concepts of imitation learning, we derive a trained policy capable of guiding the robot to autonomously explore underground mine drifts and tunnels. The algorithm utilizes only a short window of range data sampled from the onboard LiDAR and achieves an exploratory behavior similar to that of the training expert with a more than an order of magnitude reduction in computational cost, while simultaneously relaxing the need to maintain a consistent and online reconstructed map of the environment. The trained path planning policy is extensively evaluated both in simulation and experimentally within field tests relating to the autonomous exploration of underground mines.

Attributes are human describable features, which have been used successfully for face, object, and activity recognition. Facial attributes are intuitive descriptions of faces and have proven to be very useful in face recognition and verification. Despite their usefulness, to date there is only one large-scale facial attribute dataset, CelebA. Impressive results have been achieved on this dataset, but it exhibits a variety of very significant biases. As CelebA contains mostly frontal idealized images of celebrities, it is difficult to generalize a model trained on this data for use on another dataset (of non celebrities). A typical approach to dealing with imbalanced data involves sampling the data in order to balance the positive and negative labels, however, with a multi-label problem this becomes a non-trivial task. By sampling to balance one label, we affect the distribution of other labels in the data. To address this problem, we introduce a novel Selective Learning method for deep networks which adaptively balances the data in each batch according to the desired distribution for each label. The bias in CelebA can be corrected for in this way, allowing the network to learn a more robust attribute model. We argue that without this multi-label balancing, the network cannot learn to accurately predict attributes that are poorly represented in CelebA. We demonstrate the effectiveness of our method on the problem of facial attribute prediction on CelebA, LFWA, and the new University of Maryland Attribute Evaluation Dataset (UMD-AED), outperforming the state-of-the-art on each dataset.

Recent research progress in facial attribute recognition has been dominated by small improvements on the only large-scale publicly available benchmark dataset, CelebA [18]. We propose to extend attribute prediction research to unconstrained videos. Applying attribute models trained on CelebA - a still image dataset - to video data highlights several major problems with current models, including the lack of consideration for both time and motion. Many facial attributes (e.g. gender, hair color) should be consistent throughout a video, however, current models do not produce consistent results. We introduce two methods to increase the consistency and accuracy of attribute responses in videos: a temporal coherence constraint, and a motionattention mechanism. Both methods work on weakly labeled data, requiring attribute labels for only one frame in a sequence, which we call the anchor frame. The temporal coherence constraint moves the network responses of non-anchor frames toward the responses of anchor frames for each sequence, resulting in more stable and accurate attribute predictions. We use the motion between anchor and non-anchor video frames as an attention mechanism, discarding the information from parts of the non-anchor frame where no motion occurred. This motion-attention focuses the network on the moving parts of the non-anchor frames (i.e. the face). Since there is no large-scale video dataset labeled with attributes, it is essential for attribute models to be able to learn from weakly labeled data. We demonstrate the effectiveness of the proposed methods by evaluating them on the challenging YouTube Faces video dataset [31]. The proposed motion-attention and temporal coherence methods outperform attribute models trained on CelebA, as well as those fine-tuned on video data. To the best of our knowledge, this paper is the first to address the problem of facial attribute prediction in video.

Attributes, or mid-level semantic features, have gained popularity in the past few years in domains ranging from activity recognition to face verification. Improving the accuracy of attribute classifiers is an important first step in any application which uses these attributes. In most works to date, attributes have been considered independent of each other. However, attributes can be strongly related, such as heavy makeup and wearing lipstick as well as male and goatee and many others. We propose a multi-task deep convolutional neural network (MCNN) with an auxiliary network at the top (AUX) which takes advantage of attribute relationships for improved classification. We call our final network MCNN-AUX. MCNN-AUX uses attribute relationships in three ways: by sharing the lowest layers for all attributes, by sharing the higher layers for spatially-related attributes, and by feeding the attribute scores from MCNN into the AUX network to find score-level relationships. Using MCNN-AUX rather than individual attribute classifiers, we are able to reduce the number of parameters in the network from 64 million to fewer than 16 million and reduce the training time by a factor of 16. We demonstrate the effectiveness of our method by producing results on two challenging publicly available datasets achieving state-of-the-art performance on many attributes.

We introduce the concept of dynamically growing a neural network during training. In particular, an untrainable deep network starts as a trainable shallow network and newly added layers are slowly, organically added during training, thereby increasing the network's depth. This is accomplished by a new layer, which we call DropIn. The DropIn layer starts by passing the output from a previous layer (effectively skipping over the newly added layers), then increasingly including units from the new layers for both feedforward and backpropagation. We show that deep networks, which are untrainable with conventional methods, will converge with DropIn layers interspersed in the architecture. In addition, we demonstrate that DropIn provides regularization during training in an analogous way as dropout. Experiments are described with the MNIST dataset and various expanded LeNet architectures, CIFAR-10 dataset with its architecture expanded from 3 to 11 layers, and on the ImageNet dataset with the AlexNet architecture expanded to 13 layers and the VGG 16-layer architecture.

While Convolutional Neural Networks (CNNs) have recently achieved impressive results on many classification tasks, it is still unclear why they perform so well and how to properly design them. In this work, we investigate the effect of the convolutional depth of a CNN on its generalization performance for binary classification problems. We prove a sufficient condition -polynomial in the depth of the CNN- on the training database size to guarantee such performance. We empirically test our theory on the problem of gender classification and explore the effect of varying the CNN depth, as well as the training distribution and set size.

The NewsStand system enables the use of a map query interface to retrieve news articles associated with the principal locations that they mention collected as a result of monitoring the output of over 10,000 RSS news feeds, made available within minutes of publication. NewsStand has been enhanced to allow using the map query interface to access other information associated with the articles such as photos and videos, as well as names of people and diseases mentioned in these articles. Here we report on our efforts to enhance NewsStand to display the names of brands and to the articles mentioning them. The challenges in identifying interesting brand mentions are discussed.

This paper introduces an advanced rover localization system suitable for autonomous planetary exploration in the absence of Global Positioning System (GPS) infrastructure. Given an existing terrain map (image and elevation) obtained from satellite imagery and the images provided by the rover stereo camera system, the proposed method determines the best rover location through visual odometry, 3D terrain and horizon matching. The system is tested on data retrieved from a 3 km traverse of the Basalt Hills quarry in California where the GPS track is used as ground truth. Experimental results show the system presented here reduces by over 60% the localization error obtained by wheel odometry.

Single camera-based multiple-person tracking is often hindered by difficulties such as occlusion and changes in appearance. In this paper, we address such problems by proposing a robust part-based tracking-by-detection framework. Human detection using part models has become quite popular, yet its extension in tracking has not been fully explored. Our approach learns part-based person-specific SVM classifiers which capture the articulations of the human bodies in dynamically changing appearance and background. With the part-based model, our approach is able to handle partial occlusions in both the detection and the tracking stages. In the detection stage, we select the subset of parts which maximizes the probability of detection, which significantly improves the detection performance in crowded scenes. In the tracking stage, we dynamically handle occlusions by distributing the score of the learned person classifier among its corresponding parts, which allows us to detect and predict partial occlusions, and prevent the performance of the classifiers from being degraded. Extensive experiments using the proposed method on several challenging sequences demonstrate state-of-the-art performance in multiple-people tracking.