As machine learning models become increasingly prevalent in motion forecasting for autonomous vehicles (AVs), it is critical to ensure that model predictions are safe and reliable. In this paper, we examine the robustness of motion forecasting to non-causal perturbations. We construct a new benchmark for evaluating and improving model robustness by applying perturbations to existing data. Specifically, we conduct an extensive labeling effort to identify causal agents, or agents whose presence influences human drivers’ behavior, in the Waymo Open Motion Dataset (WOMD), and we use these labels to perturb the data by deleting non-causal agents from the scene. We evaluate a diverse set of state-of-the-art deep-learning models on our proposed benchmark and find that all evaluated models exhibit large shifts under non-causal perturbation: we observe a surprising 25-38% relative change in minADE as compared to the original. In addition, we investigate techniques to improve model robustness, including increasing the training dataset size and using targeted data augmentations that randomly drop non-causal agents throughout training. Finally, we release the causal agent labels as an extension to WOMD and the robustness benchmarks to aid the community in building more reliable and safe deep-learning models for motion forecasting1.

In wastewater treatment processes, the monitoring of dissolved oxygen sensor is the key to ensure the quality of effluent. In this paper, a method for fault detection of dissolved oxygen sensor is proposed using set membership identification and radial basis function(RBF) neural network. The time series model of KLa5 is built by RBF neural network in virtue of its universal approximation ability. Considering the bounded modeling error, the set description of the output weights of the network is obtained by linear-in-parameters set membership identification algorithm. This built model can give a one-step prediction of the confidence interval of KLa5 under the fault-free case. If the real of KLa5 exceeds the predicted confidence interval, a failure of the dissolved oxygen sensor can be determined.

We introduce collaborative learning in which multiple classifier heads of the same network are simultaneously trained on the same training data to improve generalization and robustness to label noise with no extra inference cost. It acquires the strengths from auxiliary training, multi-task learning and knowledge distillation. There are two important mechanisms involved in collaborative learning. First, the consensus of multiple views from different classifier heads on the same example provides supplementary information as well as regularization to each classifier, thereby improving generalization. Second, intermediate-level representation (ILR) sharing with backpropagation rescaling aggregates the gradient flows from all heads, which not only reduces training computational complexity, but also facilitates supervision to the shared layers. The empirical results on CIFAR and ImageNet datasets demonstrate that deep neural networks learned as a group in a collaborative way significantly reduce the generalization error and increase the robustness to label noise.

When searching on the web or in an app, results are often returned as lists of hundreds to thousands of items, making it difficult for users to understand or navigate the space of results. Research has demonstrated that using clustering to partition search results into coherent, topical clusters can aid in both exploration and discovery. Yet clusters generated by an algorithm for this purpose are often of poor quality and do not satisfy users. To achieve acceptable clustered search results, experts must manually evaluate and refine the clustered results for each search query, a process that does not scale to large numbers of search queries. In this article, we investigate using crowd-based human evaluation to inspect, evaluate, and improve clusters to create high-quality clustered search results at scale. We introduce a workflow that begins by using a collection of well-known clustering algorithms to produce a set of clustered search results for a given query. Then, we use crowd workers to holistically assess the quality of each clustered search result to find the best one. Finally, the workflow has the crowd spot and fix problems in the best result to produce a final output. We evaluate this workflow on 120 top search queries from the Google Play Store, some of whom have clustered search results as a result of evaluations and refinements by experts. Our evaluations demonstrate that the workflow is effective at reproducing the evaluation of expert judges and also improves clusters in a way that agrees with experts and crowds alike.

Fluorescence molecular tomography (FMT), as a promising imaging modality in preclinical research, can obtain the three-dimensional (3-D) position information of the stem cell in mice. However, because of the ill-posed nature and sensitivity to noise of the inverse problem, it is a challenge to develop a robust reconstruction method, which can accurately locate the stem cells and define the distribution. In this paper, we proposed a sparsity adaptive correntropy matching pursuit (SACMP) method. SACMP method is independent on the noise distribution of measurements and it assigns small weights on severely corrupted entries of data and large weights on clean ones adaptively. These properties make it more suitable for in vivo experiment. To analyze the performance in terms of robustness and practicability of SACMP, we conducted numerical simulation and in vivo mice experiments. The results demonstrated that the SACMP method obtained the highest robustness and accuracy in locating stem cells and depicting stem cell distribution compared with stagewise orthogonal matching pursuit and sparsity adaptive subspace pursuit reconstruction methods. To the best of our knowledge, this is the first study that acquired such accurate and robust FMT distribution reconstruction for stem cell tracking in mice brain. This promotes the application of FMT in locating stem cell and distribution reconstruction in practical mice brain injury models.

Restricted Boltzmann Machine is a fundamental method in deep learning networks. Training and generalization is an ill-defined problem in that many different networks may achieve the training goal; however each will respond differently to an unknown input. Traditional approaches include stopping the training early and/or restricting the size of the network These approaches ameliorate the problem of over-fitting where the network learns the patterns presented but is unable to generalize. Bayesian regularization addresses these issues by requiring the weights of the network to attain a minimum magnitude. This ensures that non-contributing weights are reduced significantly and the resulting network represents the essence of the inter-relations of the training. Bayesian Regularization simply introduces an additional term to the objective function. This term comprises the sum of the squares of the weights. The optimization process therefore not only achieves the objective of the original cost (i.e. the minimization of an error metric) but it also ensures that this objective is achieved with minimum-magnitude weights. We have introduced Bayesian Regularization in the training of Restricted Boltzmann Machines and have applied this method in experiments of hand-written numbers classification. Our experiments showed that by adding Bayesian regularization in the training of RBMs, we were able to improve the generalization capabilities of the trained network by reducing its recognition errors by more than 1.6%.

We describe preliminary results of experiments with an unsupervised framework for query segmentation, transforming keyword queries into structured queries. The resulting queries can be used to more accurately search product databases, and potentially improve result presentation and query suggestion. The key to developing an accurate and scalable system for this task is to train a query segmentation or attribute detection system over labeled data, which can be acquired automatically from query and click-through logs. The main contribution of our work is a new method to automatically acquire such training data - resulting in significantly higher segmentation performance, compared to previously reported methods.

Automatic detection of anomalies in financial statements can decrease the risk of exposure to fraudulent corporate behavior. This paper proposes a method to convert fraud classification rules learned from a genetic algorithm to a fuzzy score representing the degree to which a company's financial statements match those rules. Applying the method to financial data in real time can lead to the early detection of potentially fraudulent corporate behavior.

A musical piece typically has repetitive structures. Analysis of this structure can be used for music indexing, thumbnailing or segmentation. The described research aims at automatically analyzing the repetitive structure of musical signals. First, we detect the repetition of each segment in a piece using dynamic programming. Second, we summarize this repetition information and infer the structure based on some heuristic rules. The performance of our approach is demonstrated visually using figures for qualitative evaluation, and by two structural similarity measures for quantitative evaluation. The experimental results using a corpus of Beatles' songs show that automatic structural analysis of music is possible.

Music thumbnailing (or music summarization) aims at finding the most representative part of a song, which can be used for web browsing, web searching and music recommendation. Three strategies are proposed in this paper for automatically generating the thumbnails of music. All the strategies are based on the results of music structural analysis, which identifies the recurrent structure of musical signals. Instead of being evaluated subjectively, the generated thumbnails are evaluated by several criteria, mainly based on previous human experiments on music thumbnailing and the properties of thumbnails used for commercial web sites. Additionally, the performance of the structural analysis is demonstrated visually using figures for qualitative evaluation, and by three novel structural similarity metrics for quantitative evaluation. The preliminary results obtained using a corpus of Beatles' songs demonstrate the promise of our method and suggest that different thumbnailing strategies might be proper for different applications.