Stocks movement direction forecasting has received a lot of attention. Indeed, being able to make accurate forecasts has strong implications on trading strategies. Surprisingly enough little has been published, relatively to the importance of the topic. In this paper, we reviewed how well four classic classification algorithms: random forest, gradient boosted trees, artificial neural network and logistic regression perform in predicting 463 stocks of the S&P 500. Several experiments were conduced to thoroughly study the predictability of these stocks. To validate each prediction algorithm, three schemes we compared: standard cross validation, sequential validation and single validation. As expected, we were not able to predict stocks future prices from their past. However, unexpectedly, we were able to show that taking into account recent information - such as recently closed European and Asian indexes - to predict S&P 500 can lead to a vast increase in predictability. Moreover, we also found out that, among various sectors, financial sector stocks are comparatively more easy to predict than others.

Understanding the irregular crowd movement and social activities caused by urban events such as city festivals and concerts can benefit event management and city planning. Although various urban data can be exploited to detect such irregularities, the crowd mobility data (e.g., bike trip records) are usually in a mixed state with several basic patterns (e.g., eating, working, and recreation), making it difficult to separate concurrent events happening in the same region. The social activity data (e.g., social network check-ins) are usually oversparse, hindering the fine-grained characterization of urban events. In this paper, we propose a tensor cofactorization-based data fusion framework for fine-grained urban event detection and characterization leveraging crowd mobility data and social activity data. First, we adopt a nonnegative tensor cofactorization approach to decompose the crowd mobility tensor into several basic patterns, with the help of the auxiliary social activity tensor. We then use a multivariate-outlier-detection-based method to identify irregularities from the decomposed basic patterns and aggregate them to detect and characterize the associated urban events. We evaluate the performance of our framework using real-world bike trip data and check-in data from New York City and Washington, DC, respectively. Results show that by fusing the two types of urban data, our method achieves fine-grained urban event detection and characterization in both cities and consistently outperforms the baselines.

Bike sharing is booming globally as a green transportation mode, but the occurrence of over-demand stations that have no bikes or docks available greatly affects user experiences. Directly predicting individual over-demand stations to carry out preventive measures is difficult, since the bike usage pattern of a station is highly dynamic and context dependent. In addition, the fact that bike usage pattern is affected not only by common contextual factors (e.g., time and weather) but also by opportunistic contextual factors (e.g., social and traffic events) poses a great challenge. To address these issues, we propose a dynamic cluster-based framework for over-demand prediction. Depending on the context, we construct a weighted correlation network to model the relationship among bike stations, and dynamically group neighboring stations with similar bike usage patterns into clusters. We then adopt Monte Carlo simulation to predict the over-demand probability of each cluster. Evaluation results using real-world data from New York City and Washington, D.C. show that our framework accurately predicts over-demand clusters and outperforms the baseline methods significantly.

Saliency models provide heatmaps highlighting the probability of each pixel to attract human gaze. To define image's important regions, features maps are extracted. The rarity, surprise or contrast are computed leading to conspicuity maps, showing important regions of each feature map. The final saliency map is obtained by merging these maps. The fusion process is usually a linear combination of the maps where the coefficients show their importance. We propose a novel generic fusion mechanism based on 1) using a rarity-based attention module and 2) using neural networks to achieve the fusion. The first layer of the NN merges the weighted feature maps into a saliency map. The second layer takes into account the spatial information. The approach is compared to 8 models using 4 different comparison metrics on open state-of-the-art databases.

Understanding bike trip patterns in a bike sharing system is important for researchers designing models for station placement and bike scheduling. By bike trip patterns, we refer to the large number of bike trips observed between two stations. However, due to privacy and operational concerns, bike trip data are usually not made publicly available. In this paper, instead of relying on time-consuming surveys and inaccurate simulations, we attempt to infer bike trip patterns directly from station status data, which are usually public to help riders find nearby stations and bikes. However, the station status data do not contain information about where the bikes come from and go to, therefore the same observations on stations might correspond to different underlying bike trips. To address this challenge, We conduct an empirical study on a sample bike trip dataset to gain insights about the inner structure of bike trips. We then formulate the trip inference problem as an ill-posed inverse problem, and propose a regularization technique to incorporate the a priori information about bike trips to solve the problem. We evaluate our method using real-world bike sharing datasets from Washington, D.C. Results show that our method effectively infers bike trip patterns.

Statistical analysis of rank data describing preferences over small and variable subsets of a potentially large ensemble of items 1, ..., n is a very challenging problem. It is motivated by a wide variety of modern applications, such as recommender systems or search engines. However, very few inference methods have been documented in the literature to learn a ranking model from such incomplete rank data. The goal of this paper is twofold: it develops a rigorous mathematical framework for the problem of learning a ranking model from incomplete rankings and introduces a novel general statistical method to address it. Based on an original concept of multi-resolution analysis (MRA) of incomplete rankings, it finely adapts to any observation setting, leading to a statistical accuracy and an algorithmic complexity that depend directly on the complexity of the observed data. Beyond theoretical guarantees, we also provide experimental results that show its statistical performance.

Data representing preferences of users are at the core of many Big Data modern applications, such as recommender systems or search engines. While most of the introduced machine learning approaches are designed to handle preference data under the form of cardinal scores, such as ratings given by the users to the items, many situations require to deal with ordinal preferences, coming from implicit feedback data for instance. Methods relying on the analysis of ranking data are best suited for these situations, but they face a great computational challenge insofar as the number of ways to express ordinal preferences on a catalog of n items explodes with n. It is the main purpose of this paper to promote a new representation of preference data when they come under the form of incomplete rankings, that is to say ordinal preferences on small subsets of items. The representation exploits the “multiscale” structure of incomplete rankings and though it relies on recent results in algebraic topology, it is used and interpreted similar to classic wavelet multiresolution analysis on a Euclidean space. We apply it to the problem of incomplete rankings prediction and show at the same time that it is statistically consistent and that it can be computed at a reasonable cost given the complexity of the original data. It is illustrated by very encouraging empirical work based on real datasets.

It is the purpose of this paper to formulate the issue of scoring multivariate observations depending on their degree of abnormality/novelty as an unsupervised learning task. Whereas in the 1-d situation, this problem can be dealt with by means of tail estimation techniques, observations being viewed as all the more “abnormal” as they are located far in the tail(s) of the underlying probability distribution. In a wide variety of applications, it is desirable to dispose of a scalar valued “scoring” function allowing for comparing the degree of abnormality of multivariate observations. Here we formulate the issue of scoring anomalies as a M-estimation problem. A (functional) performance criterion is proposed, whose optimal elements are, as expected, nondecreasing transforms of the density. The question of empirical estimation of this criterion is tackled and preliminary statistical results related to the accuracy of partition-based techniques for optimizing empirical estimates of the empirical performance measure are established.

This paper is devoted to investigate binary classification in a distributed and on-line setting. In the Big Data era, datasets can be so large that it may be impossible to process them using a single processor. The framework considered accounts for situations where both the training and test phases have to be performed by taking advantage of a network architecture by the means of local computations and exchange of limited information between neighbor nodes. An online learning gossip algorithm (OLGA) is introduced, together with a variant which implements a node selection procedure. Beyond a discussion of the practical advantages of the algorithm we promote, the paper proposes an asymptotic analysis of the accuracy of the rules it produces, together with preliminary experimental results.

The distributed estimation of the average value of the sensors initial measures is one of the most popular issues in the Wireless Sensor Networks (WSN) area. In WSNs, broadcasting data seems natural to exchange information quickly because of the broadcast nature of the Wireless channel. Nevertheless, although broadcast-based algorithms converge faster than pairwise algorithms, the obtained consensus is not necessarily the true average. By the means of additional side-information exchange, we propose a broadcast-based algorithm converging rapidly to the true average. The convergence of this new algorithm is established and its convergence speed is exhibited. We remark that the proposed algorithm outperforms the existing ones.

Existing computer simulations of aircraft infrared signature (IRS) do not account for dispersion induced by uncertainty on input data, such as aircraft aspect angles and meteorological conditions. As a result, they are of little use to estimate the detection performance of IR optronic systems; in this case, the scenario encompasses a lot of possible situations that must be indeed addressed, but cannot be singly simulated. In this paper, we focus on low-resolution infrared sensors and we propose a methodological approach for predicting simulated IRS dispersion of poorly known aircraft and performing aircraft detection on the resulting set of low-resolution infrared images. It is based on a sensitivity analysis, which identifies inputs that have negligible influence on the computed IRS and can be set at a constant value, on a quasi-Monte Carlo survey of the code output dispersion, and on a new detection test taking advantage of level sets estimation. This method is illustrated in a typical scenario, i.e., a daylight air-to-ground full-frontal attack by a generic combat aircraft flying at low altitude, over a database of 90,000 simulated aircraft images. Assuming a white noise or a fractional Brownian background model, detection performances are very promising.

The paper addresses the convergence of a decentralized Robbins-Monro algorithm for networks of agents. This algorithm combines local stochastic approximation steps for finding the root of an objective function, and a gossip step for consensus seeking between agents. We provide verifiable sufficient conditions on the stochastic approximation procedure and on the network so that the decentralized Robbins-Monro algorithm converges to a consensus. We also prove that the limit points of the algorithm correspond to the roots of the objective function. We apply our results to Maximum Likelihood estimation in sensor networks.

We propose a new method, based on level sets, to detect aircraft on low resolution infrared images. Aircraft correspond to hot temperatures at the sensor level. Hence it is natural to rely on a test that considers the hottest pixels in the sensed image. If these pixels are close, they are likely to come from a target (i.e., an aircraft); otherwise they belong to the clutter. Instead of manually testing the neighborhood of each hot pixel, we use level sets; this is the first contribution of the paper (corresponding to eq. 2). The other contribution is the calibration of the resulting test. The method is implemented and tested over a database containing 45 604 simulated aircraft images and provides 98.5% of correct detections.

We propose a linear-time line segment detector that gives accurate results, a controlled number of false detections, and requires no parameter tuning. This algorithm is tested and compared to state-of-the-art algorithms on a wide set of natural images.

In this paper we propose a new method for detecting straight line segments in digital images. It improves upon existing methods by giving precise results while controlling the number of false detections and can be applied to any digital image without parameter setting. The method is a nontrivial extension of the approach presented by Desolneux et al. (2000). The core of the method is an algorithm to cut a binary sequences into what we call a multisegment: a set of collinear and disjoint segments. We shall define a functional that measures the so called meaningfulness of a multisegment. This functional allows us to validate detections against an a contrario non-structured model and to select the best ones. The result is a global interpretation, line by line, of the image in terms of straight segments which gives back its geometry with high accuracy. Comparisons with state of the art methods are presented (more examples are available on-line).