3D semantic occupancy has garnered considerable attention due to its abundant structural information encompassing the entire autonomous driving scene. However, existing 3D occupancy prediction methods are typically tailored for single-frame inputs, resulting in unsatisfactory performance and temporal inconsistencies in real-world continuous scenarios. In this paper, we introduce LinkOcc, a sparse-queries approach incorporating an efficient temporal association mechanism for 3D semantic occupancy prediction. LinkOcc is conceptually built on the prevalent DETR-like framework for 2D segmentation, and we further construct the temporal association mechanism on this basis. Specifically, we propose a near-online training strategy that jointly trains with two adjacent frames, which successfully combines the benefits of both online and off-online methods. Moreover, we introduce a temporal association strategy with contrastive learning to discriminate features for cross-frame semantic-level association. Comprehensive experiments demonstrate that LinkOcc not only surpasses the state-of-the-art methods in 3D occupancy prediction, but also guarantees a promising performance on foreground classes.

Recommendation systems play a vital role to keep users engaged with personalized content in modern online platforms. Deep learning has revolutionized many research fields and there is a recent surge of interest in applying it to collaborative filtering (CF). However, existing methods compose deep learning architectures with the latent factor model ignoring a major class of CF models, neighborhood or memory-based approaches. We propose Collaborative Memory Networks (CMN), a deep architecture to unify the two classes of CF models capitalizing on the strengths of the global structure of latent factor model and local neighborhood-based structure in a nonlinear fashion. Motivated by the success of Memory Networks, we fuse a memory component and neural attention mechanism as the neighborhood component. The associative addressing scheme with the user and item memories in the memory module encodes complex user-item relations coupled with the neural attention mechanism to learn a user-item specific neighborhood. Finally, the output module jointly exploits the neighborhood with the user and item memories to produce the ranking score. Stacking multiple memory modules together yield deeper architectures capturing increasingly complex user-item relations. Furthermore, we show strong connections between CMN components, memory networks and the three classes of CF models. Comprehensive experimental results demonstrate the effectiveness of CMN on three public datasets outperforming competitive baselines. Qualitative visualization of the attention weights provide insight into the model's recommendation process and suggest the presence of higher order interactions.

Automatic image annotation has been extensively studied in the recent decades. Nevertheless, existing methods usually assume a properly labeled training set, which greatly inhibits their application to real-world datasets with incomplete labels. Due to their lack of special treatments for noisy data, most existing methods simply consider the missing labels as strictly negative ones, leading to the degradation of tagging accuracy. In light of such challenges, we propose a novel model in this paper, called ranking-preserving low-rank factorization. Specifically, we construct a local training set for each test image, and conduct low-rank matrix factorization on the model coefficient matrix, to simultaneously capture the label dependency and reduce the model complexity. Furthermore, to alleviate the ambiguity introduced by missing labels, the prediction model is learnt via tag ranking regularized by sample similarities and tag correlations, and both regularization terms are incorporated into our factorization scheme. By assembling all the aforementioned components together, our method obviates the need for making binary decisions based on unreliable data, and thus is more robust towards missing labels. Extensive empirical evaluations conducted on four datasets demonstrate the effectiveness of the proposed method.

Label propagation is a popular semi-supervised learning technique that transfers information from labeled examples to unlabeled examples through a graph. Most label propagation methods construct a graph based on example-to-example similarity, assuming that the resulting graph connects examples that share similar labels. Unfortunately, example-level similarity is sometimes badly defined. For instance, two images may contain two different objects, but have similar overall appearance due to large similar background. In this case, computing similarities based on whole-image would fail propagating information to the right labels. This paper proposes a novel Instance-Level Label Propagation (ILLP) approach that integrates label propagation with multi-instance learning. Each example is treated as containing multiple instances, as in the case of an image consisting of multiple regions. We first construct a graph based on instance-level similarity and then simultaneously identify the instances carrying the labels and propagate the labels across instances in the graph. Optimization is based on an iterative Expectation Maximization (EM) algorithm. Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed approach over several state-of-the-art methods.

Low-rank matrix factorization serves as a key technique in learning latent factor models for many applications in machine learning. However, in many applications, observed data often exhibits different levels of noise. To address this issue, we propose a Robust Probabilistic Discriminative Matrix Factorization (RPDMF) method for binary matrix factorization on noise polluted data. We illustrate the benefits of our approach in real examples, and show how our method significantly outperforms Probabilistic Discriminative Matrix Factorization (PDMF) and classical method Weighted Nonnegative Matrix Factorization (WNMF) in the application of image tag completion.

Short-term traffic prediction plays a critical role in many important applications of intelligent transportation systems such as traffic congestion control and smart routing, and numerous methods have been proposed to address this issue in the literature. However, most, if not all, of them suffer from the inability to fully use the rich information in traffic data. In this paper, we present a novel short-term traffic flow prediction approach based on dynamic tensor completion (DTC), in which the traffic data are represented as a dynamic tensor pattern, which is able capture more information of traffic flow than traditional methods, namely, temporal variabilities, spatial characteristics, and multimode periodicity. A DTC algorithm is designed to use the multimode information to forecast traffic flow with a low-rank constraint. The proposed method is evaluated on real-world data sets and compared with other state-of-the-art methods, and the efficacy of the proposed approach is validated on the experiments of traffic flow prediction, particularly when dealing with incomplete traffic data.

Many visual applications have benefited from the outburst of web images, yet the imprecise and incomplete tags arbitrarily provided by users, as the thorn of the rose, may hamper the performance of retrieval or indexing systems relying on such data. In this paper, we propose a novel locality sensitive low-rank model for image tag completion, which approximates the global nonlinear model with a collection of local linear models. To effectively infuse the idea of locality sensitivity, a simple and effective pre-processing module is designed to learn suitable representation for data partition, and a global consensus regularizer is introduced to mitigate the risk of overfitting. Meanwhile, low-rank matrix factorization is employed as local models, where the local geometry structures are preserved for the low-dimensional representation of both tags and samples. Extensive empirical evaluations conducted on three datasets demonstrate the effectiveness and efficiency of the proposed method, where our method outperforms pervious ones by a large margin.

As one of the most important state-of-the-art classification techniques, Support Vector Machine (SVM) has been widely adopted in many real-world applications, such as object detection, face recognition, text categorization, etc., due to its competitive practical performance and elegant theoretical interpretation. However, it treats all samples independently, and ignores the fact that, in many real situations especially when data are in high dimensional space, samples typically lie on low dimensional manifolds of the feature space and thus a sample can be related to its neighbors by being represented as a linear combination of other samples on the same manifold. This linear representation, which is usually sparse, reflects the structure of underlying manifolds. It has been extensively explored in the recent literature and proven to be critical for the performance of classification. To benefit from both the underlying low dimensional manifold structure and the large margin classifier, this paper proposes a novel method called Sparsity Preserving Support Vector Machine(SP-SVM), which explicitly considers the sparse representation of samples while maximizing the margin between different classes. Consequently, SP-SVM inherits both the discriminative power of support vector machine and the merits of sparsity. A set of experiments on real-world benchmark data sets show that SP-SVM achieves significantly higher precision on recognition task than various competitive baselines including the traditional SVM, the sparse representation based method and the classical nearest neighbor classifier.

Hashing techniques have been widely applied for large scale similarity search problems due to the computational and memory efficiency.However, most existing hashing methods assume data examples are independently and identically distributed.But there often exists various additional dependency/structure information between data examplesin many real world applications. Ignoring this structure information may limit theperformance of existing hashing algorithms.This paper explores the research problemof learning to Hash on Structured Data (HSD) and formulates anovel framework that considers additional structure information.In particular, the hashing function is learned in a unified learning framework by simultaneously ensuring the structural consistency and preserving the similarities between data examples.An iterative gradient descent algorithm is designed as the optimization procedure. Furthermore, we improve the effectiveness of hashing function through orthogonal transformation by minimizing the quantization error.Experimentalresults on two datasets clearly demonstrate the advantages ofthe proposed method over several state-of-the-art hashing methods.

Nystrom approximation is an effective approach to accelerate the computation of kernel matrices in many kernel methods. In this paper, we consider the Nystrom approximation for sparse kernel methods. Instead of relying on the low-rank assumption of the original kernels, which sometimes does not hold in some applications, we take advantage of the restricted eigenvalue condition, which has been proved to be robust for sparse kernel methods. Based on the restricted eigenvalue condition, we have provided not only the approximation bound for the original kernel matrix but also the recovery bound for the sparse solutions of sparse kernel regression. In addition to the theoretical analysis, we also demonstrate the good performance of the Nystrom approximation for sparse kernel regression on real world data sets.

In this paper, motivated by the superior performance of sparse representation based dictionary learning for application of image classification and the usage of nonlinearity property in improving performance of image representation, we propose a locality sensitive dictionary learning algorithm with global consistency and smoothness constraint to overcome the restriction of linearity at relatively low cost. Specifically, the image features are partitioned into several groups in a locality sensitive way and a global consistency regularizer is embedded into locality sensitive dictionary learning algorithm. The proposed algorithm is efficient to capture complex nonlinear structure. Experimental results on several benchmark data sets demonstrate the efficiency of our proposed locality sensitive dictionary learning algorithm.

Similarity search, or finding approximate nearest neighbors, is an important technique in various large scale information retrieval applications such as document retrieval. Many recent research demonstrate that hashing methods can achieve promising results for large scale similarity search due to its computational and memory efficiency. However, most existing hashing methods ignore the hidden semantic structure of documents but only use the keyword features (e.g., tf-idf) in hashing codes learning. This paper proposes a novel sparse semantic hashing (SpSH) approach that explores the hidden semantic representation of documents in learning their corresponding hashing codes. In particular, a unified framework is designed for ensuring the hidden semantic structure among the documents by a sparse coding model, while at the same time preserving the document similarity via graph Laplacian. An iterative coordinate descent procedure is then proposed for solving the optimization problem. Extensive experiments on two large scale datasets demonstrate the superior performance of the proposed research over several state-of-the-art hashing methods.

The current sparse representation framework is to decouple it as two subproblems, i.e., alternate sparse coding and dictionary learning using different optimizers, treating elements in bases and codes separately. In this paper, we treat elements both in bases and codes ho-mogenously. The original optimization is directly decoupled as several blockwise alternate subproblems rather than above two. Hence, sparse coding and bases learning optimizations are coupled together. And the variables involved in the optimization problems are partitioned into several suitable blocks with convexity preserved, making it possible to perform an exact block coordinate descent. For each separable subproblem, based on the convexity and monotonic property of the parabolic function, a closed-form solution is obtained. Thus the algorithm is simple, efficient and effective. Experimental results show that our algorithm significantly accelerates the learning process.

A novel tag completion algorithm is proposed in this paper, which is designed with the following features: 1) Low-rank and error s-parsity: the incomplete initial tagging matrix D is decomposed into the complete tagging matrix A and a sparse error matrix E. However, instead of minimizing its nuclear norm, A is further factorized into a basis matrix U and a sparse coefficient matrix V, i.e. D = UV + E. This low-rank formulation encapsulating sparse coding enables our algorithm to recover latent structures from noisy initial data and avoid performing too much denoising; 2) Local reconstruction structure consistency: to steer the completion of D, the local linear reconstruction structures in feature space and tag space are obtained and preserved by U and V respectively. Such a scheme could alleviate the negative effect of distances measured by low-level features and incomplete tags. Thus, we can seek a balance between exploiting as much information and not being mislead to suboptimal performance. Experiments conducted on Corel5k dataset and the newly issued Flickr30Concepts dataset demonstrate the effectiveness and efficiency of the proposed method.

Support vector machine is one of the most successful machine learning methods in image processing and computer vision in the past decades. However, its performance strongly depends on the training data, which are sometimes expensive and of low quality. Specifically, in many real applications, such as face recognition, the images are rarely perfectly aligned, thus the misalignment between training and testing data impairs the performance. In this paper, we propose a strategy to compensate the misalignment between images while learning the classifier without looking at the testing samples. Specifically, some certain critical transformations are inferred and applied to training samples to alleviate the effect of the worst case of possible misalignment. The resulted large margin classifier generalizes better than traditional SVM, especially when there is misalignment. Experimental results on real image data sets show the efficacy of the proposed algorithm.

Nonnegative Matrix Factorization (NMF) is a widely used technique in many applications such as face recognition, motion segmentation, etc. It approximates the nonnegative data in an original high dimensional space with a linear representation in a low dimensional space by using the product of two nonnegative matrices. In many applications data are often partially corrupted with large additive noise. When the positions of noise are known, some existing variants of N-MF can be applied by treating these corrupted entries as missing values. However, the positions are often unknown in many real world applications, which prevents the usage of traditional NMF or other existing variants of NMF. This paper proposes a Robust Nonnegative Matrix Factorization (RobustNMF) algorithm that explicitly models the partial corruption as large additive noise without requiring the information of positions of noise. In particular, the proposed method jointly approximates the clean data matrix with the product of two nonnegative matrices and estimates the positions and values of outliers/noise. An efficient iterative optimization algorithm with a solid theoretical justification has been proposed to learn the desired matrix factorization. Experimental results demonstrate the advantages of the proposed algorithm.

Kernel methods have been regarded as an effective approach in image processing. However, when calculating the similarity induced by kernels, existing kernel methods usually incorporate irrelevant features (e.g., the background features of an object in a image), which are then inherited to kernel learning methods and thus lead to suboptimal performance. To attack this problem, we introduce a framework of kernel tapering, which is a simple and effective approach to reduce the effects of irrelevant features while keeping the positive semi-definiteness of kernel matrices. In theory, it can be demonstrated that the tapered kernels asymptotically approximate the original kernel functions. In practical image applications where noises or irrelevant features are widely observed, we have further shown that the introduced kernel tapering framework can greatly enhance the performance of their original kernel partners for kernel k-means and kernel nonnegative matrix factorization.

Collaborative representation based classification (CRC) has been successfully used for visual recognition and showed impressive performance recently. However, it directly uses the training samples from each class as the subspaces to calculate the minimum residual error for a given testing sample. This leads to high residual error and instability, which is critical especially for a small number of training samples in each class. In this paper, we propose a class specific subspace learning algorithm for collaborative representation. By introducing the dual form of subspace learning, it presents an explicit relationship between the basis vectors and the original image features, and thus enhances the interpretability. Lagrange multipliers are then applied to optimize the corresponding objective function, i.e., learning the weights used in constructing the subspaces. Extensive experimental results demonstrate that the proposed algorithm has achieved superior performance in several visual recognition tasks.

Sparse representation technique has been widely used in various areas of computer vision over the last decades. Unfortunately, in the current formulations, there are no explicit relationship between the learned dictionary and the original data. By tracing back and connecting sparse representation with the K-means algorithm, a novel variation scheme termed as self-explanatory convex sparse representation (SCSR) has been proposed in this paper. To be specific, the basis vectors of the dictionary are refined as convex combination of the data points. The atoms now would capture a notion of centroids similar to K-means, leading to enhanced interpretability. Sparse representation and K-means are thus unified under the same framework in this sense. Besides, an appealing property also emerges that the weight and code matrices both tend to be naturally sparse without additional constraints. Compared with the standard formulations, SCSR is easier to be extended into the kernel space. To solve the corresponding sparse coding sub problem and dictionary learning sub problem, block-wise coordinate descent and Lagrange multipliers are proposed accordingly. To validate the proposed algorithm, it is implemented in image classification, a successful applications of sparse representation. Experimental results on several benchmark data sets, such as UIUC-Sports, Scene 15, and Caltech-256 demonstrate the effectiveness of our proposed algorithm.

Display advertising is a multi-billion dollar industry where advertisers promote their products to users by having publishers display their advertisements on popular Web pages. An important problem in online advertising is how to forecast the number of user visits for a Web page during a particular period of time. Prior research addressed the problem by using traditional time-series forecasting techniques on historical data of user visits; (e.g., via a single regression model built for forecasting based on historical data for all Web pages) and did not fully explore the fact that different types of Web pages have different patterns of user visits.

Nonnegative Matrix Factorization (NMF) is a widely used technique in many applications such as clustering. It approximates the nonnegative data in an original high dimensional space with a linear representation in a low dimensional space by using the product of two nonnegative matrices. In many applications with data such as human faces or digits, data often reside on multiple manifolds, which may overlap or intersect. But the traditional NMF method and other existing variants of NMF do not consider this. This paper proposes a novel clustering algorithm that explicitly models the intrinsic geometrical structure of the data on multiple manifolds with NMF. The idea of the proposed algorithm is that a data point generated by several neighboring points on a specific manifold in the original space should be constructed in a similar way in the low dimensional subspace. A set of experimental results on two real world datasets demonstrate the advantage of the proposed algorithm.

This paper proposes a novel patch-wise image inpainting algorithm using the image signal sparse representation over a redundant dictionary, which merits in both capabilities to deal with large holes and to preserve image details while taking less risk. Different from all existing works, we consider the problem of image inpainting from the view point of sequential incomplete signal recovery under the assumption that the every image patch admits a sparse representation over a redundant dictionary. To ensure the visually plausibility and consistency constraints between the filled hole and the surroundings, we propose to construct a redundant signal dictionary by directly sampling from the intact source region of current image. Then we sequentially compute the sparse representation for each incomplete patch at the boundary of the hole and recover it until the whole hole is filled. Experimental results show that this approach can efficiently fill in the hole with visually plausible information, and take less risk to introduce unwanted objects or artifacts.

In this paper, the problem of foreground segmentation in videos is considered. The bag of superpixels is proposed to simultaneously model both the foreground and the background. Then it is demonstrated that an image has a hierarchical structure. Based on this observation, the discriminative nature of sparse representations is exploited to segment the foreground in each frame. Experimental results show that the proposed algorithm can accurately locate the object's position and segment its image support.

In this paper, one of the problems of linear discriminant analysis (LDA), that is, it pays more attention on minimizing the within-class scatter than on maximizing the between-class scatter, is treated. Though the weighted maximum margin criterion (WMMC) with an appropriate weighted coefficient can solve this problem, how to select this coefficient automatically is still difficult as most of previous works determine it manually. To deal with this problem, a novel approach to determine the coefficient automatically is proposed. The description and analysis of the approach are given in details, and the experiments on BernDBS face database and JAFFE expression database are performed. The results show that the WMMC with the weighted coefficients determined by this novel approach outperforms traditional LDA.