We present a novel method to correct flying pixels within data captured by Time-of-flight (ToF) sensors. Flying pixel (FP) artifacts occur when signals from foreground and background objects reach the same sensor pixel, leading to a confident yet incorrect depth estimation in space-floating between two objects. Commercial RGB-D cameras have a complementary setup consisting of ToF sensors to capture depth in addition to RGB cameras. We propose a novel method to correct FPs by leveraging the aligned RGB and depth image in such RGB-D cameras to estimate the true depth values of FPs. Our method defines a 3D neighborhood around each point, representing a “field of view” that mirrors the acquisition process of ToF cameras. We propose a two-step iterative correction algorithm in which the FPs are first identified. Then, we estimate the true depth value of FPs by solving a least-squares optimization problem. Experimental results show that our proposed algorithm estimates the depth value of FPs as accurately as other algorithms in the literature.

We propose a method to visualize a large image database. The key novelty is that the visualization maintains the structure often seen in clustering based visualization schemes while allowing for overlap between two clusters. Clustering is based on user-defined descriptors such as color, texture, and EXIF settings. Associations between images and descriptors are also made explicit in the visualization. Visualization of a 16000 image database downloaded from Flickr is presented.

Images in a database can be modeled by a network whose nodes represent the images and whose edges code the similarity between images with respect to descriptors such as EXIF settings, color, or texture features. In this paper we borrow from distributed computing the concept of network spanners to extract a sub-network that captures the essential relationships and leads to a meaningful, non-cluttered and systematic visual interface for the database. A spanner optimizes the total edge cost of the resultant sub-network while maintaining its distance profile - the path length between any two nodes in the resultant is never stretched beyond a bound. A visual interface for a typical image database and its usage is presented in this paper.

Marching along the DARPA SyNAPSE roadmap, IBM unveils a trilogy of innovations towards the TrueNorth cognitive computing system inspired by the brain's function and efficiency. The non-von Neumann nature of the TrueNorth architecture necessitates a novel approach to efficient system design. To this end, we have developed a set of abstractions, algorithms, and applications that are natively efficient for TrueNorth. First, we developed repeatedly-used abstractions that span neural codes (such as binary, rate, population, and time-to-spike), long-range connectivity, and short-range connectivity. Second, we implemented ten algorithms that include convolution networks, spectral content estimators, liquid state machines, restricted Boltzmann machines, hidden Markov models, looming detection, temporal pattern matching, and various classifiers. Third, we demonstrate seven applications that include speaker recognition, music composer recognition, digit recognition, sequence prediction, collision avoidance, optical flow, and eye detection. Our results showcase the parallelism, versatility, rich connectivity, spatio-temporality, and multi-modality of the TrueNorth architecture as well as compositionality of the corelet programming paradigm and the flexibility of the underlying neuron model.

Inspired by the function, power, and volume of the organic brain, we are developing TrueNorth, a novel modular, non-von Neumann, ultra-low power, compact architecture. TrueNorth consists of a scalable network of neurosynaptic cores, with each core containing neurons, dendrites, synapses, and axons. To set sail for TrueNorth, we developed Compass, a multi-threaded, massively parallel functional simulator and a parallel compiler that maps a network of long-distance pathways in the macaque monkey brain to TrueNorth. We demonstrate near-perfect weak scaling on a 16 rack IBM® Blue Gene®/Q (262144 CPUs, 256 TB memory), achieving an unprecedented scale of 256 million neurosynaptic cores containing 65 billion neurons and 16 trillion synapses running only 388x slower than real time with an average spiking rate of 8.1 Hz. By using emerging PGAS communication primitives, we also demonstrate 2x better real-time performance over MPI primitives on a 4 rack Blue Gene/P (16384 CPUs, 16 TB memory).

In this paper the associations between the low-level features and the images are utilized to create a visualization that enables the user to determine the nature of the database, e.g., what are the predominant features for a group of images and vice-versa. This is based on modeling the database with a bipartite graph and partitioning the graph to find partitions of images and low-level features. Moreover, by using the concept of orthogonal arrays, a sample of images from each partition are presented to the user as a visual snapshot of the database.

Large enterprise systems need continuous monitoring at infrastructure, application and business levels to detect and prevent problem situations. Traditionally, automated monitoring solutions are programmed once at setup based on a set of well-defined monitoring objectives and handed over to the operations team. Such solutions have underlying data models that are often complex and semantically rich but in stable environments, this complexity is generally hidden from the operations team, who only need to make minor configuration changes (e.g. setting thresholds) as and when required. However, the situation is now rapidly changing with enterprise data centers being subject to continuous transformations as new software, hardware and process components get deployed or updated. This puts an immense burden on monitoring activity because not only thousands of different parameters need to get monitored but the addition and modification of service level objectives (SLOs) may happen continuously. We describe a monitoring system architecture which simplifies the task of authoring and managing SLOs in such dynamic and heterogeneous environments. At the heart of our approach is a lightweight and extensible data model that is derived from more complex configuration models, so as to only expose data relevant for monitoring to the operations team. Simple string-tags derived from this model are then used to label SLOs and associated data streams. The approach localizes programming to the data-sensor layer and makes authoring simpler than the specification of objects in an alternate richer but complex object-oriented representation. We also describe a tag-driven real-time visualization tool that can organize data streams using their accompanying tags and ease user navigation through large volumes of monitoring data.

Supervised learning algorithms (relevance feedback (RF) algorithms) are often used in content based image retrieval (CBIR) systems to enhance interactive search and browsing of image databases. One of the issues associated with RF based CBIR systems is the lack of a large training set. Labeling of images is a time consuming activity and users usually do not have the patience to provide feedback on a large set. Thus the challenge is to select a "good" small training set in order to improve the retrieval performance of CBIR systems. In this paper we propose to use orthogonal arrays (OA), popularly used in design of experiments, in order to select this set. The property of OA that make them useful for CBIR systems is that they can be designed with or without using any prior classification information. We show that the top k retrieval accuracy increases rapidly as the number of labeled samples increase.

Supervised learning algorithms (relevance feedback (RF) algorithms) are often used in content based image retrieval (CBIR) systems to enhance interactive search and browsing of image databases. One of the issues associated with RF based CBIR systems is the lack of a large training set. Labeling of images is a time consuming activity and user's usually do not have the patience to label a large set. The challenge is to somehow leverage the much larger set of unlabeled images to improve the performance of CBIR systems. In this paper we propose a novel RF algorithm which learns from both labeled and unlabeled data. Our proposed algorithm also uses active learning so as to maximize the information gained from a given amount of user feedback.

We present an unbalanced MDC video system where two bitstreams are generated, both independently decodable and compliant with the standard H.263 video decoder, and study the performance of the system in providing robustness in transmission over a packet network.

We motivate the need for lookahead search in a context-based entropy coder. An efficient algorithm based on modeling of the context coder as a finite state machine is presented. A key contribution of this paper is the use of the per survivor processing (PSP) principle to enable a lookahead search in scenarios where adaptive entropy coding is used. Our results show that lookahead searches based on PSP result in performance improvements over traditional schemes.

We propose an algorithm for erasure recovery in predictive coding schemes, where erasures can cause catastrophic error propagation. The recovery algorithm is based on sending multiple descriptions of the source and using a deterministic distance measure to find the most likely estimate for the lost data, given the received data and the side information. Results show that we can recover from short burst erasures and that for long bursts (more than 10% of the samples are lost) we can recover to within 0.4 dB of the original DPCM performance.