One-versus-one (OVO) and one-versus-rest (OVR) are two widely adopted methods to decompose multi-class problems into several binary classification problems. It is well known that, in the case of kernel SVM, the two methods yield similar test accuracy. Thus, people generally assume that they differ mainly in training time and model size. However, our research reveals that if one considers an evaluation metric taking class imbalance into account, these two methods may give notable performance differences. To explore this phenomenon, we first conduct a detailed analysis of kernel SVM and then extend our study to neural networks. Additionally, we propose novel loss functions for neural networks that effectively integrate the OVO and OVR perspectives. Our experiments clearly demonstrate the robustness of OVO in handling imbalanced multi-class classification, highlighting its advantages over OVR in these challenging scenarios.

This paper investigates the integration of Large Vision Language Models (LVLMs) with multi-sensor information, including visual and localization data from cameras and LiDAR data to a holistic understanding of traffic videos. Traffic scene understanding is a challenging problem. With complex interaction between the road actors, infrastructure, and traffic rules, it is often difficult to answer questions related to road safety, pedestrian safety, safe maneuvering characteristics, and human factors. Typical processes use a single task-oriented neural network model and combine them through semantic and symbolic reasoning. These processes often suffer from reasoning bias and incompleteness. In recent years, LVLMs have opened new avenues to perceive spatiotemporal information. These models can leverage the large knowledge base from the world and summarize spatiotemporal information effectively. The interactive nature of most of these systems allows humans to directly interact in a visual question-answering mode.In this paper, we have extensively tested the capabilities of such LVLMs to answer key transportation research questions from videos captured through front cameras. We have curated an extensive set of multiple-choice questions to evaluate the performance of these LVLMs. Our results show that LVLMs have abilities to understand various transportation-related aspects to a great extent. Furthermore, we have shown that the addition of supplementary modalities to the VQA settings helps improve the performance of LVLMs. With the addition of 3D trajectories of surrounding objects with the 2D video frames, we observed a significant increase in MCQ performance related to vehicle-to-vehicle interaction tasks. The resources for this paper can be found at https://github.com/sandeshrjain/lvlm-scene

Three-dimensional integration circuit (3D IC) offers significant benefits in terms of performance and cost. Existing research in through-silicon via (TSV)-based 3D IC partitioning has focused on minimizing the number of TSVs to reduce costs. Partitioning methods based on heterogeneous integration have emerged as viable approaches for cost optimization. Leveraging mature processes to manufacture not timing-critical blocks can yield cost benefits. Nevertheless, none of the previous 3D partitioning work has focused on reducing the overall cost, including both design and manufacturing costs, for heterogeneous 3D integration. Moreover, throughput constraints have not been considered. This article presents a cost-aware integer linear programming-based formulation and a heuristic algorithm that partition the functional blocks in the design into different technological groups. Each group of functional blocks will be implemented using a particular process technology, and then integrated into a 3D IC. Our results show that 3D heterogeneous integration chip implementation can reduce overall cost while satisfying various timing constraints.

Eldering caring is important in today’s aging society, especially that accident anticipation/prevention plays an important role. In this paper, a novel approach to preventing elderly accidents based on a very low-resolution thermopile sensor array (TPA) (only$32\times 32$pixels) is proposed for prediction of bed-exit event that might lead to elderly falls in home caring. Low-resolution TPA sensor, capable of collecting far infrared energy, ensures cost-effective monitoring, no interference with user’s daily life, and most importantly privacy-preservation. Since most of the fall accidents occur when the elderly attempts to get off the bed without assistance, it is thus the focus of this paper to monitor his/her posture and action via TPA image sensor and then predict that an action of getting off the bed will occur in a near future (e.g.,$S$seconds later). Our system can raise an alarm to the caregivers so that they can intervene and offer the necessary assistance. A deep-learning model based on CNN-RNN (Convolutional neural network-Recurrent neural network) architecture was designed which is capable of predicting the elderly bed-exit intention by$S =5.78$seconds in advance of the action onset at an accuracy of 99.37% according to our dataset evaluation. Our system is also suitable for on-line real-time operation which will be helpful to elderly caring in our society.

Attribute editing has become an important and emerging topic of computer vision. In this paper, we consider a task: given a reference garment image A and another image B with target attribute (collar/sleeve), generate a photorealistic image which combines the texture from reference A and the new attribute from reference B. The highly convoluted attributes and the lack of paired data are the main challenges to the task. To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e.g., collar and sleeves) without paired data. Our method consists of a reconstruction learning step and an adversarial learning step. The model learns texture and location information through reconstruction learning. And, the model's capability is generalized to achieve single-attribute manipulation by adversarial learning. Meanwhile, we compose a new dataset, named GarmentSet, with annotation of landmarks of collars and sleeves on clean garment images. Extensive experiments on this dataset and real-world samples demonstrate that our method can synthesize much better results than the state-of-the-art methods in both quantitative and qualitative comparisons.

Deep reinforcement learning has been proposed to train the control agent for robotic arms, such as Deep Q-Learning (DQN) and Policy Gradient (PG). The approach of Deterministic Deep Policy Gradient (DDPG) takes the advantage of deterministic policy instead of stochastic policy to further simplify the training process and improve the performance. Reinforcement Learning takes the reward from the environment and trains the underlying control agent to achieve the task. An appropriate reward will get better performance and shorter training time, but it requires the domain knowledge and the method of trial and error to define the appropriate reward function. In this paper, we proposed a method that is based on DDPG and makes use of Prioritized Experience Replay (PER), Asynchronous Agent Learning and Meta Learning. The proposed Meta Learning approach uses multiple distributed learners, called workers, to learn from consecutive previous states and rewards. Simulations are done on 6-DOF (IRB140) and 7-DOF (LBR iiwa 14 R820) robotic arms to train the control agents to reach random targets in the three dimension space. The experiments show that the algorithm we proposed is better than the algorithm using DDPG with specialized reward function on the task success rate and the training speed.

The balanced hypercube is a variant of the hyper-cube structure and has desirable properties like connectivity, regularity, and symmetry. The cycle is a popular interconnection topology and has been widely used in distributed-memory parallel computers. Moreover, parallel algorithms of cycles have been extensively developed and used. The problem of how to embed cycles into a host graph has attracted a great attention in recent years. However, there is no systematic method proposed to generate the desired cycles in balanced hypercubes. In this paper, we develop systematic linear time algorithm to construct cycles for the balanced hypercube.

Fashion is a perpetual topic in human social life, and the mass has the penchant to emulate what large city residents and celebrities wear. Undeniably, New York City is such a bellwether large city with all kinds of fashion leadership. Consequently, to study what the fashion trends are during this year, it is very helpful to learn the fashion trends of New York City. Discovering fashion trends in New York City could boost many applications such as clothing recommendation and advertising. Does the fashion trend in the New York Fashion Show actually influence the clothing styles on the public? To answer this question, we design a novel system that consists of three major components: (1) constructing a large dataset from the New York Fashion Shows and New York street chic in order to understand the likely clothing fashion trends in New York, (2) utilizing a learning-based approach to discover fashion attributes as the representative characteristics of fashion trends, and (3) comparing the analysis results from the New York Fashion Shows and street-chic images to verify whether the fashion shows have actual influence on the people in New York City. Through the preliminary experiments over a large clothing dataset, we demonstrate the effectiveness of our proposed system, and obtain useful insights on fashion trends and fashion influence.

The snoopy-based protocol is a widely used cache coherence mechanism for a symmetric multiprocessor (SMP) system. However, this broadcast-based protocol blindly disseminates data sharing information across the system, and introduces many unnecessary data operations. This paper proposes a novel architecture of double-layer counting Bloom filter (DLCBF) to reduce the unnecessary data lookups on the local cache and redundant data transactions on the shared interconnection of an SMP system. By adding an extra filtering layer, the DLCBF effectively exploits the data locality of applications. The two-layer hierarchy reduces the storage size of DLCBF by 18.75%, and achieves 81.99% and 31.36% better filtering rates when compared with a classic Bloom filter (BF) and original counting BF, respectively. When applied on the segmented shared bus of an SMP system, the DLCBF outperforms the previous work by 58% for In-filters and $1.86\times $ for Out-filters. This paper also comprehensively explores the key design parameters of DLCBF, including the sizes of top-layer, bottom-layer, and multilayer design. The results show that enlarging the layer filters enhance the filtering rates of DLCBF, while adding an extra filter layer only provides slight benefit.

Memory Coalescing and on-chip shared Cache are two effective techniques to alleviate the memory bottleneck in modern GPGPUs. These two techniques are very useful on applications with regular memory accesses. However, they become ineffective on concurrent threads with large numbers of uncoordinated accesses and the potential performance benefit could be significantly degraded. This paper proposes a thread affinity mapping methodology to coordinate the irregular data accesses on shared cache GPGPUs. Based on the proposed affinity metrics, threads are congregated into execution groups which are able to fully exploit the memory coalescing and data sharing within an application. An average of 3.5x runtime speedup is achieved on a Fermi GPGPU. The speedup scales with the sizes of test cases, which makes the proposed methodology an effective and promising solution for the continually increasing complexities of applications in the future many-core systems.

In this work, we try to estimate the viewing directions of photos to browse landmarks with different views. To detect the viewing directions of the photos, we propose to leverage tags and geo-information associated with user-contributed photos to predict the locations of the landmarks. By utilizing the tags along with these photos, we demonstrate that the deviation between the estimated locations and the landmarks is about 30 meters on average. The detected locations of the landmarks can be used to further estimate the viewing directions in which the photos were taken. In the experiments, we show that the average error to the real viewing directions is less than 14 degrees. The estimated viewing directions enable illustrating landmarks from different views and recommending users the popular views for taking photos on the landmarks.

In this demonstration, we present a real-time system that addresses three essential issues of large-scale image object retrieval: 1) image object retrieval-facilitating pseudo-objects in inverted indexing and novel object-level pseudo-relevance feedback for retrieval accuracy; 2) time efficiency-boosting the time efficiency and memory usage of object-level image retrieval by a novel inverted indexing structure and efficient query evaluation; 3) recall rate improvement--mining semantically relevant auxiliary visual features through visual and textual clusters in an unsupervised and scalable (i.e., MapReduce) manner. We are able to search over one-million image collection in respond to a user query in 121ms, with significantly better accuracy (+99%) than the traditional bag-of-words model.

State-of-the-art object retrieval systems are mostly based on the bag-of-visual-words representation which encodes local appearance information of an image in a feature vector. A search is performed by comparing query object's feature vector with those for database images. However, a database image vector generally carries mixed information of an entire image which may contain multiple objects and background. Search quality is degraded by such noisy (or diluted) feature vectors. We address this issue by introducing the concept of pseudo-objects to approximate candidate objects in database images. A pseudo-object is a subset of proximate feature points in an image with its own feature vector to represent a local area. We investigate effective methods (e.g., Grid, G-means, and GMM-BIC) to estimate pseudo-objects. Experimenting over two consumer photo benchmarks, we demonstrate the proposed methods significantly outperforming other state-of-the-art object retrieval algorithms.

Efficient image search clustering is prominent for image search engines for exponentially growing photo collections. In this work, we propose an image search clustering approach which selects multiple canonical images from image search results and constructs image clusters in real time on an image sub-graph for the search results. The efficiency is achieved with the help of offline-computed image context graphs by distributed computing methods. Extending our prior works, we demonstrate the results of the proposed canonical image selection and preliminary outcomes of large-scale image graph construction in this proposal. We experiment in Flickr550 dataset, containing 540,321 Flickr photos.

An efficient indexing method is essential for content-based image retrieval with the exponential growth in large-scale videos and photos. Recently, hash-based methods (e.g., locality sensitive hashing - LSH) have been shown efficient for similarity search. We extend such hash-based methods for retrieving images represented by bags of (high-dimensional) feature points. Though promising, the hash-based image object search suffers from low recall rates. To boost the hash-based search quality, we propose two novel expansion strategies - intra-expansion and inter-expansion. The former expands more target feature points similar to those in the query and the latter mines those feature points that shall co-occur with the search targets but not present in the query. We further exploit variations for the proposed methods. Experimenting in two consumer-photo benchmarks, we will show that the proposed expansion methods are complementary to each other and can collaboratively contribute up to 76.3% (average) relative improvement over the original hash-based method.

In light of the strong demands for semantic search over large-scale consumer photos, which generally lack reliable user-provided annotations, we investigate the feasibility and challenges entailed by the new paradigm, concept search - retrieving visual objects by large-scale automatic concept detectors with keywords. We investigate the problem in three folds: (1) the effective concept mapping and selection methods over large-scale concept ontology; (2) the quality and feasibility of the pre-trained concept detectors applying on cross-domain consumer data (i.e., Flickr photos); (3) the search quality by fusing automatic concepts and user-annotated data (tags). Through experiments over large-scale benchmarks, TRECVID and Flickr550, we confirm the effectiveness of concept search in the proposed framework, where the semantic mapping by web-based kernel function over Google snippets significantly outperforms conventional WordNet-like methods both in accuracy and efficiency.

With the prevalence of recording devices and the ease of media sharing, consumers are embracing huge amounts of Internet videos. There arise the needs for effective video advertisement systems following their phenomenal success in text. We propose a novel advertising system, AdImage, which automatically associates relevant ads by matching characteristic images, referred to as adImages (analogous to adWords) here. The proposed image matching method is invariant to certain distortions commonly observed in shared videos. AdImage also avoids the pitfalls of poor tagging qualities in shared videos and provides a brand-new venue to specify ad targets by image objects. Moreover, we formulate the image matching scores and the parameterized bidding information as a nonlinear optimization problem for maximizing the system revenues and user perception.

In this paper, based upon the concept of object oriented methodology (OOM), the complicated sequencing rules can be modularized into several basic components with corresponding sequencing rules in SN, called sequencing object (SO). Each SO is like a middleware, which represents a basic sequencing building block as a cluster in activity tree (AT) with some corresponding sequencing rules on the SCORM view and represents a basic sequencing graph on the users' view. By appropriately combining these SOs, users can easily construct a complex structure of course with desired learning guidance. Then, a transformation algorithm is also proposed to firstly transform the created graph based course structure into activity tree (AT) with related sequencing rules, and then package the transformed activity tree and related physical learning resources into a SCORM compliant course file. Moreover, the authoring tool can be easily upgraded by importing a new SO instead of redesigning the tool only for new SCORM version. Finally, a SCORM compliant course built by the object based authoring tool is executed on the SCORM RTE system. The results show that our approach is workable and efficient.

In this paper, we propose a hybrid approach to refine the phonetic boundaries in a Mandarin speech corpus. This approach employs different sets of acoustic features for different categories of phonetic transitions, except for the most difficult case of “periodic voiced + periodic voiced”, which is therefore handled by a heuristic scheme. Several experiments are designed to demonstrate the feasibility of the proposed approach.

We present an eigenspace-based approach toward prior density selection for the MAPLR framework. The proposed eigenspace-based MAPLR approach was developed by introducing a priori knowledge analysis on the training speakers via probabilistic principal component analysis (PPCA), so as to construct an eigenspace for speaker-specific full regression matrices as well as to derive a set of bases called eigen-matrices. The priors of MAPLR transformations for each outside speaker are then chosen in the space spanned by the first K eigen-matrices. By incorporating the PPCA model into the MAPLR scheme, the number of free parameters in choosing the priors can be effectively reduced, while the underlying structure of the acoustic space as well as the precise modeling of the inter-dimensional correlation among the model parameters can be well preserved. Both supervised and unsupervised adaptation experiments showed that the proposed approach significantly outperformed the conventional maximum likelihood linear regression (MLLR) approach using either diagonal or full regression matrices.

This paper presents an eigenspace-based fast speaker adaptation approach which can improve the modeling accuracy of the conventional maximum likelihood linear regression (MLLR) techniques when only very limited adaptation data is available. The proposed eigenspace-based MLLR approach was developed by introducing a priori knowledge analysis on the training speakers via PCA, so as to construct an eigenspace for MLLR full regression matrices as well as to derive a set of bases called eigen-matrices. The full regression matrices for each outside speaker are then constrained to be located in the space spanned by the first K eigen-matrices. The proposed eigenspace-based regression matrices, serving as an initial estimate of the speaker-specific MLLR transformation, effectively reduces the number of free parameters, while precise modeling for the inter-dimensional correlation among the model parameters by full matrices was maintained. Experimental results showed that for supervised adaptation...