Robotic tactile sensors based on Electrical Impedance Tomography (EIT) have gained great attention in robotic sensing applications due to their features such as no internal wiring, "all-in-one" structure, and continuous sensing capabilities. However, the effectiveness of EIT-based tactile sensors is hampered by limited spatial resolution and artifacts in the reconstructed images. To address these challenges, various iterative optimization methods based on spatial regularizations and model-based methods have been proposed. In this study, a new EIT reconstruction method using null-space decomposition based on a diffusion model (NSDM) is proposed. Specifically, NSDM consists of a forward diffusion process that first gradually adds Gaussian noise to a clean conductivity image, followed by a backward process that learns to predict the noise that should be removed during each sampling step, utilizing a prior to ensure that the denoising process does not deviate from the correct direction. NSDM requires no training, no optimization, and only requires a pre-prepared diffusion model. Experimental results (both simulation and actual tests) demonstrate that the proposed method outperforms existing generation methods and provides higher quality reconstruction, providing a new solution for robotic tactile sensing in real scenarios.

Electrical Impedance Tomography (EIT) is a highly ill-posed inverse problem, with the challenge of reconstructing internal conductivities using only boundary voltage measurements. Although Physics-Informed Neural Networks (PINNs) have shown potential in solving inverse problems, existing approaches are limited in their applicability to EIT, as they often rely on impractical prior knowledge and assumptions that cannot be satisfied in real-world scenarios. To address these limitations, we propose a two-stage hybrid learning framework that combines Convolutional Neural Networks (CNNs) and PINNs. This framework integrates data-driven and model-driven paradigms, blending supervised and unsupervised learning to reconstruct conductivity distributions while ensuring adherence to the underlying physical laws, thereby overcoming the constraints of existing methods.

Electrical impedance tomography (EIT)-based tactile sensor has shown great potential in human–machine interaction due to its low manufacturing cost, large-area scalability. However, challenges, such as limited spatial resolution, and artifacts in reconstructed images, hinder their effectiveness. In response, this study proposes a model-driven deep learning reconstruction network for EIT-based tactile sensing, named PDCISTA-Net. The framework integrates a preprocessing filtering module and a dual-channel iterative shrinkage-thresholding algorithm (ISTA). Unlike traditional ISTA, PDCISTA-Net employs a dual-channel structural network tailored to capture and represent block correlations and sparsity within impedance change distributions. This approach enables end-to-end training, where parameters, such as step size, nonlinear transforms, and shrinkage thresholds, are learned from generated training data. In addition, a novel filtering module based on the sensitivity matrix is introduced to enhance reconstruction quality by mitigating measurement noise. Numerical metrics and visual results show that PDCISTA-Net outperforms traditional Newton's one-step error reconstructor, total variation, ISTA-Net, and FISTA-Net methods with higher structural similarity index measure and peak signal-to-noise ratio. Ablation experiments verified the effectiveness of the dual-channel structure in improving reconstruction quality. Finally, we developed an EIT-based tactile system to validate the practical application of our approach. The results from real-contact detection demonstrate enhanced image quality and greater noise robustness compared to traditional reconstruction methods.

Electrical Impedance Tomography (EIT)-based tactile sensors offer durability, scalability, and cost-effective manufacturing. However, simultaneously reconstructing force and shape from boundary measurements remains challenging due to EIT’s inherent location dependencies and image artifacts. This study presents a model-driven multimodal convolutional neural network (MM-CNN) for joint EIT-based force and shape sensing. The hybrid approach combines physics-inspired voltage preprocessing with an attention-based network to overcome EIT’s limitations. The preprocessing network applies a linearized one-step inverse solution with Tikhonov regularization to convert raw boundary voltage into a noise-reduced 2D image. The image reconstruction network uses an attention mechanism to focus on salient features, addressing location dependency issues. Quantitative metrics show that MM-CNN outperforms traditional EIT algorithms like NOSER and TV, reducing location dependency and improving shape discrimination. MM-CNN enables unified force and shape modalities, validated through real-contact experiments, enhancing EIT tactile systems for human-robot interaction by incorporating physical knowledge with deep learning.

This paper propose a novel transfer learning approach, Pseudo-Domain Adversarial Network (PDAN), to tackle the issue of electrode displacement in Electrical Impedance Tomography (EIT). Electrode displacement, caused by human movement or improper operation, significantly affects the accuracy of EIT by introducing data errors. Existing solutions either modify the electrode assembly at a high cost or employ recognition algorithms that require retraining from scratch. To overcome these limitations, our work leverages the power of transfer learning to enhance model performance in the target domain by utilizing knowledge from a related task in the source domain. PDAN extends the capabilities of deep adversarial learning by incorporating noisy images to simulate post-electrode rotation scenarios, aiding in the reduction of negative impacts caused by minor electrode displacements. Our method demonstrates superior performance in classifying leg posture data, achieving around 90% accuracy, and proving robust against sensor electrode offset. Experimental results across various datasets validate the effectiveness of PDAN, indicating its potential in addressing complex real-world situations with improved generalization capabilities.

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions in images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning. We combine this similarity learning with multiple existing object detectors to build Quasi-Dense Tracking (QDTrack), which does not require displacement regression or motion priors. We find that the resulting distinctive feature space admits a simple nearest neighbor search at inference time for object association. In addition, we show that our similarity learning scheme is not limited to video data, but can learn effective instance similarity even from static input, enabling a competitive tracking performance without training on videos or using tracking supervision. We conduct extensive experiments on a wide variety of popular MOT benchmarks. We find that, despite its simplicity, QDTrack rivals the performance of state-of-the-art tracking methods on all benchmarks and sets a new state-of-the-art on the large-scale BDD100K MOT benchmark, while introducing negligible computational overhead to the detector.

Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions on the images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of region proposals on a pair of images for contrastive learning. We can directly combine this similarity learning with existing detection methods to build Quasi-Dense Tracking (QDTrack) without turning to displacement regression or motion priors. We also find that the resulting distinctive feature space admits a simple nearest neighbor search at the inference time. Despite its simplicity, QD-Track outperforms all existing methods on MOT, BDD100K, Waymo, and TAO tracking benchmarks. It achieves 68.7 MOTA at 20.3 FPS on MOT17 without using external training data. Compared to methods with similar detectors, it boosts almost 10 points of MOTA and significantly decreases the number of ID switches on BDD100K and Waymo datasets. Our code and trained models are available at https://github.com/SysCV/qdtrack.

Existing research on action recognition treats activities as monolithic events occurring in videos. Recently, the benefits of formulating actions as a combination of atomicactions have shown promise in improving action understanding with the emergence of datasets containing such annotations, allowing us to learn representations capturing this information. However, there remains a lack of studies that extend action composition and leverage multiple view-points and multiple modalities of data for representation learning. To promote research in this direction, we introduce Home Action Genome (HOMAGE): a multi-view action dataset with multiple modalities and view-points supplemented with hierarchical activity and atomic action labels together with dense scene composition labels. Lever-aging rich multi-modal and multi-view settings, we propose Cooperative Compositional Action Understanding (CCAU), a cooperative learning framework for hierarchical action recognition that is aware of compositional action elements. CCAU shows consistent performance improvements across all modalities. Furthermore, we demonstrate the utility of co-learning compositions in few-shot action recognition by achieving 28.6% mAP with just a single sample.

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue.

Surrogate Model Based Optimization (SMBO) is an established technique for handling computationally expensive optimization problems. One important application is the optimization of Particle Reinforced Metal Matrix Composites (PRMMCs). Multi-phase materials are gaining attention. Their performance is strongly affected by microscale properties. By optimizing the microscale structure, these materials can be tailored to satisfy specific requirements. Current manufacturing techniques have limited control over the distribution of reinforcing particles and are subject to considerable uncertainty. Moreover, the simulation and optimization of PRMMCs requires significant computational effort. We propose an approach that tackles the problem of optimizing the characteristics of PRMMCs subject to uniaxial load, by improving the particles' spatial distribution. The optimization problem is split into a bilevel problem: The upper-level optimization aims to find the particle distribution parameters which maximize the PRMMC limit load. Due to potentially infeasible distributions, the lower-level problem attempts to create a particle placement that reflects the specifications of an upper-level candidate solution.

By briefly heating the local environment with a heat lamp and observing what happens with a thermal camera, robots could potentially infer properties of their surroundings. However, this form of active thermography introduces large signal variations compared to traditional active thermography, which has typically been used to characterize small regions of materials in carefully controlled settings. We demonstrate that a data-driven approach with modern machine learning methods can be used to classify material samples over relatively large surface areas and variable distances. We also introduce the use of z-normalization to improve material classification and reduce variation due to distance and heating intensity. Our best performing algorithm achieved an overall accuracy of 77.7% for multi-class classification among 12 materials placed at varying distances (20 cm, 30 cm, and 40 cm). The observations were made for 5 seconds with 1s of heating and 4s of cooling. We also provide a demonstration of performance with a multi-material scene.

MOTIVATION: Microsatellites are among the most useful genetic markers in population biology. High-throughput sequencing of microsatellite-enriched libraries dramatically expedites the traditional process of screening recombinant libraries for microsatellite markers. However, sorting through millions of reads to distill high-quality polymorphic markers requires special algorithms tailored to tolerate sequencing errors in locus reconstruction, distinguish paralogous loci, rarify raw reads originating from the same amplicon and sort out various artificial fragments resulting from recombination or concatenation of auxiliary adapters. Existing programs warrant improvement. RESULTS: We describe a microsatellite prediction framework named HighSSR for microsatellite genotyping based on high-throughput sequencing. We demonstrate the utility of HighSSR in comparison to Roche gsAssembler on two Roche 454 GS FLX runs. The majority of the HighSSR-assembled loci were reliably mapped against model organism reference genomes. HighSSR demultiplexes pooled libraries, assesses locus polymorphism and implements Primer3 for the design of PCR primers flanking polymorphic microsatellite loci. As sequencing costs drop and permit the analysis of all project samples on next-generation platforms, this framework can also be used for direct simple sequence repeats genotyping. AVAILABILITY: http://code.google.com/p/highssr/