Recent advancements in stereo matching algorithms, driven by new deep neural architectures, have revitalized interest in binocular stereo. However, the effective use of stereo vision in applications demanding precise 3D sensing with high confidence remains challenging. In this paper, we introduce a novel deep stereo matching method designed to address this challenge efficiently. Our approach estimates disparities using implicitly inferred confidence levels. This capability is facilitated by our newly developed U-net transformer, which incorporates various attention mechanisms to extract global and local contexts from rectified image pairs. Additionally, we present a novel real-world stereo dataset captured using a commercially available stereo sensor. This dataset includes challenging scenes featuring diverse objects, each annotated with accurate and dense ground truth disparities. Our dataset includes 1,000 scenes across 20 different object categories, with each scene consisting of both active and passive combinations. Through various experiments, we demonstrate the superiority of our proposed matching algorithm and dataset.

In industry, random bin picking is a complex and difficult task where instance segmentation and object pose estimation based on point clouds are key processes. Recently, learning-based segmentation and pose estimation methods for 3D point clouds have been proposed. However, many of them require supervised learning with datasets with annotations of objects. Since it is difficult to annotate all stacked instances in bin picking dataset, learning without real-world datasets has become a major interest. In this paper, we introduce an instance-level object pose estimation method for bin picking, which is trained using only simulated data and seamlessly applied to real-world scenarios without additional adaptation. To enable this, we introduce a method for generating a comprehensive synthetic dataset using a physics simulator, which incorporates 3D CAD models of objects and automatically generates annotations for both segmentation and pose estimation. Our experiments, conducted on synthetic datasets, highlight the competitive performance of our method in terms of recall and accuracy. Furthermore, we demonstrate the successful integration of our approach with real robot random bin picking, resulting in significantly improved picking success rates.

Time-sensitive networking (TSN) strives to provide an ultralow-latency real-time deterministic network for time-critical traffic using the time-aware shaper (TAS) mechanism. For this purpose, methods for routing and scheduling the time-critical flows must be specified. However, this is an NP-hard problem. Although several prior studies have suggested constraint programming (CP)-based approaches, these methods fail to provide a reasonable runtime due to the complexity of the problem. Motivated by this, we propose a TAS co-optimization (TACO) framework that solves the TAS scheduling and routing problem in TSN with a reasonable runtime. As an alternative to CP-based approaches, TACO considers a metaheuristic approach to co-optimize routing, scheduling order, and transmission timing. However, joint optimization through a metaheuristic algorithm is challenging due to the heterogeneous search spaces of the subproblems. Therefore, TACO carefully integrates the search spaces into a single domain and optimizes routing and TAS scheduling jointly with its heuristic algorithm. We evaluate TACO in various industrial networking scenarios to demonstrate that TACO achieves up to an 88% increase in the scheduling success rate with a good convergence rate and an overall low latency/jitter compared to other approaches.

Time-sensitive networking (TSN) Task Group of the IEEE proposed the frame replication and elimination for reliability (FRER) technique to guarantee reliable transmissions in TSN for the emerging Industrial Internet of Things (IIoT). FRER is a technique that manages the replication and elimination of frames of a stream sent through multiple paths as member streams. However, the standard does not specify how to find and select the multiple paths to send the replicated member streams on, nor how the time-aware shaper (TAS) should be scheduled considering frame elimination. Most prior work on routing or TAS scheduling in TSN do not consider FRER. Conversely, studies on FRER do not address the routing and scheduling issues effectively. In this article, we propose multipath routing and TAS scheduling strategies to support FRER in TSN with reduced complexity. We also identify a scheduling deadlock problem due to the unique characteristics of FRER, and propose a solution using topological sorting. Then, we propose two metaheuristic optimizers to increase the TAS scheduling success rate. Through extensive evaluation against state-of-the-art prior work, we show that our strategies can support more flows with reduced utilization and enhanced schedulability while effectively handling the complexity of routing and scheduling problems for FRER.

To guarantee real-time performance and quality-of-service (QoS) of time-critical industrial systems, time-aware shaper (TAS) in time-sensitive networking (TSN) controls frame transmission times in a bridged network using a scheduled gate control mechanism. However, most TAS scheduling methods generate schedules based on pre-configured routes without exploring alternatives for better schedulability, and methods that jointly consider routing and scheduling require enormous runtime and computing resources. To address this problem, we propose a TSN Scheduler with Reinforcement Learning-based Routing (TSLR) that identifies improved load balanced routes for higher schedulability with acceptable complexity using distributional reinforcement learning. We evaluate TSLR through TSN simulations and compare it against state-of-the-art algorithms to demonstrate that TSLR effectively improves TAS schedulability and link utilization in TSN with lower complexity. Specifically, TSLR shows a more than 66% increase in schedulability compared to the other algorithms, and TSLR’s scheduling time is reduced by more than 1 h. It also shows flows’ transmission latency is less than 25% of their latency deadline requirement and reduces maximum link utilization by approximately 50%.

In this paper, we tackle the problem of 6-DoF grasp detection which is crucial for robot grasping in cluttered real-world scenes. Unlike existing approaches which synthesize 6-DoF grasp data sets and train grasp quality networks with input grasp representations based on point clouds, we rather take a novel hierarchical approach which does not use any 6-DoF grasp data. We cast the 6-DoF grasp detection problem as a robot arm approaching direction selection problem using the existing 4-DoF grasp detection algorithm, by exploiting a fully convolutional grasp quality network for evaluating the quality of an approaching direction. To select the best approaching direction with the highest grasp quality, we propose an approaching direction selection method which leverages a geometry-based prior and a derivative-free optimization method. Specifically, we optimize the direction iteratively using the cross entropy method with initial samples of surface normal directions. Our algorithm efficiently finds diverse 6-DoF grasps by the novel way of evaluating and optimizing approaching directions. We validate that the proposed method outperforms other selection methods in scenarios with cluttered objects in a physics-based simulator. Finally, we show that our method outperforms the state-of-the-art grasp detection method in real-world experiments with robots.

Localization microscopy, such as STORM / PALM, can reconstruct super-resolution images with a nanometer resolution through the iterative localization of fluorescence molecules. Recent studies in this area have focused mainly on the localization of densely activated molecules to improve temporal resolutions. However, higher density imaging requires an advanced algorithm that can resolve closely spaced molecules. Accordingly, sparsitydriven methods have been studied extensively. One of the major limitations of existing sparsity-driven approaches is the need for a fine sampling grid or for Taylor series approximation which may result in some degree of localization bias toward the grid. In addition, prior knowledge of the point-spread function (PSF) is required. To address these drawbacks, here we propose a true grid-free localization algorithm with adaptive PSF estimation. Specifically, based on the observation that sparsity in the spatial domain implies a low rank in the Fourier domain, the proposed method converts source localization problems into Fourier-domain signal processing problems so that a truly gridfree localization is possible. We verify the performance of the newly proposed method with several numerical simulations and a live-cell imaging experiment.