This paper presents a correspondence-free, function-based sim-to-real learning method for controlling deformable freeform surfaces. Unlike traditional sim-to-real transfer methods that strongly rely on marker points with full correspondences, our approach simultaneously learns a deformation function space and a confidence map – both parameterized by a neural network – to map simulated shapes to their real-world counterparts. As a result, the sim-to-real learning can be conducted by input from either a 3D scanner as point clouds (without correspondences) or a motion capture system as marker points (tolerating missed markers). The resultant sim-to-real transfer can be seamlessly integrated into a neural network-based computational pipeline for inverse kinematics and shape control. We demonstrate the versatility and adaptability of our method on two vision devices and across four pneumatically actuated soft robots: a deformable membrane, a robotic mannequin, and two soft manipulators.

This paper presents a learning based planner for computing optimized 3D printing toolpaths on prescribed graphs, the challenges of which include the varying graph structures on different models and the large scale of nodes & edges on a graph. We adopt an on-the-fly strategy to tackle these challenges, formulating the planner as a Deep Q-Network (DQN) based optimizer to decide the next 'best' node to visit. We construct the state spaces by the Local Search Graph (LSG) centered at different nodes on a graph, which is encoded by a carefully designed algorithm so that LSGs in similar configurations can be identified to re-use the earlier learned DQN priors for accelerating the computation of toolpath planning. Our method can cover different 3D printing applications by defining their corresponding reward functions. Toolpath planning problems in wire-frame printing, continuous fiber printing, and metallic printing are selected to demonstrate its generality. The performance of our planner has been verified by testing the resultant toolpaths in physical experiments. By using our planner, wire-frame models with up to 4.2k struts can be successfully printed, up to 93.3% of sharp turns on continuous fiber toolpaths can be avoided, and the thermal distortion in metallic printing can be reduced by 24.9%.

With the increased demand for virtual reality, 360-degree video coding becomes challenging. A 360-degree video should be projected as a planar video before encoding, but this process causes distortion whose degree depends on the latitude. Traditional coding algorithms cannot effectively adapt to this feature. In this study, a fast intra prediction algorithm based on WMSE for 360-degree video is proposed. The similarity between the mother block and child blocks can reflect whether the mother block needs to be divided. Spherical weights are introduced in WMSE, and the similarity expressed by WMSE can distinguish the video of different latitude, which is consistent with the weight characteristics of ERP projection. Experimental results show that the proposed algorithm achieves 32% time reduction, with only 0.29% luma Bjontegaard delta rate (BD-rate) increase on average. The proposed algorithm provides a more effective solution to reduce the coding complexity of 360-degree video.

As virtual reality becomes more popular, 360-degree video coding becomes challenging. Projected videos of 360-degree videos and traditional videos are both planar videos, but the projected videos have distortion whose degree depends on the latitude. Traditional coding algorithms cannot effectively adapt to this feature, and 360-degree videos typically have high resolution and frame rate, which results in a high coding complexity. In this study, a fast prediction unit (PU) early termination algorithm based on weighted mean square error (WMSE) of 360-degree video is proposed. In the proposed algorithm, WMSE is used as a basis for the early termination of further PU partitioning. First, the full intra prediction process of the current CU is performed. After that, the similarity between the current CU and its four sub-CUs is calculated using WMSE for 2N×2N; and distortion between predicted and original blocks is calculated using WMSE for N×N. the similarity and distortion is used to terminate PU partitioning. The experimental results show that the algorithm achieves a 31% time reduction, and an average of only 0.3% of the luma Bjontegaard delta rate (BD rate) increases.

The increased demand for virtual reality brings more challenges to 360∘ video coding. 360∘ video needs to be projected as a planar video before encoding. This process will produce projection distortion. The degree of projection distortion depends on the location of the pixel. Traditional coding algorithms cannot respond to this feature efficiently enough. In this paper, a fast Sample Adaptive Offset (SAO) algorithm for 360∘ video is proposed. The proposed algorithm improves the SAO process. On the basis of retaining the whole SAO process, a simplified SAO process is added. First, the coding tree unit (CTU) of performing the simplified SAO process is filtered according to the rate distortion cost (RD-cost) of the intra- or inter-prediction and the location of the CTU. Subsequently, the CTU is sampled at intervals according to the equirectangular projection (ERP) characteristics, and the CTU of performing the simplified SAO process is determined. Experimental results show that the proposed algorithm achieves 60% time of SAO process reduction, with only 0.29% luma Bjontegaard delta rate (BD-rate) increases on average.