The free-bending technique, distinguished by its exceptional flexibility in axis control, is emerging as a transformative paradigm for manufacturing complex tubular structures, overcoming geometric limitations inherent to conventional tube bending manufacturing processes. However, the high flexibility in multi-axis free-bending systems introduces nonlinear control complexities that critically compromise the tube forming accuracy. Real-time machine vision approaches enable in-process tracking of tubular geometric deviations, providing a fast method for axis prediction. To this end, this paper presents a real-time vision-enhanced prediction system that integrates with an LSTM-Transformer framework. A high-precision visual sensing system is developed to capture tube-end trajectory, integrating 3D-printed markers, depth camera, kinematic decoupling, and instance segmentation for accurate motion tracking and process parameter inversion. Subsequently, a physics-informed hybrid LSTM-Transformer architecture is proposed for dynamic bend axis springback prediction, incorporating trajectory-derived physical constraints and multi-objective optimization for spatio-temporal springback prediction during dynamic forming. Additionally, an online differential geometry mapping method for real-time curvature parameter estimation is introduced, eliminating the need for post-scanning and additional equipment, enabling closed-loop process parameter compensation during bending. Experimental results show that the proposed method reduces the mean absolute error of axial springback prediction by more than 60% compared to traditional theoretical models, with the mean absolute error for all groups remaining below 12 mm.

ABSTRACT Production data, a critical component of industrial datasets derived from production processes, is widely used to train data‐driven models for forecasting and managing industrial processes. However, shifts in data distribution, caused by changes in production environments, operating conditions, and equipment states, disrupt the consistency between the training and deployment, and lead to catastrophic forgetting and a significant deterioration in both model prediction accuracy and stability. Although existing incremental learning methods have improved adaptability and mitigated forgetting, challenges remain in balancing knowledge retention with dynamic sample selection and ensemble optimization, particularly in complex industrial settings. To address these challenges, this paper proposes an incremental learning framework that includes two key strategies: sample‐importance‐aware buffer update and elastic weight consolidation (EWC) based learner construction for knowledge retention, and performance‐driven iterative strong learner construction with multi‐objective weight optimization. The buffer update dynamically adjusts capacity according to training loss fluctuations, selects high‐information samples guided by loss rates and uncertainty estimation, and maintains diversity through K‐means clustering. EWC consolidates previously acquired knowledge to mitigate forgetting during weak learner training. The ensemble construction evaluates individual learner performance comprehensively and iteratively adjusts model weights using a multi‐objective optimization method, balancing prediction accuracy, stability, and uncertainty. Experimental results on multiple publicly available industrial datasets, complemented by an external validation on a financial dataset, demonstrate that the proposed method outperforms several representative approaches in both accuracy and stability of prediction.

High DoF mobile manipulation of robots is a nonlinear, nonchain redundant problem. In this article, we focus on two subissues of robot mobile manipulation: whole-body inverse kinematics (whole-body IK) and operation-oriented motion planning (OOMP). Whole-body IK solves the robot arm joint configuration and the mobile base position configuration according to the target pose. OOMP generates a feasible trajectory from the current pose to the target pose. The trajectory can avoid obstacles and touch operated objects. We introduce neural network optimization (NNO) methods with two variations to solve whole-body IK and OOMP, respectively. For whole-body IK, we design a fully connected network (FCN) to predict ten DoF of position and joint configurations based on the target pose. We use these ten DoF configurations to derive the predicted pose for online optimization. For OOMP, we design a GRU-based network to generate trajectories based on the initial and goal states. We mainly adopt sphere masks to modify the point cloud properties of the target object dynamically. During optimization, the trajectory keeps away from point clouds but approaches sphere masks. Finally, we conduct extensive experiments both on a Franka Panda robot and a mobile dual-arm robot. The results demonstrate the superior performance of our NNO method on whole body IK and OOMP, and implement mobile manipulation in different environments successfully.

We propose a novel and effective deep learning method, called deep pattern matching, for predicting the energy consumption of complex structures, which helps designers to develop ecological solutions with minimal fabrication energy for additive manufacturing. This new method does not necessitate the real energy consumption values of complex structures for training the prediction model, substantially reducing the cost of training data collection, which can be prohibitively expensive. This novel method exploits simple structures whose real energy consumption values are far cheaper to measure, by matching the similar infill pattern of complex structures from simple structures, and then approximating the energy value of the pattern in the complex structures by the matched one in training phase. This effective algorithm is designed dynamically for allowing us to match patterns with arbitrary shapes. We evaluate our deep pattern matching algorithm on various complex structures, where the highest total energy accuracy is up to 97.3%. The extensive empirical results confirm the effectiveness and robustness of the proposed method, exhibiting a great potential to advance the real usage of deep learning models for energy consumption prediction of complex structures in ecological additive manufacturing.

As bird’s-eye-view (BEV) semantic segmentation is simple-to-visualize and easy-to-handle, it has been applied in autonomous driving to provide the surrounding information to downstream tasks. Inferring BEV semantic segmentation conditioned on multi-camera-view images is a popular scheme in the community as cheap devices and real-time processing. The recent work implemented this task by learning the content and position relationship via Vision Transformer (ViT). However, its quadratic complexity confines the relationship learning only in the latent layer, leaving the scale gap to impede the representation of fine-grained objects. In view of information absorption, when representing position-related BEV features, their weighted fusion of all view feature imposes inconducive features to disturb the fusion of conducive features. To tackle these issues, we propose a novel cross-scale hierarchical Transformer with correspondence-augmented attention for semantic segmentation inference. Specifically, we devise a hierarchical framework to refine the BEV feature representation, where the last size is only half of the final segmentation. To save the computation increase caused by this hierarchical framework, we exploit the cross-scale Transformer to learn feature relationships in a reversed-aligning way, and leverage the residual connection of BEV features to facilitate information transmission between scales. We propose correspondence-augmented attention to distinguish conducive and inconducive correspondences. It is implemented in a simple yet effective way, amplifying attention scores before the Softmax operation, so that the position-view-related and the position-view-disrelated attention scores are highlighted and suppressed. Extensive experiments demonstrate that our method has state-of-the-art performance in inferring BEV semantic segmentation conditioned on multi-camera-view images.

Virtual try-on is a promising computer vision topic with a high commercial value wherein a new garment is visually worn on a person with a photo-realistic effect. Previous studies conduct their shape and content inference at one stage, employing a single-scale warping mechanism and a relatively unsophisticated content inference mechanism. These approaches have led to suboptimal results in terms of garment warping and skin reservation under challenging try-on scenarios. To address these limitations, we propose a novel virtual try-on method via progressive inference paradigm (PGVTON) that leverages a top-down inference pipeline and a general garment try-on strategy. Specifically, we propose a robust try-on parsing inference method by disentangling semantic categories and introducing consistency. Exploiting the try-on parsing as the shape guidance, we implement the garment try-on via warping-mapping-composition. To facilitate adaptation to a wide range of try-on scenarios, we adopt a covering more and selecting one warping strategy and explicitly distinguish tasks based on alignment. Additionally, we regulate StyleGAN2 to implement re-naked skin inpainting, conditioned on the target skin shape and spatial-agnostic skin features. Experiments demonstrate that our method has state-of-the-art performance under two challenging scenarios. The code will be available athttps://github.com/NerdFNY/PGVTON.

Predicting the cross-sectional profile of the whole bending segment for metal tube bending is essential to achieve high-precision bending, yet still remains challenging. The existing prediction methods mainly base on theoretical derivation under certain assumptions and approximations, which do not fully characterize the whole bending segment profile neither do they fully utilize the information in the bending process. In this study, a Bo-LSTM-based progressive prediction method for the cross-sectional profile sequence is proposed, which comprehensively utilizes the profile information during the bending process and achieves an accurate prediction of the cross-sectional profile of the whole bending segment in the subsequent bending process. Firstly, the method of describing the cross-sectional profile in polar radial vector and the cross-sections of the bending segment in discrete sequences are proposed, which cover the information of cross-sectional distortion and wall thickness variation (viz. cross-sectional defects) for the whole bending segment. Secondly, an LSTM network is constructed integrating Bayesian-optimization-based hyper-parameters selection approach to progressively predict the tube cross-sectional profile sequence. Finally, the proposed methods are verified on simulated datasets as well as experimental data, and the accuracy is compared with networks of different structures. The results show that Bo-LSTM has better prediction accuracy. Meanwhile, the progressive prediction pattern has better robustness compared to chain prediction pattern.

TheSixdegrees of freedom 6DoF pose estimation of texture-less objects provides a spatial understanding of industrial scenes and is the basis for accurate object manipulation. Recent studies have shown that the introduction of known model information and initial poses helps CNNs-based methods achieve better performance in complex scenes. However, the mapping from image space to pose space learned by neural networks is dimensionally lifting. Due to the lack of depth information, it is difficult for neural networks to capture local clues on texture-less surfaces and directly regress the relative 3-D translation and 3-D rotation. Instead, we propose a novel framework named REG-Net, which transforms the 6DoF pose estimation task into a 2-D keypoint registration problem. The proposed network first encodes regional prior information using multi-representation, utilizes the globally-consistent offset attention module to align 2-D keypoint features in a long range, and then estimates offsets and potential regions of keypoints. The proposed regional PnP simultaneously adjusts the keypoint locations in a short range and outputs the pose. This framework compresses the learning space of the network from 3-D to 2-D. Extensive experiments on two benchmark datasets demonstrate the robustness and accuracy of REG-Net. We further demonstrate the effectiveness of REG-Net in the reflective industrial part grasping applications.

In online clothing sales, static model images only describe specific clothing statuses towards consumers. Without increasing shooting costs, it is a subject to display clothing dynamically by synthesizing a continuous image sequence between static images. This paper proposes a novel human image sequence synthesis method by pose-shape-content inference. In the condition of two reference poses, the pose is interpolated in the pose manifold controlled by a linear parameter. The interpolated pose is transferred into the end shape by AdaIN and the attention mechanism to infer target shape. Then the content in the reference image is transferred into this target shape. In the content transfer, the visual features of the human body cluster and clothing cluster are extracted, respectively. And the Sobel gradient is adopted to extract clothing texture variation. In the feature inferring, the multiscale feature-level optical flow warps source features, and style code infusion infers new region content without source features. Extensive experiments demonstrate that our method is superior in inferring clear layouts and transferring reasonable content compared to the pose transfer baselines. Moreover, our method has been verified to apply in parsing-guided image inference and dynamic display based on the pose sequence.

Pose estimation is an essential technology for industrial robots to perform precise gripping and assembly. The state-of-the-art deep learning-based approach uses an indirect strategy, i.e., first finding local correspondence between the 2-D image and 3-D model, and then using the perspective-n-point and RANSAC methods to calculate the poses of ordinary objects. However, the metal parts in industry are reflective and textureless, making it difficult to identify distinguishable point features to establish 2-D–3-D correspondences. To address this problem, in this article, we propose a novel deep learning based two-stage method for pose estimation of reflective textureless metal parts, which accurately estimates the target pose using monocular red green blue (RGB) images. Since contours play an important role in both keypoints prediction and pose estimation stages, our method is named ContourPose. First, an additional contour decoder is adopted to implicitly constrain the keypoints prediction in the former stage, which improves the accuracy of the keypoints prediction. Then, the predicted contour of the previous stage is taken as geometric prior that is used to iteratively solve for the optimal pose. Experiments indicate that the proposed approach for reflective textureless metal parts has a significant improvement over the state-of-the-art approaches.

A major obstacle in digital projector technology is that images projected onto nonideal surfaces with large depth variances can easily become blurred. In this article, present a method to overcome projection defocus for projection surfaces that inevitably have complex shapes and large depth variances. The proposed method has two main advantages over traditional methods. First, an edge-intensification-based defocus compensation algorithm is proposed to manipulate the input image to compensate for its projection defocus before blurring occurs. Unlike previous time-consuming compensation algorithms, the proposed algorithm has very high efficiency, as it is noniterative and open loop. Second, a sinusoidal-projection-based estimation method is proposed to reduce kernel estimation errors on complex surface types. Unlike previous methods limited to specific surface types, the proposed method can provide consistently good kernel estimation results even for discontinuous and textured (nonpure white) projection surfaces. Hence, the proposed method can be applied to a wider range of applicable surfaces. These two contributions are demonstrated through extensive experiments and compared with the state-of-the-art methods.

Image compression is an intensively studied subject in computer vision. The deep generative model, especially generative adversarial networks (GANs), is a popular new direction for this subject. In this study, the authors propose a new compression method based on a generative model and focus on its application by GANs. The decoder in the proposed method is modified from the GAN generator model, which can produce visually real‐like synthetic images. It is one of the two models in GANs, which is trained through a two‐players' contest game. The encoder is an optimisation algorithm called backpropagation‐to‐the‐input, which derives from an image inpainting algorithm based on generative models. In the proposed method, the authors turn the encoding process into an optimisation task to search for optimal encoded representations. Compared with traditional methods, the proposed method can compress images from certain domains into extremely small and shape‐fixed encoded space but still retain better visual representations. It is easy and convenient to apply without any retraining or additional modification to the generative models.

We present an exploratory study to improve the performance of a knowledge push system in product design. We focus on the domain of knowledge matching, where traditional matching algorithms need repeated calculations that result in a long response time and where accuracy needs to be improved. The goal of our approach is to meet designers’ knowledge demands with a quick response and quality service in the knowledge push system. To improve the previous work, two methods are investigated to augment the limited training set in practical operations, namely, oscillating the feature weight and revising the case feature in the case feature vectors. In addition, we propose a multi-classification radial basis function neural network that can match the knowledge from the knowledge base once and ensure the accuracy of pushing results. We apply our approach using the training set in the design of guides by computer numerical control machine tools for training and testing, and the results demonstrate the benefit of the augmented training set. Moreover, experimental results reveal that our approach outperforms other matching approaches.

Here, the authors present multistage estimation model embedding with structured pose prior (SPP), a novel coarse‐to‐fine framework for real‐time 3D hand estimation from single depth image. Authors’ main contributions can be summarised as follows: (i) The authors proposed SPP to enforce constraints of canonical hand pose instead of original hand pose. (ii) The authors are the first to adopt under‐complete stacked denoising auto‐encoder (SDA) to construct pose prior by mapping canonical hand pose to latent representation. In the case of enforcing constraints of canonical hand pose, the authors empirically validate that under‐complete SDA outperforms over‐complete SDA in improving the hand estimation accuracy. (iii) The authors propose candidate keypoints patches (CKP) as intermediate data to conduct further hand pose refinement. Experimental evaluation on two publically available datasets shows that authors’ model is competitive both in accuracy and computation time. Especially, authors’ method placed first in the location of palm key‐point on both two datasets, and the high accuracy of hand palm key‐point plays an important role in many applications, such as that manipulator can grasp objects to specific coordinates with the guiding of human hand palm.

Actively pushing design knowledge to designers in the design process, what we call ‘knowledge push’, can help improve the efficiency and quality of intelligent product design. A knowledge push technology usually includes matching of related knowledge and proper pushing of matching results. Existing approaches on knowledge matching commonly have a lack of intelligence. Also, the pushing of matching results is less personalized. In this paper, we propose a knowledge push technology based on applicable probability matching and multidimensional context driving. By building a training sample set, including knowledge description vectors, case feature vectors, and the mapping Boolean matrix, two probability values, application and non-application, were calculated via a Bayesian theorem to describe the matching degree between knowledge and content. The push results were defined by the comparison between two probability values. The hierarchical design content models were built to filter the knowledge in push results. The rules of personalized knowledge push were sorted by multidimensional contexts, which include design knowledge, design context, design content, and the designer. A knowledge push system based on intellectualized design of CNC machine tools was used to confirm the feasibility of the proposed technology in engineering applications.

Recent finance and debt crises have made credit risk management one of the most important issues in financial research. Reliable credit scoring models are crucial for financial agencies to evaluate credit applications and have been widely studied in the field of machine learning and statistics. In this paper, a novel feature-weighted support vector machine (SVM) credit scoring model is presented for credit risk assessment, in which an F -score is adopted for feature importance ranking. Considering the mutual interaction among modeling features, random forest is further introduced for relative feature importance measurement. These two feature-weighted versions of SVM are tested against the traditional SVM on two real-world datasets and the research results reveal the validity of the proposed method.

A suitable initial value of a good (close to the optimal value) scheduling algorithm may greatly speed up the convergence rate. However, the initial population of current scheduling algorithms is randomly determined. Similar scheduling instances in the production process are not reused rationally. For this reason, we propose a method to generate the initial population of job shop problems. The scheduling model includes static and dynamic knowledge to generate the initial population of the genetic algorithm. The knowledge reflects scheduling constraints and priority rules. A scheduling strategy is implemented by matching and combining the two categories of scheduling knowledge, while the experience of dispatchers is externalized to semantic features. Feature similarity based knowledge matching is utilized to acquire the constraints that are in turn used to optimize the scheduling process. Results show that the proposed approach is feasible and effective for the job shop optimization problem.

The paper proposes a semantics knowledge modeling approach for product assembly, which is suitable for virtual reality interaction. Assembly semantics knowledge is used to express design conception, constrain the relationship between parts, and encapsulate design knowledge in semantics knowledge modeling. Using assembly semantics, the designer is able to express design intention in an engineer's language, which is natural and convenient for designer. In addition, the intelligence of the system can be promoted through the inference of implicit knowledge of semantics. These methods are implemented in the development of VIRDAS (Virtual Reality Design and Assembly System) and several assembly examples are also given.