Proactive human-robot collaboration (HRC), as a cognition-centric approach, aims to reason the dynamic process of tasks for proactive robotic decision-making, which can be represented through the spatiotemporal evolution of non-paired relationships. Most existing works rely solely on vision-driven knowledge graph methods to reason the spatiotemporal evolution. However, the spatiotemporal evolution of non-paired relationships involves the interaction of multimodal information, and understanding such interactions requires robust analytical capabilities, which poses challenges for proactive robotic decision-making. This paper proposes an interactive semantics-enhanced vision-language model-driven spatiotemporal hypergraph reasoning method (VLSHR) to reveal the spatiotemporal evolution of non-paired relationships. First, to understand vision-language semantics, we fine-tuned a vision-language large language model (LLM) with interactive semantics. Furthermore, vision-language semantics need to be transformed into a hypergraph structure that can represent non-paired relationships. To reason the spatiotemporal evolution of non-pairwise relationships in HRC, we define temporal hyperedges, spatial hyperedges, and task hyperedges, coupling the affiliations of nodes with different types of hyperedges to construct a spatiotemporal hypergraph for HRC tasks. Then, a spatiotemporal hypergraph neural network is developed to reason the spatiotemporal evolution of non-pairwise relationships for proactive robotic decision-making. Finally, a case study on HRC assembly tasks demonstrates the effectiveness of the proposed method.

Under the social manufacturing context, geographically distributed and decentralized micro-and-small-scale manufacturing enterprises (MSMEs) self-organize into manufacturing communities (MCs), a type of decentralized autonomous organization (DAO). In MCs, MSMEs share their manufacturing resources for order-driven cross-enterprise production cooperation, which is supported through blockchain-based order tracking. However, the application of blockchain also brings concerns about data security and privacy protection to MSMEs, which leads to disputes between MSMEs about which data should be stored in the blockchain. For this problem, a granular-computing-based data-sharing decision-making (GrC-DSDM) method is proposed. In the GrC-DSDM method, a fuzzy proximity relationship is used to describe the familiarity between MSMEs in the same MC, and MC familiarity is obtained based on the granular space derived from the fuzzy proximity relation. A fuzzy preference relation is used to represent MSMEs’ preferences for all metadata related to the order, and a group decision-making method is applied to calculate the preference values for all metadata. Through constructing the mapping relationship between MC familiarity and preference values of all metadata, we can determine which metadata should be shared with the MC for blockchain-based order tracking. The GrC-DSDM method can support group decision-making on data sharing among MSMEs in the same MC. The implementation of the GrC-DSDM method is demonstrated through the example of a sheet metal parts processing MC. It is expected that the GrC-DSDM method will provide a basis for enabling blockchain-based order tracking in MCs.

Detection algorithms play an important role in the life-cycle management of smart vision-based defect detection equipment. This article proposes a brain-inspired interpretable network pruning method for smart detection equipment for online defect detection scenarios. A brain-inspired neuronal circuit decomposition model is constructed from the view of the structure physics of artificial neural networks. To meet the real-time requirements, an interpretable network pruning is proposed in three steps: First, a full-size basic convolutional neural network is constructed. Second, the convolutional neural circuit's extraction method based on a genetic algorithm is designed to evaluate the function of different neural units. Third, a pruning method is proposed to eliminate the redundant convolutional neural circuits and retain key units to balance the accuracy and time efficiency. The experimental results demonstrated the proposed pruning method can improve the frame-pre-second by 116% on the premise of maintaining the detection accuracy of 92%.

Virtual metrology (VM) is widely used for yield management and control in semiconductor manufacturing owing to its high real-time inspection, low cost, and convenient maintenance. However, the multimodal characteristics of batch processes are ignored in the existing yield VM models. The adaptive multimodal division and modal sample imbalance have also not been considered. Therefore, a data-driven adaptive VM model based on the Gath–Geva fuzzy clustering (GGFC) and multitask learning deep belief network (MLDBN) is proposed to solve the problems above. First, the GGFC model is designed to realize the feature extraction of the batch direction and the modal division of the time and variable directions. Second, the partition coefficient and classification entropy indexes are designed to determine the modal categories automatically and establish the SMOTE model to deal with the imbalance of multimodal samples. Third, the local features of multimodal are extracted by the designed MLDBN models. After that, the batch direction features and the multi-features extracted from multimodal are used as the improved MLDBN parameters to realize the fusion prediction. Finally, experiments are carried out by the accurate industrial data from a multibatch wafer fabrication process. The efforts show that the proposed VM model presents better performances in the result of different indicators and has higher accuracy and robustness than the traditional models.

Object segmentation for 3-D point clouds plays a critical role in autonomous driving, robotic navigation, and other computer version applications. In object segmentation, all points are considered to be equal of importance in the literature. However, unequal cases exist and a segmentation boundary is mainly determined by neighbor points. To investigate point inequivalence, in this article, an unequal learning approach is proposed to integrate gene expression programming (GEP) and a deep neural network (DNN). GEP is designed to discover the inequivalent function, which measures the importance of different points according to the distances to the segmentation boundary. A cost sensitive learning method is improved to guide the DNN to obtain the loss of different points unequally with the discovered inequivalent function during model training. The experimental results reveal that point inequivalence with respect to boundary distance exists and is helpful to improve the accuracy of object segmentation.

Aiming to improve the adaptability of fabric defect detection, this paper proposes an “edge-cloud” collaborative fabric defect detection architecture that contains edge layer, platform layer, and application layer. In the edge layer, the fabric defect detection machine is able to realize the collection and detection of fabric images data. In the platform layer, the cloud platform that integrates memory computing, parallel storage, and a relational library is designed to realize the efficient storage and analysis of fabric data. In the application layer, a deep learning fabric defect detection algorithm is designed to recognize the defect patterns. The interaction between the cloud platform and the detection device is designed to adaptively adjust the detection algorithm. The closed-loop optimization is achieved by implementing “edge-cloud” architecture that the fabric pictures are captured and analyzed for fast detection algorithm in edge devices. The captured data is stored and monitored by the cloud platform. The cloud platform adjusts the edge detection algorithm by transfer learning, which can adapt to the changing environment. A case study illustrates that the proposed edge-cloud collaborative fabric defect detection can achieve better dynamic adaptability.

Forecasting short-term cycle time (CT) of wafer lots is crucial for production planning and control in the wafer manufacturing. A novel recurrent neural network called “bilateral long short-term memory (bilateral LSTM)” is proposed to model a short-term cycle time forecasting (CTF) of each re-entrant period of a wafer lot. First, a two-dimensional (2-D) architecture is designed to transmit the wafer and layer correlations by using wafer and layer connections. Subsequently, aiming to store various error signals caused by the diverse CT data, a multiply memory structure is presented to extend the capacity of constant error carousel (CEC) in the LSTM model. The experiment results indicate that the proposed model outperforms conventional models in the accuracy and stability for the short-term CTF. Further comparative experiments reveal that the 2-D architecture can enhance the prediction accuracy and the multi-CEC structure can improve the forecasting stability for the short-term CTF of wafer lots.