Wireless sensor network (WSNs) stands out as a burgeoning and promising domain in intelligent sensing. Owing to various factors such as sudden sensor malfunctions or deliberate shutdown of partial nodes to save energy, the collected sensing signals from WSNs commonly have massive missing data, leading to adverse effects on subsequent analysis or decision-making. Latent factor learning (LFL) has proven to be highly effective in recovering the missing data for WSNs. However, the existing LFL models require the collected sensing signals to be maintained in one central place like a central server, which is becoming unacceptable for data owners who are getting increasingly privacy-sensitive. To address this issue, this paper innovatively proposes a f ederated l atent f actor l earning (FLFL) model for privacy-preserving spatio-temporal signal recovery. Its main idea is two-fold: 1) it designs a sensor-level federated learning framework based on LFL, where each sensor only needs to upload gradient information rather than raw data for training a privacy-preserving recovery model, and 2) it incorporates the spatio-temporal correlation into the designed federated learning framework as the regularization constraint to improve its recovery accuracy. With such designs, FLFL can not only accurately recover the missing data of WSNs but also ensure data owners' privacy-preserving of raw data. To evaluate the proposed FLFL model, extensive experiments have been conducted on four real-world WSNs datasets. The results demonstrate that FLFL significantly outperforms five state-of-the-art federated signal recovery models in terms of recovery accuracy with privacy-preserving.

Modern neural networks (NNs), while effective at learning representations from given samples and handling downstream pattern recognition tasks, typically contain tens to hundreds of millions of parameters. The growth in NN size motivates ongoing research on effective network compression with the purpose of reducing the computational burden without significantly sacrificing the model performance. It is especially critical when deploying NNs on resource-constrained devices where the computation and storage efficiency are highly concerned. A promising and currently popular solution to model compression is to replace the NN weight matrix with its low-rank tensor approximation, i.e., implementing an efficient latent factorization of tensors (LFT) process on the NNs parameters. Based on thorough investigations into the state-of-the-art LFT-based model compression methods, this survey 1) provides a comprehensive review on the latest research progress of LFT-based model compression methods on various NNs (e.g., Convolutional NNs, Recurrent NNs, and Transformers); 2) summarizes a number of widely-used LFT toolboxes; 3) evaluates LFT methods for model compression on a variety of main-stream NN backbones; and 4) discusses the development trend of LFT-based model compression technique. This survey aims to provide a systematic and comprehensive survey concerning the LFT-based model compression methods to artificial intelligence researchers and engineers, thereby promoting further research development in this crucial field.

High-dimensional and incomplete (HDI) data are ubiquitous in various Big Data-related industrial applications, such as drug innovation and recommender systems. Hash-learning is the most efficient representation learning approach to extract hidden information from HDI data owing to its fast reasoning and low storage. However, an existing hash learning approach commonly employs gradient-based optimization techniques to address the discrete objective caused by the binary nature of hash factors, where the Quantization (i.e., quantizing the real values to binary codes) loss is inevitable, resulting in accuracy loss when representing HDI data. Motivated by these critical and vital issues, this paper proposes a non-gradient hash factor (NGHF) model with three-fold ideas: a) innovating a discrete differential evolution (DDE) algorithm able to simulate the continuous optimization via disabling bits of binary codes based on the projected Hamming dissimilarity, thus enabling an effective discrete optimizer, b) applying the proposed DDE algorithm to directly optimize the discrete learning objective of NGHF defined on HDI data, thereby facilitating its efficient and precise training without any Quantization loss, and c) theoretically proving the convergence of NGHF. As such, NGHF possesses high representation learning ability comparable to that of a real-valued model, making it able to achieve precise binary representation to HDI data. Extensive experimental results on nine real-world datasets demonstrate that NGHF significantly outperforms eight state-of-the-art hash learning models. Moreover, its accuracy is amazingly comparable to that of a real-valued model for HDI data representation learning.

The prevailing techniques in textual sentiment analysis include convolutional neural networks (CNNs), bidirectional long short-term memory (BiLSTM) networks, and attention mechanisms. However, the existing sentiment analysis methods typically focus on relationships among neighboring words, neglecting long-range dependencies. Moreover, they fail to reduce redundant information in semantic features, ultimately affecting the effectiveness of sentiment analysis. To tackle these challenges, this article introduces a novel text sentiment analysis model consisting of three key modules: word embedding, a scalable multichannel sentiment feature extractor, and a sentiment classifier. The scalable multichannel sentiment feature extractor employs a scalable N-gram dilated CNN (SN-DCNN) to extract multiscale high-level semantic by capturing both neighboring and nonneighboring word associations. To further enhance the model’s ability to capture semantic features, a BiLSTM network is introduced to capture long-range dependencies within sentences. Subsequently, a structured attention (SA) mechanism is incorporated after the BiLSTM to capture multiscale semantic information, thereby enabling the mapping of individual long-range dependencies to distinct semantic levels. In addition, a semantics-driven channel attention mechanism (SDCAM) is introduced to assign weights to multichannel structures, aiming to reduce the redundancy in the semantic features. Furthermore, by integrating multihead self-attention (MHSA), the model can effectively concentrate on key sentiment-related features, resulting in a comprehensive enhancement of its semantic representation capability. The sentiment classifier utilizes an adaptive loss function based on cross-entropy to correct the probability distribution of sentiment classes. This function enables the model to dynamically adjust its learning capacity for each class, leading to significant performance improvements, especially in datasets with imbalanced class distributions. Experimental results show that the proposed model improves accuracy by 0.94%, 0.96%, 0.36%, and 0.86% on the NLPCC2017-ECGC, ChnSentiCorp, COLD, and IMDB datasets, respectively, when using word-level embeddings. With character-level embeddings, the model improves accuracy by 0.76% and 1.23% on the ChnSentiCorp and COLD datasets, respectively, compared to the state-of-the-art methods.

Large-scale dynamic weighted directed network (DWDN) is commonly utilized to illustrate the temporal interactions between nodes in numerous applications. Latent factorization of tensors (LFT) is a typical representation learning approach to extract the desired knowledge from a DWDN via low-rank tensor embedding. However, an existing LFT approach requires the target DWDN to be maintained in one central place like a central server, which is becoming unacceptable for users who are getting increasingly privacy-sensitive. To address this vital issue, this paper innovatively proposes a federated latent factorization of tensors (FLFT) model. It can perform accurate and privacy-preserving representation learning to a DWDN based on four-fold ideas: 1) establishing a data-density-oriented federated learning framework to enable different users to efficiently and cooperatively build a shared LFT model with keeping raw data privacy, 2) incorporating the linear biases into the local training of each user to eliminate the personalized biases or local fluctuations, 3) adopting an effective hybrid filling strategy to further protect each user's private interaction information, and 4) designing a customized nonlinear activation function to capture the nonlinear characteristics of users' interactions. Extensive experiments on four DWDNs collected from industrial applications validate that FLFT demonstrates a notable increase in accuracy compared with state-of-the-art federated and non-federated learning approaches in representing a DWDN. The source code of the proposed FLFT model is shared at the following link:https://github.com/wudi1989/FLFT.

Accurate and automated identification of organoids from bright-field images is essential for enabling high-throughput drug screening and precision medicine. Organoids, as 3-D in vitro cellular models, closely recapitulate the functional and structural characteristics of their tissue or organ of origin, presenting an unprecedented opportunity for biomedical research. However, the complexity of bright-field microscopy images, including heterogeneous backgrounds and diverse organoid morphologies, poses significant challenges for existing computational methods, often hindering robust feature extraction and high-throughput analysis. To address these issues at the intersection of computational vision and organoid biology, we propose FEMSSorg, a novel organoid recognition framework designed to adaptively aggregate multiscale scan-selected state space features through a fuzzy mixture-of-experts (FuzzyMoE) scoring mechanism. FEMSSorg introduces a fuzzy expert soft routing mechanism (fuzzy route), implemented via Fuzzy C-Means-based soft routing assignments, forming a new class of fuzzy MoE that leverages fuzzy expert clustering scores to dynamically integrate local (LocalSS) and global (GlobalSS) state space features. This approach enables effective balancing of global pixel dependencies and local texture information, thereby substantially reducing background interference and image noise in bright-field images and improving the accuracy of organoid identification. Furthermore, we incorporate a Dual Downsampling Adaptive Pooling Feature Fusion module, which combines original backbone features with parallel downsampled features and utilizes content-aware pooling for adaptive multilevel and multiscale feature fusion. Experimental results on multiclass organoid bright-field image datasets demonstrate that FEMSSorg achieves state-of-the-art performance in both organoid detection and morphological texture classification, highlighting its value as a robust computational tool for advancing real-time, high-throughput organoid research.

Electricity theft in smart grids has become an increasingly serious issue. Existing detection methods are often limited by their ability to learn from high-dimensional features and by resource constraints, resulting in suboptimal performance. Moreover, incomplete data caused by real-world factors, such as advanced metering infrastructure (AMI) communication interruptions or sensor failures further complicates the detection task. To address these challenges, we propose a tensor representation-driven contrastive distillation model for robust electricity theft detection. Specifically, we first employ tensor decomposition to dynamically capture multidimensional spatio-temporal features from grid data, enabling the model to learn complex temporal dependencies and better represent missing information. We further introduce a semi-supervised learning framework that integrates knowledge distillation and contrastive learning to enhance the detection capability of lightweight models, especially under conditions of limited data quality and incomplete data scenarios in dynamic environments. Finally, we formulate a multitask learning objective that harmonizes discriminative classification with representation coherence. This synergistic interplay enhances the model’s robustness in complex consumption scenarios and sharpens its ability to identify suspicious behaviors. Extensive experiments demonstrate that the proposed model outperforms state-of-the-art methods.

Pathological grading of laryngeal squamous cell carcinoma (LSCC) based on whole-slide image (WSI) is crucial for the diagnosis, treatment and prognosis. According to pathologists' knowledge, tumor regions are highly associated with grading. However, existing multiple instance learning (MIL) methods tend to overrepresent weakly relevant non-tumor regions and irrelevant background, leading to poor grading performance and interpretability. Motivated by the above problems, we propose an end-to-end knowledge-driven MIL network with hierarchical cluster-incorporated aware filtering, i.e. HCF-MIL. Firstly, we develop the tumor-guiding cluster filtering for feature representation, which awarely filters out irrelevant instance-level information and adaptively assigns learnable weights to tumor and non-tumor instances. Secondly, conventional mean-based and max-based aggregation primarily capture the overall patterns, neglecting the contributions of the most representative individual instances. Therefore, we propose a novel enhanced filtering aggregation learning strategy to strengthen hierarchical tumor-related feature representation. Through end-to-end optimization, HCF-MIL reduces model's entropy value and facilitates better alignment between decision-making process and diagnostic behaviors of pathologists. Experiments on larynx and multicentre datasets show that HCF-MIL significantly improves both pathological grading performance and interpretability, providing a strong foundation for reliable clinical deployment.

Recent studies have demonstrated that miRNA expression dysregulation is closely related to the occurrence of various diseases; thus, miRNA-based drug development strategies have received increasing research interest. Most existing computational methods focus on the attribute information of individual nodes and are limited to the direct associations between nodes, thereby ignoring the complex associations inherent in the network. This limitation may lead to the loss of key potential information, which impacts the prediction accuracy. To address these issues, we propose a multisource information fusion and metapath enhancement matrix based graph autoencoder (MSMP-GAE) to predict the potential associations between miRNAs and drugs. The proposed MSMP-GAE model comprises a metapath instance extraction module, a metapath feature enhanced encoder module, a weighted feature fusion module, and a graph autoencoder. First, we construct an miRNA-drug heterogeneous network using experimentally validated miRNA-drug interactions and integrate various miRNA and drug features into an initial feature matrix to comprehensively represent their intrinsic property information. Then, we extract metapath instances from the interaction network, generate multiple metapath enhancement matrices, and fuse them with the initial feature matrix to generate high-quality node feature embeddings. Finally, we employ the graph autoencoder for fivefold cross-validation on a public dataset and test it on an independent test set. Experimental results demonstrate that the proposed MSMP-GAE model obtained an area under the curve (AUC) and AUPR values of 98.61% and 98.23%, respectively, which is considerably better than the several state-of-the-art methods. This highlights the importance of the higher-order complex associations between nodes in the miRNA-drug association (MDA) prediction task and provides a new method and approach to advance MDA prediction.

Identifying locally dense communities closely connected to the user-initiated query node is crucial for a wide range of applications. Existing approaches either solely depend on rule based constraints or exclusively utilize deep learning technologies to identify target communities. Therefore, an important question is proposed: can deep learning be integrated with rule-based constraints to elevate the quality of community search? In this paper, we affirmatively address this question by introducing a novel approach called Neural Community Search via Attribute augmented Conductance, abbreviated as NCSAC. Specifically, NCSAC first proposes a novel concept of attribute-augmented conductance, which harmoniously blends the (internal and ex ternal) structural proximity and the attribute similarity. Then, NCSAC extracts a coarse candidate community of satisfactory quality using the proposed attribute-augmented conductance. Subsequently, NCSAC frames the community search as a graph optimization task, refining the candidate community through sophisticated reinforcement learning techniques, thereby producing high-quality results. Extensive experiments on six real world graphs and ten competitors demonstrate the superiority of our solutions in terms of accuracy, efficiency, and scalability. Notably, the proposed solution outperforms state-of-the-art methods, achieving an impressive F1-score improvement ranging from 5.3% to 42.4%. For reproducibility purposes, the source code is available at https://github.com/longlonglin/ncsac.

The latent factor analysis (LFA) model has been widely used to uncover latent relationships from high-dimensional sparse (HiDS) matrices. However, the performance of LFA depends largely on the hyper-parameter value used in the model training. Traditional hyper-parameter tuning methods such as grid search suffer from inefficiency and inaccuracy. In recent years, the particle swarm optimization (PSO) algorithm offers an intelligent approach to adaptively adjust the hyper-parameter of LFA. However, the global optimal solution of the hyper-parameter tuning problem is not fixed due to its dynamic decision space. Therefore, it is difficult for PSO to determine the best hyper-parameter for each training iteration. To address this problem, this paper proposes a novel hyper-parameter adaptive adjustment algorithm called dynamic stochastic reorientation PSO (DSR-PSO) that adapts to constantly changing decision spaces. By randomly adjusting the search directions of particles and perturbing the elite particles, the dynamic property of the DSR-PSO can be enhanced, so that the hyper-parameter can be adjusted in real time throughout the model training process. Furthermore, this paper proves the convergence of the DSR-PSO and gives its convergence condition by discussing the distribution of the characteristic roots. Finally, this paper proposes the DSR-PSO-based LFA (DPL) model by incorporating the DSR-PSO-based hyper-parameter adjustment into the LFA to promote its model training, and analyzes its complexity. Experimental results on benchmark datasets show that the proposed DPL surpasses state-of-the-art LFA models in terms of accuracy and efficiency.

The latent factor analysis (LFA) model is an effective tool for extracting valuable information from high-dimensional and sparse (HiDS) matrices. However, traditional LFA usually suffers from low accuracy due to the limitations of the stochastic gradient descent (SGD) algorithm used in its model training. First, the learning rate of SGD is adjusted manually, which greatly affects the training efficiency. A recent solution is adjusting this hyperparameter by the particle swarm optimization (PSO) algorithm. However, PSO cannot adapt well to the dynamic decision space of this problem due to its strong convergence. Second, SGD relies solely on the gradient information to perform the optimization, which decreases the training accuracy. To address the above two issues, this article proposes a novel LFA model called genetic algorithm-based two-step LFA (GA-TSLFA), which employs the GA to facilitate the model training. Compared to PSO, the GA has better flexibility, which can be employed to tune the hyperparameter of LFA in dynamic decision spaces and refine the model in high-dimensional and complex decision spaces by designing suitable evolutionary operators. The training of the proposed GA-TSLFA consists of two steps. In the first step, the model is pretrained by SGD whose learning rate is adaptively adjusted by a proposed GA. In the second step, the LF matrices generated by SGD are further refined using a proposed GA-based framework. This framework operates by optimizing a subset of partial vectors, which are selected through a dedicated strategy. In this way, the model's accuracy can be further enhanced. Empirical studies on benchmark datasets show that the GA-TSLFA surpasses state-of-the-art LFA models in prediction accuracy and has a competitive efficiency.

Since similar users frequently exhibit similar Quality of Service (QoS) when accessing similar services, effectively capturing neighborhood information hidden in QoS data becomes critical for latent factorization of tensor (LFT)-based QoS estimators. Current LFT models either calculate the complete set of neighborhoods or do not consider neighborhoods, resulting in a rapid rise in model complexity and poor estimation accuracy. Moreover, not every neighbor in the neighborhood set is beneficial to the user/service entity. To address these limitations, this study proposes a sampling-neighborhood-regularized latent factorization of tensor (SNLFT) model with three key ideas: 1) extracting primal latent factors (LFs), which are obtained to express related entities on the basis of high-dimensional and incomplete QoS data; 2) constructing the sampling-neighborhood set, which is acquired using the Gibbs sampling to reflect the similarities between the primal LF vectors of entities over time; 3) developing a sampling-neighborhood-regularized LFT model, where all the sampling neighborhoods of entities and L2-norm of desirable LFs are employed to regularize the objective function. Extensive experiments on eight dynamic QoS datasets demonstrate that SNLFT significantly outperforms state-of-the-art models in both estimation accuracy and computational efficiency.

Session-based recommendation (SBR) in service computing is pivotal in predicting a user's next action based on their current anonymous session. While Graph Neural Network (GNN)-based methods have shown promise in capturing intricate item transformation relationships within sessions, they often fall short in accurately modeling user preferences. This is primarily due to the common practice of solely considering the last item in the session as the user's current interest, neglecting potentially valuable information embedded in other session items which is essential for capturing user global preferences. Moreover, existing models typically optimize performance solely through cross-entropy loss between predicted items and ground truth labels, while overlooking latent valuable knowledge embedded in intermediate features and item-item relationships that lends support to the model in accurately capturing and modeling user preferences. To address these shortcomings, we propose Multi-Scale Collaborative Distillation (MSCD) for SBR. Our approach introduces a current interest adaptive selection module, which dynamically selects appropriate item embeddings as session-local embeddings by evaluating the importance of each item within the session. This allows for a more accurate capture of the user's current true preferences. Additionally, we propose collaborative knowledge distillation, where multiple models are trained concurrently, enabling the transfer of three types of knowledge including response-based, feature-based, and relationship-based knowledge between models, thereby enriching the model's understanding of user preferences. Experimental evaluations conducted on three popular SBR datasets demonstrate that our MSCD model outperforms recent state-of-the-art methods in terms of recommendation accuracy. Our codes are available at:https://github.com/lonely-ice/MSCD.

Traces and logs record the behaviors and interactions between microservices, making them indispensable for diagnosing system anomalies. However, the intricate and flexible architecture of microservices presents significant challenges for automated anomaly detection. Existing approaches often struggle to capture the internal dynamics and correlations among microservices, which limits their ability to comprehensively detect anomalies arising from synchronous or asynchronous calls. Therefore, we categorize three prevalent anomaly paradigms in microservice systems: intra-service, inter-service, and joint anomalies. To address these challenges, we propose a novel unsupervised learning framework for dual-view anomaly detection, named TraceHG. Specifically, it constructs the trace graph composed of logs, response times, and traces. Besides, TraceHG utilizes hypergraph transformation to establish two views, providing comprehensive information in microservices. Furthermore, we introduce a dual-view framework that leverages both a hypergraph-view and a graph-view encoders to explicitly learn associations among microservices. These two views, employing graph and HGNNs, capture representations of internal dynamics, invocations, and their associated contexts in microservices. By constructing two minimized hyperspheres and measuring the distances from their centers in the latent space, we identify anomalies based on the anomaly scores. Extensive experiments on public benchmark datasets demonstrate that the proposed framework outperforms several state-of-the-art baselines in microservice anomaly detection.

Predicting temporal Quality of Service (QoS) data is critical for optimizing network services and rationalizing resource allocation in cloud computing and service-oriented systems. Existing mainstream methods have achieved promising predictive performance. However, their purely data-driven manner limits their ability to capture non-stationary temporal patterns, thereby leading to accuracy degradation when temporal QoS data exhibits fluctuations. To tackle this limitation, we propose a novelExtendedKalman Filter-EnhancedLatent Feature Analysis (EKL) model to perform efficient and accurate temporal QoS prediction from the perspective of bidirectional model-data-driven. Its main idea is three-fold: a) designing a model-driven feature producer to obtain the temporal latent features to capture the intricate temporal pattern following the principle of an Extended Kalman Filter; b) building a data-driven feature producer based on the alternating least squares algorithm to identify time-invariant latent features describing intrinsic user-service characteristics; c) exploiting a density-oriented parallel strategy that achieves workload balancing by sorting users in accordance with their service invocation density, which effectively elevates computational efficiency. In addition, we provide a rigorous theoretical analysis to formally prove the convergence of the proposed EKL. Experimental evaluations conducted on real-world temporal QoS datasets reveal that our proposed EKL surpasses existing state-of-the-art models with respect to both computational efficiency and prediction accuracy for missing temporal QoS data.

The traditional nonnegative latent factorization of tensors (NLFTs) models can effectively represent high-dimensional and incomplete (HDI) tensors, but they currently face two main problems: 1) existing models are linear and cannot capture the nonlinear features of HDI tensors and 2) they rely on random initialization of nonnegative parameter and constraint-combined training schemes. To address these issues, this article proposes a neural NLFTs model with acceleration and unconstraint. The main ideas are given as follows: 1) utilizing a neural network (NN) structure and a nonlinear activation function to capture the nonlinear features within the HDI tensor accurately; 2) constructing a nonnegative mapping domain that transfers nonnegativity constraints from latent factors (LFs) to output decision parameters via a single-element-dependent mapping function, enabling an unconstrained optimization framework; and 3) utilizing the highly compatible momentum-incorporated stochastic gradient descent (SGD) algorithm as the backward propagation (BP) learning scheme of the model, which not only ensures training effectiveness and scalability but also accelerates convergence. Empirical studies on ten HDI tensors demonstrate that the proposed model achieves impressive estimation accuracy and per-iteration time cost compared to state-of-the-art models.

High-dimensional and incomplete (HDI) data are ubiquitous for representing the intricate interactions among a vast number of nodes arising from diverse real application scenarios. Nonnegative latent factor (NLF) models have demonstrated their effectiveness in extracting critical latent features from HDI data by leveraging a single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) algorithm. However, the SLF-NMU algorithm always leads NLF models to converge sluggishly as it updates an LF based only on the current updated information. To address this critical issue, we present APNLF, which innovatively adopts an adaptive proportional–integral–derivative (PID) controller to enable the learning process of SLF-NMU to be more efficient. The proposed APNLF encompasses the following twofold ideas: 1) establishing a PID-increment-based SLF-NMU (PSN) algorithm, which updates an LF by comprehensively modeling the current, past, and future update increment information guided by the principle of a PID controller; and 2) designing an effective fuzzy reasoning rule to implement all hyperparameters adaptation, which boosts model’s applicability in practical applications. Moreover, APNLF’s convergence analysis is provided in theory. Experiments on five HDI datasets demonstrate that the APNLF model outperforms the state-of-the-art models in efficiency and accuracy on an HDI matrix. The source code is available athttps://github.com/Aaaaapplege/APNLF-Development

High-dimensional and incomplete (HDI) data commonly arise in many complex application scenarios, such as bioinformatics and recommender systems. Deep neural networks (DNNs) exhibit cutting-edge performance in representing HDI data due to their formidable capacity for nonlinear learning. However, previous research primarily concentrates on single-metric-focused models utilizing fixed and exclusive$L_{2}$-norm-based strategies for both loss and regularize terms. Such strategies limit the model’s ability to effectively learn from diverse and heterogeneous HDI data. Recognizing this limitation, this article presents the multimetric autoencoder (MMA) with the following twofold ideas: 1) utilizing multiple$L_{p}$-norms to create four distinct autoencoders, each defining a unique metric representation space with diverse regularize and loss characteristics; 2) integrating these four diverse autoencoders by using a customized, self-adjusting weighting strategy. This innovative approach enhances the model’s capacity to handle heterogeneous and inclusive HDI data effectively, addressing the limitations of previous studies. The theoretical analysis supports the effectiveness of the MMA in harnessing the benefits of diverse multimetric spaces. In the experiments, the MMA is evaluated on six real HDI datasets. The experimental results reveal that the MMA outperforms seven contemporary models in effectively representing HDI data.

Federated Learning (FL) offers a means to train models collaboratively across decentralized data sources while preserving privacy. However, prevalent real-valued FL models face challenges with high computational and communication costs, along with potential privacy breaches. Hash Learning, by converting real values to binary codes, presents a solution but is traditionally limited by its lack of privacy considerations and failure to account for the varying importance of bit positions, resulting in information loss. We propose the, Federated Projection-Hamming Hash Latent Factor (FPHLF) model, which addresses three key challenges in traditional FL: computational costs, storage overhead, and privacy risks. FPHLF innovates by a) Utilizing Projected Hamming Distance to enhance binary code representation by recognizing bit significance; b) Implementing efficient bitwise operations to minimize FL's computational and storage demands; and c) Opting for binary update direction matrices over real-valued gradients to reduce communication costs and strengthen privacy. FPHLF achieves high accuracy with reduced computational, storage, and communication expenses, and mitigates privacy risks. Experiments on four real-world datasets show FPHLF outperforms existing hashing and FL methods in accuracy, efficiency, and security. The source code, accessible at https://github.com/LiCaCO3/FPHLF.

Dynamic graphs offer more precise modeling of real-world applications compared to static graphs. Therefore, learning on dynamic graphs has garnered significant research attention in recent years. Unfortunately, the current approaches for learning dynamic graphs remain inadequate in capturing global spatial-temporal dependency. This issue arises from capturing biased spatial and temporal dependencies, thereby weakening the coupling between spatial and temporal dimensions. To overcome it, we propose a Self-Reliant Global Dependency Apprehending Framework on Dynamic Graphs, namely SGD-DyG. Specifically, we first design a frequency self-enhanced learning module that examines the global inherent interactions of the node features hidden in the frequency domain. Furthermore, we propose a global-local-mixed self-supervised learning module that maximizes spatial-temporal mutual information between local node and global graph embeddings. Extensive experiments on seven real-world dynamic graph datasets validate that the proposed SGD-DyG consistently exceeds state-of-the-art models.

Traditional Chinese medicine (TCM), as a treasure of traditional Chinese medical heritage, has developed a unique therapeutic system through millennia of practice. Its multi-compound, multi-target mechanism demonstrates distinct advantages in disease treatment. However, under the modern medical framework, the modernization of TCM research faces dual challenges: on one hand, the complex interaction network between herbal compounds and target proteins has not yet been systematically elucidated, and traditional methods based on in vitro experiments or statistical correlations suffer from long experimental cycles and high failure rates; on the other hand, compared to chemical drugs with well-established large-scale annotated databases, the association data between herbal compounds and targets exhibit significant sparsity, making it difficult for existing methods to effectively model cross-domain knowledge transfer due to mismatched data dimensions. To address this scientific challenge, this study proposes a cross-domain association prediction framework, DGCPI, which integrates graph neural networks and adversarial learning. To begin with, DGCPI constructs two heterogeneous graph networks for modelling chemical drug-target (source domain) and herbal compound-target (taget domain) associations respectively. It then applies a graph convolutional network to capture source-domain-specific topological features in the chemical drug-target graph. After that, an adversarial discriminator is introduced to dynamically align the embedding distributions between the source and target domains, overcoming the dependence of traditional methods on the assumption of homomorphic data. Experimental results show that only with a small number of TCM samples involved in the training, the model significantly outperforms baseline models on two TCM datasets.

Traditional Chinese medicine (TCM), with its roots in centuries of clinical practice, has established itself as an effective therapeutic system that involves a diverse range of herbal plants. Despite its proven efficacy, the intricate relationships between herbal multi-component preparations and multi-target therapies present challenges for systematic study, thereby limiting its broader application in managing chronic diseases. In this work, we aim to identify novel therapeutic targets of natural compounds found in TCM herbs by leveraging advanced hypergraph representation learning techniques. Following the multi-component, multi-target pharmacological mechanisms, we first construct two hypergraphs to represent herb-compound and disease-target interactions, respectively. The connection between these hypergraphs is established through compound-target associations. A convolutional operator is then employed to capture the high-order correlations between compound (or target) nodes and herb (or disease) hyperedges within each hypergraph. Furthermore, we incorporate the PageRank algorithm and a multi-head attention mechanism to enhance the representation capabilities of node embeddings. By integrating these methods, our model is able to accurately identify novel therapeutic targets of natural compounds in TCM herbs in an end-to-end manner. Extensive experiments conducted on three benchmark datasets demonstrate the superior performance of our model when compared with several state-of-the-art approaches. Furthermore, case studies on two natural compounds, coumarin and progesterone, reveal that 7 and 8 out of the Top-10 identified targets, respectively, have been validated through literature review. These results highlight the effectiveness of our model in discovering new therapeutic targets for natural compounds in TCM.

Spatio-temporal data like a Dynamically Weighted Directed Network (DWDN) are ubiquitous in bigdata applications like an intelligent recommender system. They commonly illustrate the complex yet dynamic interactions between tremendous nodes, as well as contain rich knowledge regarding the involved nodes’ behavioral patterns with time dynamics. On the other hand, since the nodes constantly increase as the time accumulates, a DWDN becomes High-Dimensional and Incomplete (HDI) due to the limited interactions, widely-spread time slots and huge node count. Moreover, the inner temporal-spatio patterns exhibit strong nonlinearity, making it very difficult to grasp them from HDI data. To address this critical issue, this paper proposes a novel Tensor Causal Convolution Network (TCCN) model with three-fold ideas: a) innovatively building a feedforward tensor neural network with the incorporation of the Latent Factorization of Tensors (LFT) principle to model complex nonlinear interactions in an HDI DWDN efficiently; b) establishing a Tensor Causal Convolution (TCC) structure, which is able to accurately fuse the time-varying information from node interactions with high scalability; and c) developing a neighborhood regularization scheme to boost the local structural representation, thus capturing spatial dependencies among tremendous nodes. Extensively experimental results on ten real DWDNs from real bigdata applications evidently demonstrate that our proposed TCCN model outperforms several state-of-the-art models in both representation learning accuracy and convergence ability. It provides a highly-efficient representation learning approach for diverse bigdata applications.

Online streaming feature selection (OSFS) is a critical technique for addressing high-dimensional streaming data in various real applications. The challenge of OSFS arises from missing entries due to various reasons such as equipment failure or human mistakes. Online sparse streaming feature selection (OS$^{2}$FS) is a feasible approach to this challenge by pre-estimating the missing data before feature selection based on latent factor analysis (LFA). However, such an approach separates the processes of the LFA-based estimates and the down-streaming feature selection, which cannot represent the uncertain relationships between the sparse features and the labels, thereby leading to accuracy losses. To address this critical issue, this paper proposes arecursion-and-fuzziness reinforcedonlinesparsestreamingfeatureselection (RF-OS$^{2}$FS) model, which consists of two-fold ideas: 1) connecting the LFA-based feature estimation process and the consequent feature selection process via a recursively completion sampling strategy, therefore enabling the information feedback loop from the feature selection validation results to the missing feature estimation, and 2) adopting the three-way decisions (3WD) strategy to establish the fuzzy feature selection for representing the uncertain relationships between the sparse features and the labels. Experimental results on ten real-world benchmark datasets demonstrate that the proposed RF-OS$^{2}$FS model significantly out-performs existing state-of-the-arts, both OSFS and OS$^{2}$FS models in terms of accuracy when performing sparse streaming feature selection for down-streaming classification tasks.

In the intelligent transportation system, the collected traffic data are usually incomplete. The low-rank completion models are effective in processing missing traffic data (MTD) imputation. However, the existing low-rank completion models encounter the following challenges: 1) Incorporating spatio-temporal information of traffic data often leads to significant time overhead; 2) To reduce the time overhead, some models based on the low-rank completion only embed the temporal information and discard the spatial information, thereby resulting in low imputation accuracy; 3) Missing traffic data carries complex spatio-temporal information, requiring multi-faceted analysis to extract valuable insights. To address these issues, we propose an efficient fourth-order dimension preserved tensor completion (FDPTC) with temporal constraint model. It works based on our proposed fourth-order dimension preserved (FDP) tensor decomposition model to capture the spatio-temporal information of traffic data from a high-dimensional perspective by extending dimensions. Additionally, we embed a temporal constraint into FDP tensor decomposition model to ensure consistency of MTD and introduce a non-negative constraint to accelerate convergence speed. By constructing this model, we successfully avoid direct operations on the extra information matrix/tensor, thereby optimizing efficiency. Experimental results on four real traffic datasets demonstrate that our proposed model achieves significantly higher imputation accuracy at an affordable computational burden compared with state-of-the-art models.

Traffic data prediction is a crucial component of Intelligent Transport Systems (ITS) as it contributes significantly to real-time navigation and congestion management. However, due to incomplete deployment of sensor and instability of data transmission, the traffic data that we collect is usually a high-dimensional and incomplete (HDI) matrix or tensor. Currently, there are two challenges with traffic data prediction as follows: a) Most of the existing models are designed for a full set of data, but traffic data are unavoidably missing. b) Most of the existing models often suffer from excessive complexity due to long sequences. To address these issues, we propose a novel3D Convolution-IncorporatedDimensionPreserved Decomposition (3DCIDP) model for traffic data prediction with three main fold ideas: a) enhancing the low-rank property of traffic data to accurately capture its structure, b) learning the constraints of historical sequences to predict sequences through representation modelling and c) capturing spatio-temporal interaction information in traffic data through multidimensional interaction features. To evaluate the performance of the proposed 3DCIDP, we conduct extensive experiments using five publicly available datasets with three different missing rates. When the proposed 3DCIDP is compared to state-of-the-art models, experimental results show that the Root Mean Square Error (RMSE) is reduced by an average of 4.22% and the training time is reduced by an average of 89.76% on the large-scale traffic datasets.

Intelligent Transportation Systems (ITSs) are designed to alleviate traffic congestion and provide convenience for travelers or decision-makers. However, the challenge lies in obtaining complete and accurate traffic data because of various factors. Therefore, many models have been studied for traffic data imputation. Most studies overlook the extreme case where the missing rate of observed traffic data exceeds 90%, causing the existing traffic data imputation models to struggle in achieving satisfactory results. To address this extreme case, this paper proposes a novel Latent Factor Analysis model with Temporal Regularized Constraint (LFA-TRC) for handling road traffic data imputation. The proposed model is built on three main parts: a) Enhancing the stability of the training process through spatio-temporal linear biases, b) Capturing information in the traffic data time series by the first-order temporal difference constraint, and c) Accelerating convergence during the training process by incorporating generalized momentum. Experimental results on seven real-world traffic datasets demonstrate that the proposed LFA-TRC model outperforms eight state-of-the-art models in traffic data imputation and reduces the Mean Absolute Percentage Error (MAPE) by an average of 20.81%.

High-dimensional and incomplete (HDI) data are involved frequently in big data-related industrial applications. Latent factor (LF) analysis aims at extracting the knowledge of great value from such extremely sparse HDI data efficiently. Non-negative LF models based on the single LF-dependent, non-negative, and multiplicative update rules exactly are the representative of LF analysis. However, these models face low generalization dilemma due to incompatible with general unconstrained optimization techniques. To address this issue, this article proposes a novel momentum-accelerated and biased unconstrained non-negative latent factor (MBUNLF) model, which matches with unconstrained optimization techniques. The proposed MBUNLF model is built on three main ideas: (a) Improving the generalization through a non-negative mapping function; (b) Capturing information among different entities through linear biases; (c) Accelerating convergence during the training process through generalized momentum method. Empirical studies on six datasets from industrial applications indicate that the proposed MBUNLF model outperforms nine state-of-the-art models when processing HDI data, reducing the root mean square error by 19.47% on average. It demonstrates the validity of the MBUNLF model in extracting non-negative LFs from HDI data.

High-Dimensional and Incomplete (HDI) tensors contain a wealth of knowledge and patterns, which are typically utilized to characterize complex relationships between entities in a variety of industrial applications. Currently, the neural network-based tensor factorization model has shown superiority when handling the missing data in HDI tensors. However, it only uses the outer product of latent factors (LFs) of entities and neglects the interactions between the LF. In addition, the simple linear operation does not consider the nonlinear structure of the HDI tensor. To overcome the aforementioned issues, an Attention-mechanism-based Neural Latent-Factorization-of-Tensors (ANLFT) model is provided in this article. It encompasses three primary ideas: (a) incorporating the theory of neural networks with the latent factorization of tensor to construct the nonlinear structure in the HDI tensor effectively; (b) adopting the attention mechanism to depict the interactions between LF; (c) using the position-transitional particle swarm optimization backward propagation learning ( \(\rm{P}^{2}\) BP) scheme to train the ANLFT model efficiently. The experimental results on eight HDI datasets show that the ANLFT model can obtain higher estimation performance gain than state-of-the-art models. The convergence performance of the proposed model is also competitive with that of state-of-the-art models.

High-dimensional and incomplete (HDI) data are frequently encountered in diverse real-world applications involving complex interactions among numerous nodes. Approaches based on latent feature analysis (LFA) have proven effective in performing representation learning in HDI data. Nevertheless, they cannot handle the high-order connectivity among nodes in HDI data well, resulting in severe accuracy loss. To address the previously mentioned issue, we present a novel model in this paper, namely Graph Linear Convolution Pooling Network (GLCPN). The proposed GLCPN adopts the three-fold ideas. First, it leverages simplified graph convolutions to efficiently capture high-order connectivity among nodes for learning representations of matrix factorization. Second, a simple yet effective priori convolution operator is adopted by each graph neural layer to capture node-node collaboration for aggregation. Third, a locality-enhanced pooling scheme is designed to holistically utilize multi-layer representations of the neighborhood. Therefore, GLCPN can effectively acquire the hidden information in HDI data with high efficiency. In addition, we have conducted a theoretical analysis demonstrating that the proposed GLCPN is more expressive compared with existing graph neural networks for HDI data. Extensive experiments have been further conducted on ten well-established HDI datasets from various applications. The experimental results demonstrate that the proposed GLCPN significantly outperforms state-of-the-art models for learning representations in HDI data evaluated by accuracy and efficiency metrics.

Low-rank latent factorization of tensors is a powerful method for analyzing high-dimensional and incomplete (HDI) data derived from cyber-physical systems, particularly when computational resources are limited. However, traditional tensor factorization models are inherently linear and struggle to capture the complex nonlinear spatiotemporal dependencies embedded in the data. This paper introduces a novel latent factorization model, namelyAuto-encodingNeuralTuckerFactorization (ANTucF) for accurate spatiotemporal representation learning on the HDI tensor. It constructs a low-rank Tucker factorization-based neural network to capture a potential latent manifold in space and time, built upon three core ideas: a) applying density-oriented modeling principles with neural networks to facilitate latent feature learning via positional and temporal encoding of mode indices; b) constructing a Tucker interaction tensor to represent all possible spatiotemporal interactions among distinct spatial and temporal modes; and c) enhancing the uniqueness of the core tensor in Tucker factorization by incorporating nonlinear spatiotemporal representation learning via auto-encoding latent interaction learning. The ANTucF model outperforms several state-of-the-art LFT models in estimating missing observations on real-world datasets. Additionally, visualizations demonstrate its ability to capture finer spatiotemporal dynamics by nonlinearly exploiting an optimal Tucker core tensor using a data-driven approach.

In this article, we present the dynamic graph mixer (DGM), a novel model for learning spatiotemporal-individual coupled features from high-dimensional and incomplete (HDI) tensors, which frequently represent dynamic interactions among real-world data samples. In contrast to existing methods, the proposed DGM possesses the following three advantages when learning representations from HDI tensors. First, it performs light graph message passing based on the conjoint attentions learned by jointly modeling latent features and implicit structures to extract the high-order connectivity. Second, a multilayer nonlinear tensor neural network (TNN) is adopted to learn the intricate attribute features of node-node-time from different views. Third, it follows the Tucker decomposition paradigm in a data density-oriented modeling mechanism to integrate node representations, preserving the overall multidimensional interaction patterns. In addition, we provide theoretical evidence that the key components in DGM can significantly improve expressiveness. Extensive experiments conducted on eight testing datasets of HDI tensors demonstrate that DGM outperforms state-of-the-art methods in both learning accuracy and efficiency.

An undirected weighted graph (UWG) is the fundamental data representation in various real applications. A graph convolution network is frequently utilized for representation learning to a UWG. Nevertheless, existing graph convolutional networks (GCNs) only consider a node's neighborhood during the embedding propagation, which regrettably decreases its representation learning capability due to the information loss in the modeling phase. Motivated by this discovery, this study proposes an adaptive neighborhood-resonated graph convolution network (ANR-GCN) with the following ideas: 1) establishing the weighted embedding propagation with the consideration of link weights in a UWG, thereby incorporating the interaction strength of each node pair into the ANR-GCN model; 2) building the neighborhood-regularization (NR) to make each node resonate with its neighborhoods, thus reinforcing the informative neighborhood information for improving the ANR-GCN's representation capability to the complex topology of the target UWG; and 3) diversifying the NR effects following the attention principle for guaranteeing the ANR-GCN's learning capacity. The proposed ANR-GCN's representation learning ability to a UWG is theoretically guaranteed from the perspectives of bounded generalization error and uniform stability. Extensive experiments on four UWG datasets illustrate that the proposed ANR-GCN significantly outperforms state-of-the-art GCNs in missing edge detection in a UWG, which evidently demonstrates its superior performance.

The selection and utilization of different color spaces significantly impact the recognition performance of deep learning models in downstream tasks. Existing studies typically leverage image information from various color spaces through model integration or channel concatenation. However, these methods result in excessive model size and suboptimal utilization of image information. In this study, we propose the structure-preserved self-attention network (SPSANet) model for efficient fusion of image information from different color spaces. This model incorporates a novel structure-preserved self-attention (SPSA) module that employs a single-head pixel-wise attention mechanism, as opposed to the conventional multihead self-attention (MHSA) approach. Specifically, feature maps from all color space grouping paths are utilized for similarity matching, enabling the model to focus on critical pixel locations across different color spaces. This design mitigates the dependence of the SPSANet model on the choice of color space while enhancing the advantages of integrating multiple color spaces. The SPSANet model also employs channel shuffle operations to facilitate limited interaction between information flows from different color space paths. Experimental results demonstrate that the SPSANet model, utilizing eight common color spaces-RGB, Luv, XYZ, Lab, HSV, YCrCb, YUV, and HLS-achieves superior recognition performance with reduced parameters and computational cost.

High-dimensional and incomplete (HDI) data are frequently encountered in big date-related applications for describing restricted observed interactions among large node sets. How to perform accurate and efficient representation learning on such HDI data is a hot yet thorny issue. A latent factor (LF) model has proven to be efficient in addressing it. However, the objective function of an LF model is nonconvex. Commonly adopted first-order methods cannot approach its second-order stationary point, thereby resulting in accuracy loss. On the other hand, traditional second-order methods are impractical for LF models since they suffer from high computational costs due to the required operations on the objective's huge Hessian matrix. In order to address this issue, this study proposes a generalized Nesterov-accelerated second-order LF (GNSLF) model that integrates twofold conceptions: 1) acquiring proper second-order step efficiently by adopting a Hessian-vector algorithm and 2) embedding the second-order step into a generalized Nesterov's acceleration (GNA) method for speeding up its linear search process. The analysis focuses on the local convergence for GNSLF's nonconvex cost function instead of the global convergence has been taken; its local convergence properties have been provided with theoretical proofs. Experimental results on six HDI data cases demonstrate that GNSLF performs better than state-of-the-art LF models in accuracy for missing data estimation with high efficiency, i.e., a second-order model can be accelerated by incorporating GNA without accuracy loss.

The absolute positioning accuracy of an industrial robot arm is vital for advancing manufacturing-related applications like automatic assembly, which can be improved via the data-driven approaches to robot arm calibration. Existing data-driven calibrators have illustrated their efficiency in addressing the issue of robot arm calibration. However, they mostly are single learning models that can be easily affected by the insufficient representation of the solution space, therefore, suffering from the calibration accuracy loss. To address this issue, this study proposes a calibrator fuzzy ensemble (CFE) with twofold ideas: 1) implementing eight data-driven calibrators relying on different sophisticated machine learning algorithms for an industrial robot arm, which guarantees the accuracy of individual base models and 2) innovatively developing a fuzzy ensemble of the obtained eight diversified calibrators to obtain impressively high calibration accuracy for an industrial robot arm. Extensive experiments on an ABB IRB120 industrial robot implemented with MATLAB demonstrate that compared with state-of-the-art calibrators, CFE decreases the maximum error at 8.59%. Hence, it has great potential for real applications.

Wireless sensor network (WSN) is an emerging and promising developing area in the intelligent sensing field. Due to various factors like sudden sensors breakdown or saving energy by deliberately shutting down partial nodes, there are always massive missing entries in the collected sensing data from WSNs. Low-rank matrix approximation (LRMA) is a typical and effective approach for pattern analysis and missing data recovery in WSNs. However, existing LRMA-based approaches ignore the adverse effects of outliers inevitably mixed with collected data, which may dramatically degrade their recovery accuracy. To address this issue, this article innovatively proposes a latent feature analysis (LFA) based spatiotemporal signal recovery (STSR) model, named LFA-STSR. Its main idea is twofold: 1) incorporating the spatiotemporal correlation into an LFA model as the regularization constraint to improve its recovery accuracy and 2) aggregating the -norm into the loss part of an LFA model to improve its robustness to outliers. As such, LFA-STSR can accurately recover missing data based on partially observed data mixed with outliers in WSNs. To evaluate the proposed LFA-STSR model, extensive experiments have been conducted on four real-world WSNs datasets. The results demonstrate that LFA-STSR significantly outperforms the related six state-of-the-art models in terms of both recovery accuracy and robustness to outliers.

An undirected weighted graph (UWG) is regularly adopted to portray the interactions among a solo set of nodes from big data-connected applications such as the interactive confidence between proteins in a protein network. A graph convolutional network (GCN) is able to represent a UWG for subsequent pattern analysis tasks such as missing link estimation. However, existing GCNs mostly neglect the local collaborative information hidden in connected node pairs, which leads to severe information loss. To address this issue, this study proposes a node-collaboration-informed graph convolutional network (NGCN) model for implementing the precise UWG representation learning with threefold ideas: 1) extracting the nodes' global graph characteristics via incorporating the residual connection and weighted representation propagation into the GCN module; 2) learning the nodes' local collaborative information from the observed interactive node pairs via a symmetric latent factor analysis (SLFA) module; and 3) designing an effective strategy to fuse the nodes' global graph characteristics and local collaborative information adaptively for highly accurate representation to the target UWG. Its high representation ability to target UWG is proved in theory. Empirical studies on six UWGs generated by real-world applications indicate that owing to its elegant modeling for the node collaborations, the proposed NGCN significantly outperforms several leading-edge models in estimation accuracy to the missing links of a UWG. Its high scalability ensures its compatibility with other GCN extensions, which will be investigated in the future.

Time-varying quality-of-service (QoS) data are usually utilized for Web service evaluation and selection. To accurately estimate the unknown information in time-varying QoS data, it is crucial to capture the temporal patterns hidden in the known data. The Non-negative Latent Factorization of Tensors (NLFT) model has performed well in describing the temporal patterns in time-varying QoS data. However, it assigns a single bias to each dimension of the target QoS tensor, making it suffer from estimation accuracy loss when describing the fluctuations of time-varying QoS data. To address this vital issue, this paper proposes an Adaptively Bias-extended NLFT (ABNT) model based on the fuzzy logic with two-fold ideas: a) extending the linear biases on each dimension of tensor for describing the complex fluctuations of QoS data precisely, b) building a fuzzy logic-incorporated particle swarm optimization algorithm to establish a self-adaptation mechanism for the count of extended linear biases and regularization coefficients. Detailed algorithms and analyses are provided for the proposed ABNT model. Empirical studies on two practical time-varying QoS datasets indicate that the estimation accuracy of the ABNT model outperforms that of state-of-the-art QoS data estimation models (with an average 23.94% improvement in MAE).

Industrial robots inevitably incur kinematic errors in the advanced manufacturing and assembly processes, resulting in the severe reduction of the absolute positioning accuracy (APA). Kinematic calibration (KC) is well-known as a vital technique in APA-promoting tasks. However, existing KC models generally adopt a single distance-oriented Loss, e.g., an$L_{2}$norm-oriented one that neglects the featured$L_{p}$norms. In response to this critical issue, this study presents an Adaptive p-norms-oriented Kinematic Calibration (ApKC) model on the basis of threefold ideas: 1) studying the effects of diversified$L_{p}$norms on the industrial robot calibration performance; 2) combining multiple$L_{p}$norms to obtain the aggregated loss with the hybrid effects by different norms; and 3) implementing the weight adaptation on the norm components of the aggregated loss, and rigorously prove its ensemble capability benefiting the calibration performance. Afterwards, a novel Newton interpolated Adaptive Differential Evolution (NADE) algorithm is further proposed to optimize the ApKC model. Empirical studies on an HRS JR680 industrial robot demonstrate that the achieved ApKC-NADE calibrator can significantly reduce the robot’s maximum positioning error from 4.610 to 0.856 mm. It can vigorously support the high-accuracy application of industrial robots.

Cryptocurrency services, as one of the most successful applications of blockchain technology, have recently garnered significant attention from the graph learning community. Its large-scale dynamic transaction records contain a variety of behavioral patterns and rich knowledge involving accounts, making the dynamic cryptocurrency transaction network embedding (DCTNE) a hot, yet thorny research topic. As the trading accounts increase and time accumulates, considerable transaction services are dispersed into various time slots, leading to very sparse transaction data within a time slot, that is, the transaction service data is high-dimensional and incomplete (HDI). To efficiently mine high-value knowledge from HDI data, this article proposes a proximal-ADMM-incorporated nonnegative latent-factorization-of-tensors (PNL) model for DCTNE that adopts threefold ideas: 1) incorporating the proximal terms into the alternating-direction-method-of-multipliers (ADMMs)-based learning scheme to reduce the oscillations for high estimation accuracy and fast convergence; 2) implementing a parallel training process with hyperparameter self-adaptation for high computational efficiency; and 3) proving that the proximal-incorporated learning scheme guarantees the convergence to a Karush–Kuhn–Tucker (KKT) stationary point. Experimental results on eight real-world DCTNs show that the PNL significantly outperforms several state-of-the-art (SOTA) models, demonstrating not only high efficiency and accuracy in performing DCTNE, but also strong potential to enhance the operational reliability and stability of cryptocurrency transaction systems.

high-dimensional and incomplete (HDI) matrices are commonly encountered in a variety of big data-related industrial applications, which describe complex interactions between entities. The complete interaction relationship in the HDI matrix is essential to deal with various problems such as pattern recognition in industrial applications. Therefore, estimating the missing data in the HDI matrix is crucial. latent factor analysis (LFA) models have achieved advanced results in solving such problems. However, the existing LFA models cannot model the nonlinear structure hidden in the HDI matrix. neural networks (NNs) can handle the nonlinearity in the HDI data, but their high estimation accuracy relies on high computation cost and storage burden. To address the aforementioned problems, this article proposes a novel NNLFA model. It contains the following primary ideas: 1) it can model the nonlinear structure of the HDI matrix efficiently through NNs and 2) it incorporates the NNs into the LFA model to improve estimation accuracy while maintaining high computational and storage efficiency. To validate the superiority of the NNLFA model, experiments with six state-of-the-art models are conducted on six practical industrial application datasets. The experimental results indicate that the NNLFA model enhances estimation accuracy by up to 33.3%. In addition, NNLFA model shows strong competitiveness in terms of both time and storage efficiency when compared to baseline models.

A dynamic directed graph (DDG) can describe complex dynamic interactions among massive entities, for example, traffic transmissions in a metropolitan area network (MAN), in a natural way. Due to the rapid expansion of a network, it is impossible to capture all the interactions at each time slot, making a resultant DDG be high-dimensional and incomplete (HDI). A nonnegative latent factorization of tensors (NLFT) model has proven to be highly efficient in extracting desired knowledge from an HDI DDG. Nevertheless, an existing NLFT model attempts to be easily affected by the instantaneous data fluctuations. Motivated by this discovery, this article innovatively proposes a convolution bias-incorporated NLFT (CB-NLFT) model with threefold ideas: 1) utilizing the Tucker decomposition framework for accurately representing the complex patterns hidden in an HDI DDG; 2) establishing a novel convolution bias scheme for precisely depicting the instantaneous data fluctuations; and 3) theoretically proving the CB-NLFT model’s convergence. Extensively empirical studies on six real-world datasets illustrate that the proposed CB-NLFT achieves significantly higher accuracy and computational efficiency when addressing the representation learning to a DDG in comparison with state-of-the-art models.

An online streaming feature selection (OSFS) model is highly efficient in processing the high-dimensional streaming features. In practical big data-related applications, streaming features are mostly highly-incomplete due to various unpredictable reasons like the privacy protection, leading to the issue of online sparse streaming feature selection (OS2FS). The incomplete streaming features can lead to the uncertain relationship between the labels and sparse features during the feature selection process, yet existing OSFS and OS2FS models focus on the certain relationships, resulting in accuracy loss by improperly-selected features. To address this critical issue, this article presents a three(3)-way decision-incorporated OS2FS (3WDO) model with the following two-fold ideas: 1) utilizing the latent factor analysis (LFA) approach to pre-estimate the missing data of the concerned sparse streaming features and 2) integrating the three-way decision (3WD) into the streaming features selection process for appropriately modeling the uncertainty within the label-feature interactions. By doing so, the uncertain relationships between labels and sparse features are characterized by more information and looser tolerance, thereby minimizing the decision risk of feature selection. Experimental results on twelve real-world datasets demonstrate that the proposed 3WDO model significantly outperforms seven state-of-the-art OSFS and OS2FS models, which strongly supports its ability of addressing practical issues.

To understand the mechanisms of complex systems, attributed graphs (AGs) are recognized as a valuable model by their capability of describing nontrivial topological structures and rich node contents, and their emergence raises new challenges on the task of graph clustering. Although a variety of computational algorithms have been proposed to perform accurate clustering analysis on AGs, most of them are incapable of inferring the cluster labels of nodes through links, thus falling short of explaining node behaviors on how to formulate overlapping clusters. Moreover, the vast amount of links considerably decreases the computation efficiency if they are explicitly taken into account for AG clustering. To overcome this problem, we present a novel variational Bayesian learning model, which avoids generating a complete AG by only simulating the generative process of its skeleton with the prior knowledge on the cluster labels of links. When addressing the inference problem, we develop an efficient algorithm, namely, LCAAG, for determining the optimal cluster labels of nodes by estimating local community structures of links. The convergence of LCAAG has been proved theoretically. Compared with several state-of-the-art algorithms, LCAAG has demonstrated its promising performance in terms of both accuracy and scalability on five different scaled benchmark datasets. The source code and datasets are available at https://github.com/shallowdreamoon/LCAAG.git.

Protein complexes play a crucial role in regulating various biological processes that govern cell activities. Numerous computational algorithms have been proposed to identify protein complexes from protein-protein interaction (PPI) networks. However, many of these algorithms face limitations in effectively leveraging multiview biological information of proteins, restricting their ability to capture the intricate characteristics of protein complexes in PPI networks. While deep learning-based algorithms have significantly advanced the identification of protein complexes, they often integrate graph representation learning techniques into traditional clustering algorithms without explicitly capturing the dependency between protein embeddings and resulting complexes. To address these issues, we present a multiview fusion neural network, named FMvPCI, for protein complex identification via fuzzy clustering. In FMvPCI, we introduce a novel multiview graph convolution encoder to effectively manipulate and fuse the biological information of proteins from different perspectives. Subsequently, the optimization of FMvPCI incorporates our expectations about protein complexes through the concept of fuzzy clustering. This approach unifies the embeddings of proteins and their cluster memberships within a coherent framework. Leveraging a heuristic search strategy, FMvPCI can discover overlapping protein complexes based on the cluster memberships of proteins. A series of experiments on five different PPI networks collected from two species have been conducted to evaluate the performance of FMvPCI by comparing it with state-of-the-art identification algorithms, and the results demonstrate the superior performance of FMvPCI by significantly improving the identification accuracy for protein complexes.

Taking incompatible multiple drugs together may cause adverse interactions and side effects on the body. Accurate prediction of drug-drug interaction (DDI) events is essential for avoiding this issue. Recently, various artificial intelligence-based approaches have been proposed for predicting DDI events. However, DDI events are associated with complex relationships and mechanisms among drugs, targets, enzymes, transporters, molecular structures, etc. Existing approaches either partially or loosely consider these relationships and mechanisms by a non-end-to-end learning framework, resulting in sub-optimal feature extractions and fusions for prediction. Different from them, this paper proposes a Multimodal Knowledge Graph Fused End-to-end Neural Network (MKGFENN) that consists of two main parts: multimodal knowledge graph (MKG) and fused end-to-end neural network (FENN). First, MKG is constructed by comprehensively exploiting DDI events-associated relationships and mechanisms from four knowledge graphs of drugs-chemical entities, drug-substructures, drugs-drugs, and molecular structures. Correspondingly, a four channels graph neural network is designed to extract high-order and semantic features from MKG. Second, FENN designs a multi-layer perceptron to fuse the extracted features by end-to-end learning. With such designs, the feature extractions and fusions of DDI events are guaranteed to be comprehensive and optimal for prediction. Through extensive experiments on real drug datasets, we demonstrate that MKG-FENN exhibits high accuracy and significantly outperforms state-of-the-art models in predicting DDI events. The source code and supplementary file of this article are available on: https://github.com/wudi1989/MKG-FENN.

Cognitive diagnosis (CD) utilizes students' existing studying records to estimate their mastery of unknown knowledge concepts, which is vital for evaluating their learning abilities. Accurate CD is extremely challenging because CD is associated with complex relationships and mechanisms among students, knowledge concepts, studying records, etc. However, existing approaches loosely consider these relationships and mechanisms by a non-end-to-end learning framework, resulting in sub-optimal feature extractions and fusions for CD. Different from them, this paper innovatively proposes an End-to-end Graph Neural Networks-based Cognitive Diagnosis (EGNN-CD) model. EGNN-CD consists of three main parts: knowledge concept network (KCN), graph neural networks-based feature extraction (GNNFE), and cognitive ability prediction (CAP). First, KCN constructs CD-related interaction by comprehensively extracting physical information from students, exercises, and knowledge concepts. Second, a four-channel GNNFE is designed to extract high-order and individual features from the constructed KCN. Finally, CAP employs a multi-layer perceptron to fuse the extracted features to predict students' learning abilities in an end-to-end learning way. With such designs, the feature extractions and fusions are guaranteed to be comprehensive and optimal for CD. Extensive experiments on three real datasets demonstrate that our EGNN-CD achieves significantly higher accuracy than state-of-the-art models in CD. Our code is available at https://github.com/Yangfl-1112/EGNN-CD.

With the rapid development of industry, there is a growing concern over the vibration control challenges associated with flexible structures and underactuated systems. Input shaping technology enables stable performance for high-speed motion in industrial motion systems. However, existing input shapers commonly suffer from the ineffective control performance caused by the ignorance of observation error errors. To address this critical issue, this paper proposes an Extended Kalman Filter-incorporated Residual Neural Network-based input Shaping (ERS) model for vibration control. Its main ideas are two-fold: a) adopting an extended Kalman filter to address a vertical flexible beam's model errors; and b) adopting a residual neural network to cascade with the extended Kalman filter for eliminating the remaining observation errors. Detailed experiments on a real dataset collected from a vertical flexible beam demonstrate that the proposed ERS model has achieved significant vibration control performance over several state-of-the-art models.

In this paper, we present Polarized message-passing (PMP), a novel paradigm to revolutionize the design of message-passing graph neural networks (GNNs). In contrast to existing methods, PMP captures the power of node-node similarity and dissimilarity to acquire dual sources of messages from neighbors. The messages are then coalesced to enable GNNs to learn expressive representations from sparse but strongly correlated neighbors. Three novel GNNs based on the PMP paradigm, namely PMP graph convolutional network (PMP-GCN), PMP graph attention network (PMP-GAT), and PMP graph PageRank network (PMP-GPN) are proposed to perform various downstream tasks. Theoretical analysis is also conducted to verify the high expressiveness of the proposed PMP-based GNNs. In addition, an empirical study of five learning tasks based on 12 real-world datasets is conducted to validate the performances of PMP-GCN, PMP-GAT, and PMP-GPN. The proposed PMP-GCN, PMP-GAT, and PMP-GPN outperform numerous strong message-passing GNNs across all five learning tasks, demonstrating the effectiveness of the proposed PMP paradigm.

With the increasing demand for automotive glass, improving the efficiency of automotive glass manufacturing systems can make full use of production resources and reduce the waste of natural and social resources. Therefore, this work aims to provide an efficient method for a real-world two-stage hybrid flow shop scheduling problem with small batches in an automotive glass manufacturing system. For the investigated problem, there is a significant setup time at the first stage, the second stage is served by machines that should not be interrupted, and each batch has a due date. Such constraints make this scheduling problem challenging. To solve this problem, a mixed integer linear programming model is established. Also, two properties and three theorems are given to enhance the problem-solving process. Subsequently, an efficient genetic algorithm is carefully designed to solve large-scale problems by considering the properties of the system. Meanwhile, an improvement scheme is proposed to decrease the running time of the algorithm, and experimental results show that this scheme can reduce the running time by 280 s at most from the average results of different scale problems . Finally, extensive experiments are carried out and a real-world case is solved to demonstrate the efficiency and effectiveness of the designed genetic algorithm. Also, the Taguchi method is adopted to tune the parameters for the designed algorithm.

Cluster tools are the key equipment in semiconductor manufacturing . For high-end chip manufacturing processes, to ensure the wafer quality , periodical cleaning operations for process chambers in such tools are required for eliminating the residual gas and chemicals. In operating a tool, the wafer processing time in a process chamber may vary within a range. It is meaningful to predict the cycle time of cluster tools with chamber cleaning operations and processing time variation. By doing so, automation material handling system can be told when a lot is completed by a cluster tool such that it can assign an overhead hoist transporter to transport this lot at the right time. This plays an important role in improving the productivity of a whole semiconductor manufacturing system . This work conducts the cycle time analysis of cluster tools with chamber cleaning operations and processing time variation. Specifically, it proposes a novel method to calculate the average cycle time of a single-arm cluster tool under which an optimal schedule can be obtained. Then, for a dual-arm cluster tool, an efficient algorithm is developed to approximate the average system cycle time under which an optimal schedule can be obtained. Experiments show that the gap between the average system cycle time obtained by simulations and the average system cycle time obtained by the proposed method is no more than 0.1% which demonstrate the effectiveness of the proposed method.

High-dimensional and incomplete (HDI) data are commonly encountered in various big data-related applications concerning the complex interactions among numerous nodes, such as the user-item iterations in a recommender system. A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can perform efficient representation learning to such HDI data, thereby extracting useful knowledge from them. However, a standard SGD algorithm updates a latent factor based on the current stochastic gradient only, without the considerations on the past information, making a resultant model suffer from slow convergence. To address this critical issue, this paper proposes an Adaptive Non-linear PID-incorporated SGD (ANPS) algorithm with two-fold ideas: 1) rebuilding the instant learning error when computing the stochastic gradient following the principle of a nonlinear PID controller to incorporate past update information into the learning scheme efficiently, and 2) implementing gain parameter adaptation following the principle of particle swarm optimization (PSO). Experiments on six widely-adopted HDI datasets demonstrate that compared with state-of-the-art LFA models, an ANPS-based LFA model achieves significant advantage in both efficiency and accuracy. Moreover, its flexible gain parameter adaptation mechanism greatly boosts its practicability for real issues.Note to Practitioners—In many industrial applications like recommender systems, social network systems, and cloud service systems, people usually encounter numerous nodes and their highly-incomplete relationships. An HDI matrix is commonly adopted to describe such specific relationships. One of the major challenges is to acquire useful knowledge from an HDI matrix efficiently and accurately for various data analysis tasks, e.g., accurate recommendation, community detection, and web service selection. An SGD-based LFA model has been widely adopted to tackle this issue. However, it suffers from slow convergence that leads to considerable time cost on large-scale datasets. This study proposes an ANPS algorithm following the principle of a nonlinear PID controller. With it, an ANPS-based LFA model is achieved, which possesses fast convergence rate on an industrial HDI matrix. The proposed ANPS algorithm can be leveraged for different types of various machine learning models, thereby improving their utility and scalability in practice.

Community is a fundamental and highly desired pattern in a Large-scale Undirected Network (LUN). Community detection is a vital issue when LUN representation learning is performed. Owing to its good scalability and interpretability, a Symmetric and Non-negative Matrix Factorization model is frequently utilized to tackle this issue. It adopts a unique Latent Factor (LF) matrix for precisely representing LUN’s symmetry, which, unfortunately, leads to a reduced LF space that decreases its representation learning ability to a target LUN. Motivated by this discovery, this study proposes a Symmetry and Graph Bi-regularized Non-negative Matrix Factorization (B-NMF) method that: a) leverages multiple LF matrices when representing LUN, thereby boosting the representation learning ability; b) constructs a symmetry regularization term that implies the equality constraint among its multiple LF matrices, thereby illustrating LUN’s intrinsic symmetry; and c) incorporates graph regularization into its learning objective, thereby illustrating LUN’s local geometry. A theoretical proof is given to theoretically validate B-NMF’s convergence ability. The regularization hyperparameters are selected by validating model modularity, thereby guaranteeing B-NMF’s practicability in addressing real application issues. Extensive experimental results on ten LUNs from real applications demonstrate that the proposed B-NMF-based community detector significantly outperforms several baseline and state-of-the-art models in achieving highly-accurate community detection results. Note to Practitioners—LUNs are very-commonly seen in real applications like a social network system. Communities in LUNs are vital for various knowledge discovery-related applications. For accurately detecting them, a detector should guarantee its high representation learning ability to a target LUN. To do so, this paper presents a B-NMF model that is able to perform precise representation learning to LUNs, thereby achieving accurate community detection results. In comparison with conventional Symmetric and Non-negative Matrix Factorization-based community detectors, a B-NMF-based community detector enjoys its enlarged latent feature space, which ensures its higher representation ability to a target LUN. It depends on two regularization hyperparameters, which can be selected by performing grid-search on the target LUN via its modularity evaluation. This paper gives the empirical values of B-NMF’s regularization hyperparameters based on the parametersensitivity tests on the involved experimental datasets. The proposed B-NMF model is shown to be highly suitable for addressing community detection and clustering tasks on LUNs from real applications.

Latent factor analysis (LFA) is efficient in knowledge discovery from a high-dimensional and incomplete (HDI) matrix frequently encountered in industrial big data-related applications. A stochastic gradient descent (SGD) algorithm is commonly adopted as a learning algorithm for LFA owing to its high efficiency. However, its sequential nature makes it less scalable when processing large-scale data. Although alternating SGD decouples an LFA process to achieve parallelization, its performance relies on its hyper-parameters that are highly expensive to tune. To address this issue, this paper presents three extended alternating SGD algorithms whose hyper-parameters are made adaptive through particle swarm optimization. Correspondingly, three Parallel Adaptive LFA (PAL) models are proposed and achieve highly efficient latent factor acquisition from an HDI matrix. Experiments have been conducted on four HDI matrices collected from industrial applications, and the benchmark models are LFA models based on state-of-the-art parallel SGD algorithms including the alternative SGD, Hogwild!, distributed gradient descent, and sparse matrix factorization parallelization. The results demonstrate that compared with the benchmarks, with 32 threads, the proposed PAL models achieve much speedup gain. They achieve the highest prediction accuracy for missing data on most cases.Note to Practitioners—HDI data are commonly encountered in many industrial big data-related applications, where rich knowledge and patterns can be extracted efficiently. An SGD based-LFA model is popular in addressing HDI data due to its efficiency. Yet when dealing with large-scale HDI data, its serial nature greatly reduces its scalability. Although alternating SGD can decouple an LFA process to implement parallelization, its performance depends on its hyper-parameter whose tuning is tedious. To address this vital issue, this study proposes three extended alternating SGD algorithms whose hyper-parameters are made via through a particle swarm optimizer. Based on them, three models are realized, which are able to efficiently obtain latent factors from HDI matrices. Compared with the existing and state-of-the-art models, they enjoy their hyper-parameter-adaptive learning process, as well as highly competitive computational efficiency and representation learning ability. Hence, they provide practitioners with more scalable solutions when addressing large HDI data from industrial applications.

High-dimensional and incomplete (HDI) interactions among numerous nodes are commonly encountered in a big data-related application, like user-item interactions in a recommender system. Owing to its high efficiency and flexibility, a stochastic gradient descent (SGD) algorithm can enable efficient latent feature analysis (LFA) of HDI data for its precise representation, thereby enabling efficient solutions to knowledge acquisition issues like missing data estimation. However, LFA on HDI data involves a bilinear issue, making SGD-based LFA a sequential process, i.e., the update on a feature can impact the results on the others. Intervening the sequence of SGD-based LFA on HDI data can affect the training results. Therefore, a parallel SGD algorithm to LFA should be designed with care. Existing parallel SGD-based LFA models suffer from a) low parallelization degree, and b) slow convergence, which significantly restrict their scalability. Aiming at addressing these vital issues, this paper presents an Adaptively-accelerated Parallel Stochastic Gradient Descent (AP-SGD) algorithm to LFA by: a) establishing a novel local minimum-based data splitting and scheduling scheme to reduce the scheduling cost among threads, thereby achieving high parallelization degree; and b) incorporating the adaptive momentum method into the learning scheme, thereby accelerating the convergence rate by making the learning rate and acceleration coefficient self-adaptive. The convergence of the achieved AP-SGD-based LFA model is theoretically proved. Experimental results on three HDI matrices generated by real industrial applications demonstrate that the AP-SGD-based LFA model outperforms state-of-the-art parallel SGD-based LFA models in both estimation accuracy for missing data and parallelization degree. Hence, it has the potential for efficient representation of HDI data in industrial scenes. The future study will focus on the GPU-deployed extensions of the proposed AP-SGD algorithm and an AP-SGD-based LFA model.

Role negotiation (RN) is situated at the initial stage of the role-based collaboration (RBC) methodology and is independent of the subsequent agent evaluation and role assignment (RA) processes. RN is to determine the roles and the resource requirements for each role. In existing RBC-related research, RN is assumed to be static. This means that the roles and the resource requirements for each role are predetermined by decision-makers. However, the resources allocated to each role can vary. At this time, iterative RN outcomes will have different RA results. There may not be a direct dominant relationship between different RA outcomes, especially when solving group role assignment (GRA) with multiple objectives (GRA++) problems, which makes it even more complex. To address these concerns, we introduce the original bilevel GRA++ (BGRA++) model. Specifically, at the lower level of BGRA++, a strategy is designed for quantifying iterative RNs. For the upper level, we introduce the novel GRA-NSGA-II algorithm for the RA process. Finally, we introduce the concept of decision tolerance to assist decision-makers in selecting the optimal solution from the multiple RNs. Last, simulation experiments are conducted to verify the robustness and practicability of the proposed method. Comparisons and discussions show that the proposed solution is highly competitive for solving the GRA++ problem with iterative RN.

This study focuses on exploring the solution to coordination in multirobot systems faced with competitive states, where the competition behavior can be described as a$k$-winner-take-all ($k$WTA) operation. To address the$k$WTA problem, a fuzzy$k$WTA (F-$k$WTA) network is proposed, which incorporates a fuzzy mechanism into the network that adapts the convergence rate of the F-$k$WTA network based on the error, leading to exceptional convergence behavior. In addition, a set of activation functions are provided to further enhance the capabilities of the F-$k$WTA network. In addition, the proposed network demonstrates remarkable convergence performance, as illustrated through both theorems and simulations. Finally, the practical application value of the F-$k$WTA network is demonstrated on a multirobot system.

A latent feature analysis (LFA) model is very efficient in representation learning to high-dimensional incomplete (HDI) data emerging from various Big Data-based applications. The stochastic gradient descent (SGD) is frequently adopted as the learning algorithm by an LFA model owing to its ease of implementation. However, the standard SGD-based LFA model's training process is serial, which greatly decreases its scalability on massive HDI data. On the other hand, all existing parallel SGD-based LFA models suffer from low speedup due to their frequent synchronizations during the training process. Motivated by this discovery, this article proposes an asynchronous parallel fuzzy stochastic gradient descent (APF-SGD) algorithm to establish an efficiently parallelized LFA model on shared memory with threefold ideas: first, decoupling the update interdependences among heterogeneous latent features alternatively, thereby achieving two parallelizable subtasks without learning information loss; second, proposing a novel parallelized learning scheme that eliminates the synchronizations from both the subtask and thread perspectives, i.e., subtasks and their affiliated threads are taken simultaneously and continuously without any synchronization; third, applying the fuzzy optimization to its hyperparameters, thus enabling its adaptation in training. Theoretical convergence proof illustrates that the newly proposed APF-SGD algorithm guarantees the convergence of a resultant LFA model. Experimental results on four real HDI datasets illustrate that an APF-SGD-based LFA model outperforms several state-of-the-art parallel SGD-based LFA models in both missing data estimation accuracy and parallelization speedup.

A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can obtain superior performance when performing representation to a high-dimensional and incomplete (HDI) matrix, which is encountered in various big data-related applications cause of the great demand for describing the highly complex interactions among tremendous nodes. However, an SGD-based LFA model is often stacked by slow convergence since a standard SGD algorithm updates a single latent factor depending on the stochastic gradient of current instance learning error, which disregards the past learning information. To address this critical issue, this paper innovatively proposes a Fuzzy PID-incorporated SGD (FPS) algorithm with the following two-fold ideas: a) refining the instance learning error by modeling the past update information guided by the principle of a PID controller efficiently, and b) designing a fuzzy reasoning process to implement the gain parameter adaptation in a PID controller effectively. With it, an FPS-based LFA model is further built for fast and accurate latent factor analysis on an HDI matrix. Experiments conducted on six HDI datasets reveal that the proposed FPS-based LFA model surpasses state-of-the-art LFA models in computational efficiency and accuracy when estimating missing data within an HDI matrix.

Calibration technology has become an essential part of the task of improving robot precision. However, existing calibration schemes suffer from accuracy loss in actual working conditions due to the ignorance of the target robot's constrained workspace. In response to this challenge, this study proposes a novel robot calibrator with three-fold ideas: 1) optimizing the measurement configuration via the measurement configurations selection (MCS) for suppressing the measurement noises, 2) developing the alternation-direction-method-of-multipliers with multiplanar constraints (AMPC) algorithm for superior calibration accuracy and mitigatory long-tail convergence, with its convergence being theoretically proved, and 3) building the MCS-AMPC-based robot calibrator for efficient kinematic parameters calibration. For validating its performance, the HRS-P dataset that contains 2400 samples collected from different working spaces of an HRS-JR680 industrial robot has been established and publicly released. Empirical studies conclusively affirm that, compared to the Levenberg–Marquardt algorithm (the most accurate among several state-of-the-art algorithms), the proposed AMPC algorithm reduces the mean error by 10.34%. After denoising the calibration data with MCS and then employing the AMPC algorithm to identify the robot's kinematic parameters, the calibration accuracy can be further improved. Moreover, building an ensemble to address the issue of various local optimal solutions can also contribute to better calibration accuracy. The empirical results highlight the superiority and scalability of the MCS-AMPC calibrator for industrial robot-related applications.

In the field of artificial intelligence for information technology operations, operational data are often modeled as aperiodic multivariate time series, which contain rich multidimensional and nonlinear patterns. However, the existing approaches are unable to effectively acquire knowledge and recognize patterns due to their reliance on processing and modeling periodic patterns. To address this issue, this article proposes a novel deep-saliency-aware dual embedded attention network for aperiodic multivariate time-series forecasting. Our network consists of three main components: 1) a convolutional-neural-network- and transformer-based component for saliency representation of the aperiodic patterns; 2) a lightweight recurrent neural network component for capturing long-term dependence features; and 3) an attention mechanism for fusing the latent representations from the former components. Extensive empirical studies are conducted on a real-world dataset and five other public datasets to evaluate the proposed network against four state-of-the-art models. The results show that our method achieves impressive high performance on most evaluation metrics. Furthermore, the data and code used in this study are publicly available, which can facilitate progress in the community.

For nonconvex optimization problems, a routine is to assume that there is no perturbation when executing the solution task. Nevertheless, dealing with the perturbation in advance may increase the burden on the system and take up extra time. To remedy this weakness, we propose a robust coevolutionary neural-based optimization algorithm with inherent robustness based on the hybridization between the particle swarm optimization and a class of robust neural dynamics (RND). In this framework, every neural agent guided by the RND supersedes the place of the particle, mutually searches for the optimal solution, and stabilizes itself from different perturbations. The theoretical analysis ensures that the proposed algorithm is globally convergent with probability one. Besides, the effectiveness and robustness of the proposed approach are illustrated by illustrative examples compared with the existing methods. We further apply this proposed algorithm to the source localization and manipulability optimization of the redundant manipulator, simultaneously disposing of perturbation from the internal and exogenous system with satisfactory performance.

Performing highly accurate representation learning on a high-dimensional and sparse (HiDS) matrix is of great significance in a big data-related application such as a recommender system. A latent factor (LF) model is one of the most efficient approaches to the HiDS matrix representation. However, an LF model's representation learning ability relies heavily on an HiDS matrix's known data density, which is extremely low due to numerous missing data entities. To address this issue, this work proposes a prediction-sampling-based multilayer-structured LF (PMLF) model with twofold ideas: 1) constructing a loosely connected multilayered LF architecture to increase the known data density of an input HiDS matrix by generating synthetic data layer by layer and 2) constraining this synthetic data generating process through a random prediction-sampling strategy and nonlinear activations to avoid overfitting. In the experiments, PMLF is compared with six state-of-the-art LF-and deep neural network (DNN)-based models on four HiDS matrices from industrial applications. The results demonstrate that PMLF outperforms its peers in well-balancing prediction accuracy and computational efficiency.

The augmented Sylvester equation, as a comprehensive equation, is of great significance and its special cases (e.g., Lyapunov equation, Sylvester equation, Stein equation) are frequently encountered in various fields. It is worth pointing out that the current research on simultaneously eliminating the lagging error and handling noises in the nonstationary complex-valued field is rather rare. Therefore, this article focuses on solving a nonstationary complex-valued augmented Sylvester equation (NCASE) in real time and proposes two modified recurrent neural network (RNN) models. The first proposed modified RNN model possesses gradient search and velocity compensation, termed as RNN-GV model. The superiority of the proposed RNN-GV model to traditional algorithms including the complex-valued gradient-based RNN (GRNN) model lies in completely eliminating the lagging error when employed in the nonstationary problem. The second model named complex-valued integration enhanced RNN-GV with the nonlinear acceleration (IERNN-GVN) model is proposed to adapt to a noisy environment and accelerate the convergence process. Besides, the convergence and robustness of these two proposed models are proved via theoretical analysis. Simulative results on an illustrative example and an application to the moving source localization coincide with the theoretical analysis and illustrate the excellent performance of the proposed models.

High-Dimensional and Incomplete (HDI) data are frequently encountered in various Big Data-related applications. Despite its incompleteness, an HDI data repository contains rich knowledge and patterns concerning the complex interactions among numerous nodes. Recently, a Neural Network (NN)-based approach to Latent Feature Analysis (LFA) model becomes popular owing to its strong representation learning ability to HDI data. Nevertheless, existing NN-based LFA models neglect the inherent nonnegativity in most HDI data, resulting in representation accuracy loss. Motivated by this discovery, this study innovatively proposes a Fast Nonnegative AutoEncoder (FNAE)-based approach to LFA on HDI data, whose ideas are three-fold: a) constructing a multilayered autoencoder subject to nonnegativity constraints for high representation learning ability; b) incorporating the data density-oriented modeling mechanism into FNAE's input and output layers for high computational and storage efficiency; and c) implementing an Adam-based single latent factor-dependent, nonnegative and multiplicative update algorithm for efficient model training as well as fulfilling the nonnegativity constraints. Experimental results on eight commonly-adopted HDI matrices from industrial applications demonstrate that the proposed FNAE significantly outperforms several state-of-the-art NN-based LFA models in both estimation accuracy for missing links of an HDI matrix and computational efficiency.

A Dynamically Weighted Directed Network (DWDN) fundamentally illustrates the complex interactions among massive nodes from a big-data-oriented application, like the dynamic interactions among numerous terminals in a metropolitan network management system (MNMS). A High-Dimensional and Incomplete (HDI) tensor is able to flexibly quantize it, where lots of entries are missing primarily due to the impossibility in discovering the full interactions among numerous nodes. Such an HDI tensor can be effectively represented by a Latent Factorization of Tensors (LFT) model for extracting useful knowledge like potential links from it, while existing LFT models commonly adopt general regularization schemes without considering an HDI tensor's imbalanced known data, which impairs their generality. To address this issue, this paper develops an Fine-grained Regularized Nonnegative Latent factorization of tensors (FRNL) model based on two-fold ideas: a) innovatively proposing an Swish-p-based and fine-grained regularization scheme where the regularization effects acting on individual latent feature is proportional to its related instance count for precisely representing the imbalanced distribution of an HDI tensor's known data; b) implementing the self-adaptation of the model hyper-parameters via a fuzzy controller to achieve high practicability. The convergence ability of FRNL is justified theoretically. Experimental studies on eight DWDNs emerging from a real MNMS illustrate that compared with state-of-the-art LFT models, the proposed FRNL model obtains significantly higher learning accuracy and computational efficiency in representing a DWDN.

High-dimensional and incomplete (HDI) data are omnipresent in a variety of Big Data-related applications. Latent feature analysis (LFA) is a typical representation learning method that can extract useful yet latent knowledge from HDI data via low-rank embedding. Existing LFA-based models mostly adopt a single-metric-based modeling strategy, where the representation designed for the embeddingLossfunction is fixed and exclusive. However, real-world HDI data are commonly heterogeneous and have large diverse underlying patterns, making a single-metric-based model cannot represent such HDI data in a comprehensive and unbiased fashion. Motivated by this discovery, this article proposes a multi-metric latent feature (MMLF) model whose ideas are two-fold: 1) two vector spaces and threeLp-norms are simultaneously adopted to develop six LFA variants, each of which possesses a unique merit, and 2) all the variants are aggregated with a tailored, self-adaptive weighting strategy. As such, the proposed MMLF enjoys the merits originated from a set of disparate metric spaces all at once, achieving the comprehensive and unbiased representation of HDI data. Theoretical study guarantees that MMLF attains evident performance gain. Extensive experiments on ten real-world HDI matrices, spanning a wide range of industrial and scientific areas, verify that the proposed MMLF significantly outperforms nine state-of-the-art, shallow and deep counterparts.

A latent factor analysis (LFA) model can be efficiently built via the stochastic gradient descent (SGD) algorithm to address high-dimensional and incomplete (HDI) data generated by various big data-related applications. However, its performance depends hugely on its hyperparameters, which is conventionally decided through the grid-search that yields extremely high-computational costs. Particle swarm optimization (PSO)-based hyperparameter adaptation provides a potential solution to this severe problem, while it leads to accuracy loss due to the untimely convergence of the PSO algorithm. Moreover, existing PSO extensions mostly perplex the evolution scheme to make the hyperparameter adaptation time-consuming and computationally expensive. Aiming at implementing an accurate and adaptive LFA model, a pseudo gradient-adjusted PSO (PGA-PSO) algorithm with twofold ideas is proposed: 1) modeling the position transitions in a PSO algorithm as the pseudo gradients for optimizing each particle’s position and 2) incorporating the principle of adaptive moment estimation into the pseudo gradient estimation to make it consider previous pseudo gradient information for addressing the untimely issue. With it, a PGA-PSO-incorporated LFA (PPL) model is successfully constructed. The empirical studies on six HDI datasets demonstrate that owing to the efficient hyperparameter adaptation implemented by the proposed PGA-PSO algorithm, the obtained PPL model surpasses state-of-the-art LFA models in accuracy when predicting an HDI matrix’s missing data. Hence, it satisfies the scalability and efficiency demands emerging from practical applications.

Complete and valid spatio-temporal traffic data play a vital role in intelligent transportation systems applications, such as congestion avoidance and route guidance. However, traffic data from real-world scenarios is usually incomplete or corrupted due to communication or sensors malfunctions, which makes the traffic analytics difficult. Since traffic data contains complex spatio-temporal patterns, it is very challenging to develop an efficient learning model that can accurately recover incomplete traffic data. To tackle this issue, this work propose a Tucker Decomposition-based Latent factorization of tensors (TDL) model with two interesting ideas: 1) modeling spatio-temporal traffic data as an incomplete third-order tensor and building a Tucker decomposition based learning objective according to the density-oriented principle for precisely recovering missing traffic data; and 2) adopting a proportional-integral-derivative (PID) control principle-incorporated parameters learning scheme for achieving high computational efficiency. Empirical studies on four traffic speed datasets generated from different cities demonstration that the proposed TDL model achieves significant performance gain in both accuracy and computational efficiency compared with state-of-the-art models.

Online streaming feature selection (OSFS) is an effective approach to addressing high-dimensional data. In real big data-related applications, streaming features commonly have massive missing data due to various uncertain factors. The missing data may cause some uncertain relationships between sparse features and labels. However, existing OSFS methods tend to select sparse streaming features based on certain relevance and redundancy analysis, which may erroneously discard some weak relevant but irredundant features. As a result, some essential information is discarded. Motivated by this, this paper proposes a Decision- Risk-incorporated OSFS (DRO) algorithm. Its main idea is two-fold: 1) the missing data of sparse streaming features are pre-estimated by using the Latent Factor Analysis (LFA), and 2) the decision risk of relevance and redundancy analysis is minimized on the estimated complete streaming features via the three-way decision (3WD). Extensive empirical studies are conducted on eight real-world datasets. The results show that DRO significantly outperforms five state-of-the-art competitors.

Mid-term electricity load forecasting (LF) plays a critical role in power system planning and operation. To address the issue of error accumulation and transfer during the operation of existing mid-term LF models, a novel model called error correction based LF (ECLF) is proposed in this paper, which is designed to provide more accurate and stable LF. Firstly, time series analysis and feature engineering act on the original data to decompose load data into three components and extract relevant features. Then, based on the idea of stacking ensemble, long short-term memory is employed as an error correction module to forecast the components separately, and the forecast results are treated as new features to be fed into extreme gradient boosting for the second-step forecasting. Finally, the component sub-series forecast results are reconstructed to obtain the final LF results. The proposed model is evaluated on real-world electricity load data from two cities in China, and the experimental results demonstrate its superior performance compared to the other benchmark models.

An Undirected Weighted Network (UWN) is frequently encountered in a big-data-related application concerning the complex interactions among numerous nodes. A Symmetric High-Dimensional and Incomplete (SHDI) matrix can smoothly illustrate such a UWN, which contains rich knowledge like node interaction behaviors and local complexes. To extract desired knowledge from an SHDI matrix, an analysis model should carefully consider its topology for describing a UWN's intrinsic symmetry precisely. Representation learning to a UWN borrows the success of a pyramid of symmetry-aware models like a Symmetric Nonnegative Matrix Factorization (SNMF) model whose objective function utilizes a sole Latent Factor (LF) matrix for representing SHDI's symmetry precisely. However, they suffer from the following drawbacks: 1) their computational complexity is high; and 2) their modeling strategy narrows their representation features, making them suffer from low learning ability. Aiming at addressing the above critical issues, this paper proposes a Multi-constrained Symmetric Nonnegative Latent-factor-analysis (MSNL) model with two-fold ideas: 1) introducing multi-constraints composed of multiple LF matrices, i.e., inequality and equality ones into a data-density-oriented objective function for precisely representing the intrinsic symmetry of an SHDI matrix with broadened feature space; and 2) implementing an alternating direction method of multipliers (ADMM)-incorporated learning scheme for efficiently solving such a multi-constrained model. Empirical studies on three SHDI matrices from a real bioinformatics or industrial application demonstrate that the proposed MSNL model achieves higher representation accuracy than state-of-the-art models do, as well as promising computational efficiency.

Motion-force control of redundant manipulators is universally regarded as a pivotal issue in industrial manufacturing, especially for the processing of precision instruments. This paper proposes a kinematics-based motion-force control (KBMFC) scheme for redundant manipulators, which is driven by joint velocity commands and different from the dynamics-based methods. Specifically, the force and motion are modeled and decoupled in the end-effector frame with the help of a stiffness coefficient. To control the orientation of the force, a quaternion control equation is designed by combining the rotation matrix and neural dynamics method. Different from traditional motion-force control methods, the proposed scheme is constructed as quadratic programming with the corresponding recurrent neural network (RNN) solver derived, which considers the kinematic optimization index and joint constraints. According to the generated control signals, a redundant manipulator is able to accurately fulfill the hybrid control of motion and force with the desired quaternion, which is intuitively confirmed by simulations and experiments.Note to Practitioners—This paper is motivated by the deficiencies that restrict the real-world applications of the motion-force control of redundant manipulators. On the one hand, most existing motion-force control schemes are implemented under the framework of dynamics, which inevitably leads to some kinematics-related defects. On the other hand, the latest kinematics-based techniques introduce an admittance control to achieve motion-force control. However, they model the force in the Z-axis of the base coordinate while the motion planning is limited in the X-Y plane, which dramatically reduces real-world applications. In this paper, the deformation force is designed in the end-effector frame, and a quaternion control technology of the end-effector is developed. Such a scheme can realize the real-time control of the orientation and magnitude of the force while ensuring trajectory tracking. In addition, the introduction of optimization indexes and joint constraints dramatically improves the functionality of the proposed scheme. Finally, the contributions of this paper are verified through simulations, experiments and comparisons. This work proposes a feasible framework for the motion-force control and orientation control of redundant manipulators.

A Non-negative Latent-factorization-of-tensors model relying on aNonnegative andMultiplicativeUpdate onIncompleteTensors (NMU-IT) algorithm facilitates efficient representation learning to aDynamicWeightedDirectedNetwork (DWDN). However, a NMU-IT algorithm leads to slow model convergence and inefficient selection of hyper-parameters. Aiming to address these challenging issues, this work proposes a Momentum-incorporated Biased Non-negative and Adaptive Latent-factorization-of-tensors (MNL) model. It adopts two-fold ideas: 1) incorporating a generalized momentum method into the NMU-IT algorithm to enable fast model convergence; 2) facilitating hyper-parameter slef-adaptation via Particle Swarm Optimization. Empirical studies on four real DWDNs indicate that the proposed MNL is superior to state-of-the-art models in performing efficient representation learning to a DWDN, which is definitely supported by its high computational efficiency and prediction accuracy for missing links of a DWDN. Moreover, its hyper-parameter-free training enables its high practicability in real scenes.

Protein-protein interactions (PPIs) play a critical role in the proteomics study, and a variety of computational algorithms have been developed to predict PPIs. Though effective, their performance is constrained by high false-positive and false-negative rates observed in PPI data. To overcome this problem, a novel PPI prediction algorithm, namely PASNVGA, is proposed in this work by combining the sequence and network information of proteins via variational graph autoencoder. To do so, PASNVGA first applies different strategies to extract the features of proteins from their sequence and network information, and obtains a more compact form of these features using principal component analysis. In addition, PASNVGA designs a scoring function to measure the higher-order connectivity between proteins and so as to obtain a higher-order adjacency matrix. With all these features and adjacency matrices, PASNVGA trains a variational graph autoencoder model to further learn the integrated embeddings of proteins. The prediction task is then completed by using a simple feedforward neural network. Extensive experiments have been conducted on five PPI datasets collected from different species. Compared with several state-of-the-art algorithms, PASNVGA has been demonstrated as a promising PPI prediction algorithm.

An echo state network (ESN) draws widespread attention and is applied in many scenarios. As the most typical approach for solving the ESN, the matrix inverse operation of high computational complexity is involved. However, in the modern big data era, addressing the heavy computational burden problem is necessary. In order to reduce the computational load, an inverse-free ESN (IFESN) is proposed for the first time in this article. Besides, an incremental IFESN is constructed to attain the network topology with theoretical proof on the training error's monotone decline property. Simulations and experiments are conducted on several numerical and real-world time-series benchmarks, and corresponding results indicate that the proposed model is superior to some existing models and possesses excellent practical application potential. The source code is publicly available at https://github.com/LongJin-lab/the-supplementary-file-for-CYB-E-2021-04-0944.

In this article, a distributed and time-delayed k-winner-take-all (DT-kWTA) network is established and analyzed for competitively coordinated task assignment of a multirobot system. It is considered and designed from the following three aspects. First, a network is built based on a k-winner-take-all (kWTA) competitive algorithm that selects k maximum values from the inputs. Second, a distributed control strategy is used to improve the network in terms of communication load and computational burden. Third, the time-delayed problem prevalent in arbitrary causal systems (especially, in networks) is taken into account in the proposed network. This work combines distributed kWTA competition network with time delay for the first time, thus enabling it to better handle realistic applications than previous work. In addition, it theoretically derives the maximum delay allowed by the network and proves the convergence and robustness of the network. The results are applied to a multirobot system to conduct its robots' competitive coordination to complete the given task.

With the rapid development of services computing in the past decade, Quality-of-Service (QoS)-aware selection of Web services has become a hot yet thorny issue. Conducting warming-up tests on a large set of candidate services for QoS evaluation is time consuming and expensive, making it vital to implement accurate QoS-estimators. Existing QoS-estimators barely consider the temporal patterns hidden in QoS data. However, such data are naturally time dependent. For addressing this critical issue, this study presents a Kalman-filter-incorporated latent factor analysis (KLFA)-based QoS-estimator for accurate representation to temporally dynamic QoS data. Its main idea is to make the user latent features (LFs) time dependent, while the service ones time consistent. A novel iterative training scheme is designed, where the user LFs are learned through a Kalman filter for precisely modeling the temporal patterns, and the service ones are alternatively trained via an alternating least squares algorithm for precisely representing the historical QoS data. Empirical studies on large-scale and real Web service QoS datasets demonstrate that the proposed KLFA model significantly outperforms state-of-the-art QoS-estimators in estimation accuracy for dynamic QoS data.

Performing an accurate clustering analysis is of great significance for us to understand the behavior of complex networks, and a variety of graph clustering algorithms have, thus, been proposed to do so by taking into account network topology and node attributes. Among them, fuzzy clustering algorithm for complex networks (FCAN) is an established fuzzy clustering algorithm that optimizes the memberships of nodes based on dense structures and content relevance. This article proposes an improved fuzzy-based graph clustering algorithm, namely FCAN-multi objective particle swarm optimization (MOPSO) that retains all the benefits associated with FCAN while achieving significantly increased convergence rate using multiobjective particle swarm optimization (MOPSO). To do so, FCAN-MOPSO first modifies the original optimization model of FCAN by adopting an instance-frequency-weighted regularization, which enhances the ability of FCAN-MOPSO to handle the imbalance observed in the distribution of fuzzy memberships of nodes. After that, FCAN-MOPSO decomposes its optimization problem into a set of suboptimization problems. Following the MOPSO framework, FCAN-MOPSO develops an effective solution to reach a consensus optimization among them by balancing the global exploration and local exploitation abilities of particles. A theoretical analysis is provided to prove the global convergence of FCAN-MOPSO. Extensive experiments have been conducted to evaluate the performance of FCAN-MOPSO on five real-world complex networks with different scale, and experimental results demonstrate that when compared with state-of-the-art clustering algorithms, FCAN-MOPSO achieves a better accuracy performance with improved convergence. Hence, FCAN-MOPSO is a promising graph clustering algorithm to precisely and efficiently discover clusters in complex networks.

With the rapid development and wide applications of industrial manipulators, a vital concern rises regarding a manipulator's absolute positioning accuracy. The manipulator calibration models have proven to be highly efficient in improving the absolute positioning accuracy of an industrial manipulator. However, existing calibration models commonly suffer from the low calibration accuracy caused by the ignorance of nongeometric errors. To address this critical issue, this article proposes anextended Kalman filter-incorporatedResidual Neural Network-basedCalibration (ERC) model for kinematic calibration. Its main ideas are two-fold: 1) adopting anextended Kalman filter (EKF) to address a manipulator's geometric errors; and 2) adopting aresidual neural network to cascade with theEKFfor eliminating the remaining nongeometric errors. Detailed experiments on three real datasets collected from industrial manipulators demonstrate that the proposed ERC model has achieved significant calibration accuracy gain over several state-of-the-art models.

Intelligent Transportation Systems (ITSs) are vital for alleviating traffic congestion and improving traffic efficiency. Due to the delay of network transmission and failure of detectors, massive missing traffic data are often produced in ITSs, which evidently decreases the accuracy of decision-making in road traffic management. Hence, how establishing a precise and efficient estimation of missing traffic data becomes a hot yet thorny issue. Low-rank matrix completion (LR-MC) model has proven to be highly effective to address this issue owing to its fine representativeness of such high-dimensional and incomplete data. However, the existing LR-MC models mostly fail to model the inherently temporal and spatial correlations hidden in traffic network structure, resulting in low estimation accuracy. To improve it, this paper proposes a Hessian regularization spatio-temporal low rank (HRST-LR) algorithm with three main-fold ideas: a) imposing low-rank property into the global features of a traffic matrix for precisely learning its structure, b) capturing the temporal evolvement via a second-order difference of time-series constraint, and c) modeling the similar space of road segments through a Hessian regularization spatial constraint, thus exploring the local correlation between road segments for representing the spatial patterns in the traffic data. Experimental results on four traffic data sets prove that HRST-LR outperforms several state-of-the-art methods in the missing traffic data estimation with the root mean squared error improvements often higher than 14% when the missing rate is 90%. Hence, the HRST-LR algorithm is highly valuable for traffic data imputation with the need of performing spatio-temporal low-rank analysis.

A High-Dimensional and Incomplete (HDI) tensor is frequently encountered in a big data-related application concerning the complex dynamic interactions among numerous entities. Traditional tensor factorization-based models cannot handle an HDI tensor efficiently, while existing latent factorization of tensors models are all linear models unable to model an HDI tensor's nonlinearity. Motivated by this critical discovery, this paper proposes a Neural Latent Factorization of Tensors model, which provides a novel approach to nonlinear Canonical Polyadic decomposition on an HDI tensor. It is implemented with three-fold interesting ideas: a) adopting the density-oriented modeling principle to build rank-one tensor series with high computational efficiency and affordable storage cost; b) treating each rank-one tensor as a hidden neuron to achieve an efficient neural network structure; and c) developing an adaptive backward propagation (ABP) learning scheme for efficient model training. Experimental results on six HDI tensors from a real system demonstrate that compared with state-of-the-art models, the proposed model achieves significant performance gain in both convergence rate and accuracy. Hence, it is of great significance in performing challenging HDI tensor analysis.

A fast non-negative latent factor (FNLF) model for a high-dimensional and sparse (HiDS) matrix adopts a Single Latent Factor-dependent, Non-negative, Multiplicative and Momentum-incorporated Update (SLF-NM2U) algorithm, which enables its fast convergence. It is crucial to achieve a rigorously theoretical proof regarding its fast convergence, which has not been provided in prior research. Aiming at addressing this critical issue, this work theoretically proves that with an appropriately chosen momentum coefficient, SLF-NM2U enables the fast convergence of an FNLF model in both continuous and discrete time cases. Empirical analysis of HiDS matrices generated by representative industrial applications provides empirical evidences for the theoretical proof. Hence, this study represents an important milestone in the field of HiDS matrix analysis.

Recently, robot arms have become an irreplaceable production tool, which play an important role in the industrial production. It is necessary to ensure the absolute positioning accuracy of the robot to realize automatic production. Due to the influence of machining tolerance, assembly tolerance, the robot positioning accuracy is poor. Therefore, in order to enable the precise operation of the robot, it is necessary to calibrate the robotic kinematic parameters. The least square method and Levenberg-Marquardt (LM) algorithm are commonly used to identify the positioning error of robot. However, it generally has the overfitting caused by improper regularization schemes. To solve this problem, this article discusses six regularization schemes based on its error models, i.e.,$L_{1}$,$L_{2}$, dropout, elastic, log, and swish. Moreover, this article proposes a scheme with six regularization to obtain a reliable ensemble, which can effectively avoid overfitting. The positioning accuracy of the robot is improved significantly after calibration by enough experiments, which verifies the feasibility of the proposed method.

Large-scale undirected weighted networks are frequently encountered in big-data-related applications concerning interactions among a large unique set of entities. Such a network can be described by a Symmetric, High-Dimensional, and Incomplete (SHDI) matrix whose symmetry and incompleteness should be addressed with care. However, existing models fail in either correctly representing its symmetry or efficiently handling its incomplete data. For addressing this critical issue, this study proposes an Alternating-Direction-Method of Multipliers (ADMM)-based Symmetric Non-negative Latent Factor Analysis (ASNL) model. It adopts fourfold ideas: 1) implementing the data density-oriented modeling for efficiently representing an SHDI matrix's incomplete and imbalanced data; 2) separating the non-negative constraints from the decision parameters to avoid truncations during the training process; 3) incorporating the ADMM principle into its learning scheme for fast model convergence; and 4) parallelizing the training process with load balance considerations for high efficiency. Empirical studies on four SHDI matrices demonstrate that ASNL significantly outperforms several state-of-the-art models in both prediction accuracy for missing data of an SHDI and computational efficiency. It is a promising model for handling large-scale undirected networks raised in real applications.

Historical Quality-of-Service (QoS) data describing existing user-service invocations are vital to understanding user behaviors and cloud service conditions. Collaborative Filtering (CF) models based on Matrix Factorization (MF) have proven to be highly efficient in performing representation learning in QoS data. However, its performance is hindered by its linear inherence and implicit encoding of collaborative QoS signal. To address this critical issue, we present a novel approach in this article, dubbed asTwo-streamGraph convolutional network-incorporatedLatentFeatureAnalysis (TGLFA). The proposed TGLFA is significantly different from previous approaches to representation learning in QoS data in the following three aspects. First, it constructs a multilayer fully-connected network to capture the attribute characteristics representing the nonlinear latent features of service. Second, TGLFA constructs a biparty graph to represent the user-service interactions, where the light graph convolutional network is adopted to acquire the high-order connectivity in QoS data. Last, Aiming to improve computational efficiency, the proposed approach considers the mechanism for data density-oriented modeling when building the input and output layers. Detailed experimental results on eight large-scale cases constructed on two real QoS datasets demonstrate that the proposed TGLFA significantly outperforms its state-of-the-art peers in both estimation accuracy for missing QoS data and computational efficiency. The notable results show that TGLFA is a novel and effective approach to QoS data representation learning.

Service-oriented applications commonly involve high-dimensional and sparse (HiDS) interactions among users and service-related entities, e.g., user-item interactions from a personalized recommendation services system. How to perform precise and efficient representation learning on such HiDS interactions data is a hot yet thorny issue. An efficient approach to it is latent factor analysis (LFA), which commonly depends on large-scale non-convex optimization. Hence, it is vital to implement an LFA model able to approximate second-order stationary points efficiently for enhancing its representation learning ability. However, existing second-order LFA models suffer from high computational cost, which significantly reduces its practicability. To address this issue, this paper presents a Momentum-accelerated Hessian-vector algorithm (MH) for precise and efficient LFA on HiDS data. Its main ideas are two-fold: a) adopting the principle of a Hessian-vector-product-based method to utilize the second-order information without manipulating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning scheme for accelerating its convergence rate to a stationary point. Experimental results on nine industrial datasets demonstrate that compared with state-of-the-art LFA models, an MH-based LFA model achieves gains in both accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with the algorithms, e.g., a second-order algorithm, which implicitly rely on gradients.

Quality-of-Service (QoS), which describes the non-functional characteristics of Web service, is of great significance in service selection. Since users cannot invoke all services to obtain the corresponding QoS data, QoS prediction becomes a hot yet thorny issue. To date, a latent factor analysis (LFA)-based QoS predictor is one of the most successful and popular approaches to address this issue. However, current LFA-based QoS predictors are mostly modeled on inner product space with anL2-norm-orientedLossfunction only. They cannot comprehensively represent the characteristics of target QoS data to make accurate predictions because inner product space andL2-norm have their respective limitations. To address this issue, this study proposes a Double-space and Double-norm Ensembled Latent Factor (D2E-LF) model. Its main idea is three-fold: 1) Double-space—inner product space and distance space are employed to model two kinds of LFA-based QoS predictors, respectively, 2) Double-norm—both of these two predictors adopt anL1-and-L2-norm-orientedLossfunction, and 3) Ensembled—building an ensemble of these two predictors by a weighting strategy. By doing so, D2E-LF integrates multi-merits originating from inner product space, distance space,L1-norm, andL2-norm, making it achieve highly accurate QoS prediction. Experiments on two real-world QoS datasets demonstrate that D2E-LF has significantly higher prediction accuracy than state-of-the-art models.

A High-dimensional and incomplete (HDI) matrix is regularly adopted to portray the inherent non-negativity of interactions among numerous nodes, which is involved in countless industrial applications driven by big data. An inherently non-negative latent factor (LF) model can take out the intrinsical features from such data conveniently and effectually due to its unimpeded training process. However, it constructs the learning objective relying on a standard Euclidean distance, thereby seriously restricting its representative ability to HDI data generated by different domains. To address this issue, this work proposes an adaptive divergence-based non-negative LF (ADNLF) model following: 1) constructing a generalized objective function based on$\alpha - \beta $-divergence to inflate its ability to represent various HDI data; 2) connecting the optimization variables with output LFs by a smooth and single LF-dependent bridging function to satisfy the non-negativity constraints constantly; and 3) facilitating adaptive divergence in the learning objective through particle swarm optimization for high scalability. Empirical studies on eight HDI matrices validate that an ADNLF model evidently outstrips state-of-the-art models in terms of estimation accuracy as well as computational efficiency for missing data of an HDI dataset.

Historical Quality-of-Service (QoS) data regarding past user-service invocations are vital to understand the user behaviors and cloud service conditions. A Matrix Factorization (MF)-based Collaborative Filtering (CF) model has proven to be highly effective in performing representation learning to such QoS data. However, its performance is hindered by its linear interaction and implicit encoding of collaborative QoS signal. To address this critical issue, this paper presents a Two-stream Light Graph Convolution Network-based latent factor (TLGCN) model with the three-fold ideas: 1) constructing a multilayered and fully-connected network to represent services’ nonlinear latent features; 2) integrating the user-service interactions, i.e., the bipartite graph structure into the representation learning process with a light graph convolution network for illustrating the high-order connectivity information in QoS data; and 3) incorporating the data density-oriented modeling mechanism into the input and output of TLGCN for high computational efficiency. Experimental results on two real QoS datasets demonstrate that the proposed TLGCN model significantly outperforms its state-of-the-art peers in both estimation accuracy for missing QoS data and computational efficiency.

A dynamically weighted directed network (DWDN) is frequently encountered in various big data-related applications like a terminal interaction pattern analysis system (TIPAS) concerned in this study. It consists of large-scale dynamic interactions among numerous nodes. As the involved nodes increase drastically, it becomes impossible to observe their full interactions at each time slot, making a resultant DWDN High Dimensional and Incomplete (HDI). An HDI DWDN, in spite of its incompleteness, contains rich knowledge regarding involved nodes various behavior patterns. To extract such knowledge from an HDI DWDN, this paper proposes a novel Alternating direction method of multipliers (ADMM)-based Nonnegative Latent-factorization of Tensors (ANLT) model. It adopts three-fold ideas: a) building a data density-oriented augmented Lagrangian function for efficiently handling an HDI tensors incompleteness and nonnegativity; b) splitting the optimization task in each iteration into an elaborately designed subtask series where each one is solved based on the previously solved ones following the ADMM principle to achieve fast convergence; and c) theoretically proving that its convergence is guaranteed with its efficient learning scheme. Experimental results on six DWDNs from real applications demonstrate that the proposed ANLT outperforms state-of-the-art models significantly in both computational efficiency and prediction accuracy.

This paper investigates a distributed competition behavior in multi-robot coordination under variable communication topology and switching one. In terms of multi-robot competition-based coordination, a winner-take-all (WTA) strategy is leveraged to address this issue with inevitable environmental barriers incorporated. Moreover, an innovative control theory stimulated gradient neural network (CTSGNN) algorithm is proposed to realize the WTA with prominent robustness and convergence over the traditional ones. Besides, to adapt to diversified local communication modes among multi-robot systems, fast variable and low switching topologies are constructed to establish two dynamic consensus estimators, accompanied by the proposed distributed control schemes. Traditional algorithms are introduced and served as a contrast. Afterward, the global convergence of the proposed algorithm in dealing with multi-robot competitive coordination, the universality of the application scenario, as well as the weaknesses of traditional methods are substantiated theoretically. The effectiveness and superiority of the proposed CTSGNN algorithm and the resultant distributed control schemes by integrating consensus estimators are further sustained via simulations. Note to Practitioners—The motivation of this paper is the coordination operation of multi-robot systems, but it is also applicable to other fields adopting multi-agent systems. Most of the existing researches on multi-robot coordination only exploit their collaborative behavior, which usually leads to the system redundancy and overflow of control costs. To this end, an innovative control algorithm is proposed for this competitive coordination and remains to be perfect in terms of stability and accuracy. Then, this paper establishes new distributed control schemes for multi-robot systems to compete for optimal dynamic task allocation. In this sense, under the premise of ensuring a successful task execution, only a few individuals with strong abilities and advantages are assigned. It is promising to maximize resource utilization, increase efficiency, and be extended to multi-objective scenarios. Note that the design of this scheme takes into account the environmental constraints and physical constraints of the robot itself. Theoretical analysis and preliminary simulation experiments prove the high efficiency of the control scheme. In ongoing research, the competitive coordination tasks of the mobile robot systems and communication delays or fault in control scheme design are explored to expand the operation scope and system extensibility.

A Stochastic Gradient Descent (SGD)-based Latent Factor Analysis (LFA) model is highly efficient in representative learning on a High-Dimensional and Sparse (HiDS) matrix, where the learning rate adaptation is vital in its efficiency and practicability. The learning rate adaptation of an SGD-based LFA model can be achieved efficiently by learning rate evolution with an evolutionary computing algorithm. However, a resultant model commonly suffers from twofold premature convergence issues, i.e., a) the premature convergence of the learning rate swarm relying on an evolution algorithm, and b) the premature convergence of an LFA model relying on the compound effects of evolution-based learning rate adaptation and adopted optimization algorithm. Aiming at addressed such issues, this work proposes an Hierarchical Particle swarm optimization-incorporated Latent factor analysis (HPL) model with a two-layered structure. The first layer pre-trains desired latent factors with a position-transitional particle swarm optimization-based LFA model with learning rate adaptation; while the second layer performs latent factor refinement with a newly-proposed mini-batch particle swarm optimization algorithm. Experimental results on four HiDS matrices generated by industrial applications demonstrate that an HPL model can well handle the mentioned premature convergence issues, thereby achieving highly-accurate representation to HiDS matrices.

Attributed graphs are widely used to represent network data where the attribute information of nodes is available. To address the problem of identifying clusters in attributed graphs, most of existing solutions are developed simply based on certain particular assumptions related to the characteristics of clusters of interest. However, it is yet unknown whether such assumed characteristics are consistent with attributed graphs. To overcome this issue, we innovatively introduce an inductive clustering algorithm that tends to address the clustering problem for attributed graphs without any assumption made on the clusters. To do so, we first process the attribute information to obtain pairwise attribute values that significantly frequently co-occur in adjacent nodes as we believe that they have potential ability to represent the characteristics of a given attributed graph. For two adjacent nodes, their likelihood of being grouped in the same cluster can be weighted by their ability to characterize the graph. Then based on these verifed characteristics instead of assumed ones, a depth-first search strategy is applied to perform the clustering task. Moreover, we are able to classify clusters such that their significance can be indicated. The experimental results demonstrate the performance and usefulness of our algorithm.

Latent factor analysis (LFA) via stochastic gradient descent (SGD) is highly efficient in discovering user and item patterns from high-dimensional and sparse (HiDS) matrices from recommender systems. However, most LFA-based recommender systems adopt a standard SGD algorithm, which suffers limited scalability when addressing big data. On the other hand, most existing parallel SGD solvers are either under the memory-sharing framework designed for a bare machine or suffering high communicational costs, which also greatly limits their applications in large-scale systems. To address the above issues, this article proposes a distributed alternative stochastic gradient descent (DASGD) solver for an LFA-based recommender. Its training-dependences among latent features are decoupled via alternatively fixing one-half of the features to learn the other half following the principle of SGD but in parallel. It's distribution mechanism consists of efficient data partition, allocation and task parallelization strategies, which greatly reduces its communicational cost for high scalability. Experimental results on three large-scale HiDS matrices generated by real-world applications demonstrate that the proposed DASGD algorithm outperforms state-of-the-art distributed SGD solvers for recommender systems in terms of prediction accuracy as well as scalability. Hence, it is highly useful for training LFA-based recommenders on large scale HiDS matrices with the help of cloud computing facilities.

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes amultilayered-and-randomizedlatentfactor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

Symmetric high-dimensional and sparse (SHiDS) networks are frequently found in various industrial applications. A symmetric non-negative latent factor (SNLF) model can acquire essential features from them precisely, yet it suffers from slow convergence. To address this issue, this article integrates a generalized momentum method into a symmetric, single latent factor-dependent, non-negative and multiplicative update (S2LF-NMU) algorithm, thereby achieving amomentum-incorporated,symmetric,single-latent-factor-dependentnon-negative-multiplicative-update (MS2N) algorithm. Based on an MS2N algorithm, momentum-incorporated symmetric non-negative latent factor (MSNLF) models are proposed for an SHiDS network, which ensures fast convergence as well as high representative learning ability. Empirical studies on four SHiDS networks from industrial applications demonstrate that compared with state-of-the-art models, the proposed MSNLF models have significantly higher computational efficiency and representative learning ability.

In this article, a robust k -winner-take-all ( k -WTA) neural network employing the saturation-allowed activation functions is designed and investigated to perform a k -WTA operation, and is shown to possess enhanced robustness to disturbance compared to existing k -WTA neural networks. Global convergence and robustness of the proposed k -WTA neural network are demonstrated through analysis and simulations. An application studied in detail is competitive multiagent coordination and dynamic task allocation, in which k active agents [among ] are allocated to execute a tracking task with the static m-k ones. This is implemented by adopting a distributed k -WTA network with limited communication, aided with a consensus filter. Simulation results demonstrating the system's efficacy and feasibility are presented.

To quantify user-item preferences, a recommender system (RS) commonly adopts a high-dimensional and sparse (HiDS) matrix. Such a matrix can be represented by a non-negative latent factor analysis model relying on a single latent factor (LF)-dependent, non-negative, and multiplicative update algorithm. However, existing models' representative abilities are limited due to their specialized learning objective. To address this issue, this study proposes an α- β -divergence-generalized model that enjoys fast convergence. Its ideas are three-fold: 1) generalizing its learning objective with α- β -divergence to achieve highly accurate representation of HiDS data; 2) incorporating a generalized momentum method into parameter learning for fast convergence; and 3) implementing self-adaptation of controllable hyperparameters for excellent practicability. Empirical studies on six HiDS matrices from real RSs demonstrate that compared with state-of-the-art LF models, the proposed one achieves significant accuracy and efficiency gain to estimate huge missing data in an HiDS matrix.

Complex networks have been widely adopted to represent a variety of complicated systems. Given a complex network, it is of great significance to perform accurate clustering for better understanding its intrinsic organization. To this end, a fuzzy-based clustering algorithm, i.e., FCAN, has been developed. Though effective, FCAN suffers from the disadvantage of slow convergence, which in return constrains its efficiency. To address this issue, this article proposes a fast fuzzy clustering algorithm, namely, F$^2$CAN, which incorporates a generalized momentum method into FCAN. Its fast convergence is rigorous justified in theory. Empirical studies on five datasets from real applications demonstrate that F$^2$CAN achieves a better performance when compared with FCAN and several state-of-the-art clustering algorithms in terms of convergence rate and clustering accuracy simultaneously. Hence, F$^2$CAN has potential for addressing the clustering analysis of large-scale complex networks emerging from industrial applications.

Fuzzy$c$-means (FCM) clustering is a promising method to handle uncertainties in data clustering. However, the traditional FCM and most of its variants cannot address incomplete inputs. To this aim, a novel fuzzy clustering framework is put forward to perform highly accurate clustering on incomplete data. It adopts twofold ideas: 1) Utilizing a nonnegative latent factor model to prefill the missing data in the inputs by rigidly extracting involved entities’ latent features, where the principle of a minibatch gradient descent algorithm is incorporated into a single latent factor-dependent, nonnegative and multiplicative update algorithm to accelerate the convergence rate; and 2) integrating the distribution of inputs and the weights of local features into the objective function through sparse self-representation and weighting allocation to focus on crucial features. In this way, a NLF analysis-incorporated and feature-weighted fuzzy double$c$-means clustering (NF$^2$D) method is achieved, where the data distribution and instance correlation are simultaneously considered with care. Experiments on 12 real-world datasets including both data and images with different missing rates show that the proposed NF$^2$D method has a significant superiority over state-of-the-art fuzzy clustering methods.

Recently, tensor Singular Value Decomposition (t-SVD)-based low-rank tensor completion (LRTC) has achieved unprecedented success in addressing various pattern analysis issues. However, existing studies mostly focus on third-order tensors while order- d ( d ≥ 4 ) tensors are commonly encountered in real-world applications, like fourth-order color videos, fourth-order hyper-spectral videos, fifth-order light-field images, and sixth-order bidirectional texture functions. Aiming at addressing this critical issue, this paper establishes an order- d tensor recovery framework including the model, algorithm and theories by innovatively developing a novel algebraic foundation for order- d t-SVD, thereby achieving exact completion for any order- d low t-SVD rank tensors with missing values with an overwhelming probability. Emperical studies on synthetic data and real-world visual data illustrate that compared with other state-of-the-art recovery frameworks, the proposed one achieves highly competitive performance in terms of both qualitative and quantitative metrics. In particular, as the observed data density becomes low, i.e., about 10%, the proposed recovery framework is still significantly better than its peers. The code of our algorithm is released at https://github.com/Qinwenjinswu/TIP-Code.

A mobile edge computing (MEC) paradgim is evolving as an increasingly popular means for developing and deploying smart-city-oriented applications. MEC servers can receive a great deal of requests from devices of mobile users, especially in crowded scenes, e.g., a city’s central business district and school areas. It thus remains a great challenge for appropriate scheduling and managing strategies to avoid hotspots, guarantee load-fairness among MEC servers, and maintain high resource utilization at the same time. To address this challenge, we propose a coalitional-game-based and location-aware approach to MEC service migration for mobile user reallocation in crowded scenes. Our proposed method includes: 1) dividing MEC servers into multiple coalitions according to their inter-Euclidean distance by using a modified$k$-means clustering method; 2) discovering hotspots in every coalition area and scheduling services based on their corresponding cooperations; and 3) migrating services to appropriate edge servers to achieve high utilization and load-fairness among coalition members. Experimental results based on a real-world mobile trajectory dataset for crowded scenes, and an urban-edge-server-position dataset demonstrate that our method outperforms existing ones in terms of load fairness, number of migrations, and utilization rate of edge servers.

High-dimensional and sparse (HiDS) matrices are frequently found in various industrial applications. A latent factor analysis (LFA) model is commonly adopted to extract useful knowledge from an HiDS matrix, whose parameter training mostly relies on a stochastic gradient descent (SGD) algorithm. However, an SGD-based LFA model's learning rate is hard to tune in real applications, making it vital to implement its self-adaptation. To address this critical issue, this study firstly investigates the evolution process of a particle swarm optimization algorithm with care, and then proposes to incorporate more dynamic information into it for avoiding accuracy loss caused by premature convergence without extra computation burden, thereby innovatively achieving a novel position-transitional particle swarm optimization (P2SO) algorithm. It is subsequently adopted to implement a P2SO-based LFA (PLFA) model that builds a learning rate swarm applied to the same group of LFs. Thus, a PLFA model implements highly efficient learning rate adaptation as well as represents an HiDS matrix precisely. Experimental results on four HiDS matrices emerging from real applications demonstrate that compared with an SGD-based LFA model, a PLFA model no longer suffers from a tedious and expensive tuning process of its learning rate, and it can achieve even higher prediction accuracy for missing data of an HiDS matrix. On the other hand, compared with state-of-the-art adaptive LFA models, a PLFA model's prediction accuracy and computational efficiency are highly competitive. Hence, it has high potential in addressing real industrial issues.

How to accurately predict unknown quality-of-service (QoS) data based on observed ones is a hot yet thorny issue in Web service-related applications. Recently, a latent factor (LF) model has shown its efficiency in addressing this issue owing to its high accuracy and scalability. An LF model can be improved by identifying user and service neighborhoods based on user and service geographical information. However, such information can be difficult to acquire in most applications with the considerations of information security, identity privacy, and commercial interests in a real system. Besides, the existing LF model-based QoS predictors mostly ignore the reliability of given QoS data where noises commonly exist to cause accuracy loss. To address the above issues, this paper proposes a data-characteristic-aware latent factor (DCALF) model to implement highly accurate QoS predictions, where ‘data-characteristic-aware’ indicates that it can appropriately implement QoS prediction according to the characteristics of given QoS data. Its main idea is two-fold: a) it detects the neighborhoods and noises of users and services based on the dense LFs extracted from the original sparse QoS data, b) it incorporates a density peaks-based clustering method into its modeling process for achieving the simultaneous detections of both neighborhoods and noises of QoS data. With such designs, it precisely represents the given QoS data in spite of their sparsity, thereby achieving highly accurate predictions for unknown ones. Experimental results on two QoS datasets generated by real-world Web services demonstrate that the proposed DCALF model outperforms state-of-the-art QoS predictors, making it highly competitive in addressing the issue of Web service selection and recommendation.

Community detection is a popular yet thorny issue in social network analysis. A symmetric and nonnegative matrix factorization (SNMF) model based on a nonnegative multiplicative update (NMU) scheme is frequently adopted to address it. Current research mainly focuses on integrating additional information into it without considering the effects of a learning scheme. This study aims to implement highly accurate community detectors via the connections between an SNMF-based community detector's detection accuracy and an NMU scheme's scaling factor. The main idea is to adjust such scaling factor via a linear or nonlinear strategy, thereby innovatively implementing several scaling-factor-adjusted NMU schemes. They are applied to SNMF and graph-regularized SNMF models to achieve four novel SNMF-based community detectors. Theoretical studies indicate that with the proposed schemes and proper hyperparameter settings, each model can: 1) keep its loss function nonincreasing during its training process and 2) converge to a stationary point. Empirical studies on eight social networks show that they achieve significant accuracy gain in community detection over the state-of-the-art community detectors.

Recent decades have witnessed a trend that control-theoretical techniques are widely leveraged in various areas, e.g., design and analysis of computational models. Computational methods can be modeled as a controller and searching the equilibrium point of a dynamical system is identical to solving an algebraic equation. Thus, absorbing mature technologies in control theory and integrating it with neural dynamics models can lead to new achievements. This work makes progress along this direction by applying control-theoretical techniques to construct new recurrent neural dynamics for manipulating a perturbed nonstationary quadratic program (QP) with time-varying parameters considered. Specifically, to break the limitations of existing continuous-time models in handling nonstationary problems, a discrete recurrent neural dynamics model is proposed to robustly deal with noise. This work shows how iterative computational methods for solving nonstationary QP can be revisited, designed, and analyzed in a control framework. A modified Newton iteration model and an improved gradient-based neural dynamics are established by referring to the superior structural technology of the presented recurrent neural dynamics, where the chief breakthrough is their excellent convergence and robustness over the traditional models. Numerical experiments are conducted to show the eminence of the proposed models in solving perturbed nonstationary QP.

Low-rank minimization aims to recover a matrix of minimum rank subject to linear system constraint. It can be found in various data analysis and machine learning areas, such as recommender systems, video denoising, and signal processing. Nuclear norm minimization is a dominating approach to handle it. However, such a method ignores the difference among singular values of target matrix. To address this issue, nonconvex low-rank regularizers have been widely used. Unfortunately, existing methods suffer from different drawbacks, such as inefficiency and inaccuracy. To alleviate such problems, this article proposes a flexible model with a novel nonconvex regularizer. Such a model not only promotes low rankness but also can be solved much faster and more accurate. With it, the original low-rank problem can be equivalently transformed into the resulting optimization problem under the rank restricted isometry property (rank-RIP) condition. Subsequently, Nesterov's rule and inexact proximal strategies are adopted to achieve a novel algorithm highly efficient in solving this problem at a convergence rate of O(1/K) , with K being the iterate count. Besides, the asymptotic convergence rate is also analyzed rigorously by adopting the Kurdyka- ojasiewicz (KL) inequality. Furthermore, we apply the proposed optimization model to typical low-rank problems, including matrix completion, robust principal component analysis (RPCA), and tensor completion. Exhaustively empirical studies regarding data analysis tasks, i.e., synthetic data analysis, image recovery, personalized recommendation, and background subtraction, indicate that the proposed model outperforms state-of-the-art models in both accuracy and efficiency.

A recommender system (RS) is highly efficient in filtering people’s desired information from high-dimensional and sparse (HiDS) data. To date, a latent factor (LF)-based approach becomes highly popular when implementing a RS. However, current LF models mostly adopt single distance-oriented Loss like an$L_{2}$norm-oriented one, which ignores target data’s characteristics described by other metrics like an$L_{1}$norm-oriented one. To investigate this issue, this article proposes an$L_{1}$-and-$L_{2}$-norm-oriented LF ($\text{L}^{3}\text{F}$) model. It adopts twofold ideas: 1) aggregating$L_{1}$norm’s robustness and$L_{2}$norm’s stability to form its Loss and 2) adaptively adjusting weights of$L_{1}$and$L_{2}$norms in its Loss. By doing so, it achieves fine aggregation effects with$L_{1}$norm-oriented Loss ’s robustness and$L_{2}$norm-oriented Loss ’s stability to precisely describe HiDS data with outliers. Experimental results on nine HiDS datasets generated by real systems show that an$\text{L}^{3}\text{F}$model significantly outperforms state-of-the-art models in prediction accuracy for missing data of an HiDS dataset. Its computational efficiency is also comparable with the most efficient LF models. Hence, it has good potential for addressing HiDS data from real applications.

For the existing repetitive motion generation (RMG) schemes for kinematic control of redundant manipulators, the position error always exists and fluctuates. This article gives an answer to this phenomenon and presents the theoretical analyses to reveal that the existing RMG schemes exist a theoretical position error related to the joint angle error. To remedy this weakness of existing solutions, an orthogonal projection RMG (OPRMG) scheme is proposed in this article by introducing an orthogonal projection method with the position error eliminated theoretically, which decouples the joint space error and Cartesian space error with joint constraints considered. The corresponding new recurrent neural networks (NRNNs) are structured by exploiting the gradient descent method with the assistance of velocity compensation with theoretical analyses provided to embody the stability and feasibility. In addition, simulation results on a fixed-based redundant manipulator, a mobile manipulator, and a multirobot system synthesized by the existing RMG schemes and the proposed one are presented to verify the superiority and precise performance of the OPRMG scheme for kinematic control of redundant manipulators. Moreover, via adjusting the coefficient, simulations on the position error and joint drift of the redundant manipulator are conducted for comparison to prove the high performance of the OPRMG scheme. To bring out the crucial point, different controllers for the redundancy resolution of redundant manipulators are compared to highlight the superiority and advantage of the proposed NRNN. This work greatly improves the existing RMG solutions in theoretically eliminating the position error and joint drift, which is of significant contributions to increasing the accuracy and efficiency of high-precision instruments in manufacturing production.

A non-negative latent factor (NLF) model with a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is frequently adopted to extract useful knowledge from non-negative data represented by high-dimensional and sparse (HiDS) matrices arising from various service-oriented applications. However, its convergence rate is slow. To address this issue, this study proposes aGeneralized Nesterov's acceleration-incorporated,Non-negative andAdaptiveLatentFactor (GNALF) model. It results from a) incorporating a generalized Nesterov's accelerated gradient (NAG) method into an SLF-NMU algorithm, thereby achieving anNAG-incorporated andelement-orientednon-negative (NEN) algorithm to perform efficient parameter update; and b) making its regularization and acceleration parameters self-adaptive via incorporating the principle of a particle swarm optimization algorithm into the training process, thereby implementing a highly adaptive and practical model. Empirical studies on six large sparse matrices from different recommendation service applications show that a GNALF model achieves very high convergence rate without the need of hyper-parameter tuning, making its computational efficiency significantly higher than state-of-the-art models. Meanwhile, such efficiency gain does not result in accuracy loss, since its prediction accuracy is comparable with its peers. Hence, it can better serve practical service applications with real-time demands.

Neighborhood regularization is highly important for a latent factor (LF)-based Quality-of-Service (QoS)-predictor since similar users usually experience similar QoS when invoking similar services. Current neighborhood-regularized LF models rely prior information on neighborhood obtained from common raw QoS data or geographical information. The former suffers from low prediction accuracy due to the difficulty of constructing the neighborhood based on incomplete QoS data, while the latter requires additional geographical information that is usually difficult to collect considering information security, identity privacy, and commercial interests in real-world scenarios. To address the above issues, this work proposes a posterior-neighborhood-regularized LF (PLF) model for QoS prediction. The main idea is to decompose the LF analysis process into three phases: a) primal LF extraction, where the LFs are extracted to represent involved users/services based on known QoS data, b) posterior-neighborhood construction, where the neighborhood of each user/service is achieved based on similarities between their primal LF vectors, and c) posterior-neighborhood-regularized LF analysis, where the objective function is regularized by both the posterior-neighborhood of users/services and$L_{2}$L2-norm of desired LFs. Experimental results from large scale QoS datasets demonstrate that PLF outperforms state-of-the-art models in terms of both accuracy and efficiency.

Dynamic relationships are frequently encountered in big data and services computing-related applications, like dynamic data of user-side QoS in Web services. They are modeled into a high-dimensional and sparse (HiDS) tensor, which contain rich knowledge regarding temporal patterns. A non-negative latent factorization of tensors (NLFT) model is very effective in extracting such patterns from an HiDS tensor. However, it commonly suffers from overfitting with improper regularization schemes. To address this issue, this article investigates NLFT models with diversified regularization schemes. Six regularized NLFT models, i.e.,$L_{2}, L_{1}$, elastic net, log, dropout, and swish-regularized ones, are proposed and carefully investigated. Moreover, owing to their diversified regularization designs, they possess strong model diversity to achieve an effective ensemble. Empirical studies on HiDS QoS tensors from real applications demonstrate that compared with state-of-the-art models, the proposed ones better describe the temporal patterns hidden in an HiDS tensor, thereby achieving significantly higher prediction accuracy for missing data. Moreover, their ensemble further outperforms each of them in terms of prediction accuracy for missing QoS data.

Spectral clustering is becoming more and more popular because it has good performance in discovering clusters with varying characteristics. However, it suffers from high computational cost, unstable clustering results and noises. This work presents a novel approximate spectral clustering based on dense cores and density peaks, called DCDP-ASC. It first finds a reduced data set by introducing the concept of dense cores; then defines a new distance based on the common neighborhood of dense cores and calculates geodesic distances between dense cores according to the new defined distance; after that constructs a decision graph with a parameter-free local density and geodesic distance for obtaining initial centers; finally calculates the similarity between dense cores with their new defined geodesic distance, employs normalized spectral clustering method to divide dense cores, and expands the result on dense cores to the whole data set by assigning each point to its representative. The results on some challenging data sets and the comparison of our algorithm with some other excellent methods demonstrate that the proposed method DCDP-ASC is more advantageous in identifying complex structured clusters containing a lot of noises.

A non-negative latent factor analysis (NLFA)-based recommender can make precise recommendations by correctly representing the non-negative characteristic of industrial data. It commonly relies on a nonconvex and bilinear optimization process, where the effects of first-order solvers maybe significantly reduced. Higher order solvers like a Newton-type method are expected to make a breakthrough; however, its computation efficiency and scalability are greatly limited due to the numerous parameters involved in a Hessian matrix. To address this issue, this article proposes an approach for assimilating second-order information for building NLFA-based recommenders. The key idea is an inner second-order solver that employs a Hessian-free method for avoiding the highly expensive manipulations of a Hessian matrix. Empirical studies on eight data cases emerging from real industrial applications indicate that the proposed approach outperforms state-of-the-art models in prediction accuracy with affordable computational burden.

Up to date, the existing methods for nonlinear optimization with time-dependent parameters can be classified into two types: 1) static methods are capable of handling inequality constraints but may generate large lagging errors in the solution of the intrinsically time-dependent constrained nonlinear optimization (TDCNO) problem due to the hypothesis of short-time invariance and 2) time-variant methods, e.g., zeroing neural networks, are able to remedy the lagging error but fail to solve the TDCNO problem under inequality constraints. To resolve this contradiction, a noise-suppressing neural dynamics (NSND) model is proposed to solve the TDCNO problem subject to both equality and inequality constraints via the nonlinear complementary problem (NCP) function. The proposed method allows inequality constraints for unknown variables, removes the short-time invariance hypothesis, and further eliminates lagging errors during the solving process in the presence of noises. Besides, the rapid convergence, global stability, and noise processing of the NSND model are verified by the theoretical analyses. Simulation results of illustrative examples, including dimensionality reduction on principal component analyses (PCA) and a robot motion control, show that the NSND model outperforms the existing models for the TDCNO problem.

Online streaming feature selection (OSFS) has attracted extensive attention during the past decades. Current approaches commonly assume that the feature space of fixed data instances dynamically increases without any missing data. However, this assumption does not always hold in many real applications. Motivated by this observation, this study aims to implement online feature selection from sparse streaming features, i.e., features flow in one by one with missing data as instance count remains fixed. To do so, this study proposes a latent-factor-analysis-based online sparse-streaming-feature selection algorithm (LOSSA). Its main idea is to apply latent factor analysis to pre-estimate missing data in sparse streaming features before conducting feature selection, thereby addressing the missing data issue effectively and efficiently. Theoretical and empirical studies indicate that LOSSA can significantly improve the quality of OSFS when missing data are encountered in target instances.

It is generally considered that controlling a robot precisely becomes tough on the condition of unknown structure information. Applying a data-driven approach to the robot control with the unknown structure implies a novel feasible research direction. Therefore, in this article, as a combination of the structural learning and robot control, an acceleration-level data-driven repetitive motion planning (DDRMP) scheme is proposed with the corresponding recurrent neural network (RNN) constructed. Then, theoretical analyses on the learning and control abilities are provided. Moreover, simulative experiments on employing the acceleration-level DDRMP scheme as well as the corresponding RNN to control a Sawyer robot and a Baxter robot with unknown structure information are performed. Accordingly, simulation results validate the feasibility of the proposed method and comparisons among the existing repetitive motion planning (RMP) schemes indicate the superiority of the proposed method. This work offers sufficient theoretical and simulative solutions for the acceleration-level redundancy problem of redundant robots with unknown structure and joint limits considered.

User-side Quality-of-Service (QoS) data are vital for efficient cloud service selection, while available QoS data in real applications are commonly described by a high-dimensional and Sparse (HiDS) matrix due to a) increasing users and services, and b) the impossibility of observing the full invoking mapping among users and services. A latent factor analysis (LFA) model has proven to be efficient in performing representation learning on such an HiDS QoS matrix. However, existing LFA models commonly adopt a first-order optimizer that cannot makes it approach the second-order stationary point of the learning objective, thereby resulting in accuracy loss. Aiming at addressing this issue, this study proposes to incorporate a Generalized Nesterov's Acceleration (GNA) method in to a Hessian-vector algorithm for LFA, thereby establishing a GNA-incorporated Hessian-vector-based LFA (GNHL) model with two-fold ideas: a) adopting the principle of a Hessian-vector method to acquire a proper Newton step efficiently, and b) incorporating a GNA method into its linear search for accelerating its convergence rate. Experimental results on six real QoS datasets demonstrate that a GNHL model outperforms state-of-the-art LFA models in generating highly accurate predictions for missing QoS data with low computational burden.

Temporally dynamic QoS data are commonly encountered in large-scale cloud services environments. They can be quantized into a high-dimensional and incomplete (HDI) tensor defined on the user×service×time. Despite its HDI nature, it contains various temporal patterns highly helpful in representing involved users and services. A latent factorization of tensors (LFT) model is able to discover such patterns from an HDI tensor, while its model generality cannot be ensured due to the complex structure and incomplete data of an HDI tensor. To address this issues, this paper proposes an Instance-frequency-weighted regularized, Nonnegative and Adaptive LFT (INAL) model with three-fold ideas: a) adopting the principle of data density-oriented modeling to reduce the computation and storage complexity; b) refining the regularization effects on each latent factor with its relevant instance-frequency for illustrating the imbalanced distribution of known data in an HDI tensor; and c) making its hyper-parameter self-adaptive via incorporating the principle of a particle swarm optimization (PSO) algorithm into the training process, thereby achieving a highly adaptive and practical model. Empirical studies on two dynamic QoS datasets from real applications demonstrate that compared with state-of-the-art models, the proposed model achieves significant gain in prediction accuracy for unobserved dynamic QoS data and achieves highly competitive computational efficiency.

Big-data related applications frequently concern how to analyze large-scale undirected weighted network effectively. Such a network can be quantized into a Symmetric, High-Dimensional and Sparse (SHiDS) matrix, owing to its sparsity and symmetry. For considering these characteristics of an SHiDS matrix with care, a symmetric non-negative latent factor (SNLF) model is proposed. However, representation learning ability of an SNLF model is limited owing to its commonly-adopted learning objective, i.e., Euclidean distance. For addressing this issue, this study proposes a generalized symmetric nonnegative latent factor analysis (GSNL) model. Its main idea is two-fold: a) adopting$\alpha-\beta$-divergence to generalize SNLF's learning objective for achieving accurate representation ability of an SHiDS matrix; and b) utilizing a self-adaptive scheme on all involved hyperparameters brought by the resultant model for strong practicability. Empirical studies on four SHiDS matrices demonstrate that a GSNL model outperforms its peers regarding accuracy gain and computational efficiency.

A biased non-negative latent factorization of tensors model can perform accurate representation learning to dynamic quality-of-service (QoS) data. It relies on a non-negative and multiplicative update on incomplete tensors learning algorithm. However, this algorithm suffers from slow convergence when the target data increases to perplex such model’s learning objective. To address such challenging issue, this study incorporates a generalized Nesterov’s accelerated gradient method into the mentioned learning algorithm, thereby achieving a new algorithm. Based on this algorithm, a new model is successfully constructed. Empirical studies on two industrial datasets indicate that this new model is superior to the state-of-the-art models in both computational efficiency and prediction accuracy for missing QoS data after performing presentation learning on dynamic QoS data. Hence, this work represents an important progress in the field of fast and accurate latent factorization of tensors.

A stochastic-gradient-descent-based Latent Factor Analysis (LFA) model is highly efficient in representative learning of a High-Dimensional and Sparse (HiDS) matrix. Its learning rate adaptation is vital in ensuring its efficiency. Such adaptation can be realized with an evolutionary computing algorithm. However, a resultant model tends to suffer from two issues: a) the pre-mature convergence of the swarm of learning rates as caused by an adopted evolution algorithm, and b) the pre-mature convergence of the LFA model as caused jointly by evolution-based learning rate adaptation and an optimization algorithm. This paper focuses on the methods to address such issues. A Hierarchical Particle-swarm-optimization-incorporated Latent factor analysis (HPL) model with a two-layered structure is proposed, where the first layer pre-trains desired latent factors with a position-transitional particle-swarm-optimization-based LFA model, and the second layer performs latent factor refining with a newly-proposed mini-batch particle swarm optimizer. With such design, an HPL model can well handle the pre-mature convergence, which is supported by the positive experimental results achieved on HiDS matrices from industrial applications.

Protein-protein interactions are of great significance for us to understand the functional mechanisms of proteins. With the rapid development of high-throughput genomic technology, the amount of protein-protein interaction data has become so big that most of existing prediction algorithms are no longer applicable. To address this problem, we develop a distributed framework by reimplementing one of state-of-the-art algorithms, i.e., CoFex, by using MapReduce. In particular, we adopt a novel tree-based data structure to reduce the heavy memory consumption cased by the huge sequence information of proteins. After that, the procedure of CoFex is modified by following the paradigm of MapReduce such that the prediction task can be completed in a distributed manner, thus fulfilling the demanding requirements of large-scale protein-protein interaction prediction. A series of experiments have been conducted to evaluate the performance of the proposed distributed framework in terms of both efficiency and effectiveness. Experimental results demonstrate that the proposed framework can considerably improve the efficiency of CoFex by achieving more than two-orders-of-magnitude improvement in computational efficiency while retaining a comparable level of accuracy.

Large-scale undirected networks representation can be formulated into a non-convex optimization problem, which can be efficiently addressed by a symmetric non-negative latent factor analysis (SNLFA)-based approach. However, an SNLFA model commonly adopts a first-order optimization algorithm that cannot well handle its non-convex learning objective, thereby resulting in inaccurate representation to a target network. On the other hand, higher-order learning algorithms like a Newton-type one is expected to solve a non-convex optimization problem well, but its computation efficiency and scalability are greatly limited due to its direct manipulation of the Hessian matrix that can be huge in an SNLFA model. To address this issue, this paper proposes a Truncated Newton-method-based Symmetric Non-negative Latent Factor Analysis (TNS) model for utilizing second-order information in the latent factor analysis process without manipulating the Hessian matrix, thereby achieving high computational efficiency and scalability. Empirical studies indicate that TNS outperforms state-of-the-art models in prediction accuracy with an affordable computational burden.

A big-data-related application like a Terminal Interaction Pattern Analysis System (TIPAS) commonly involves massive entities interacting with each other dynamically. Such interactions can be represented by a Dynamically Weighted Directed Network (DWDN). A large number of involved entities results in a high-dimensional and incomplete (HDI) network, which can be represented by an HDI tensor with numerous missing entries. In spite of its HDI nature, this tensor contains much useful knowledge regarding various desired patterns like unobserved links in DWDN. However, due to its extremely high dimensionality and low data density, it is very challenging to build a learning model that can precisely represent an HDI tensor. To address this issue, this work proposes a Neural Latent Factorization of Tensors (NeuLFoT) model with three interesting ideas: a) adopting the principle of density-oriented modeling and Canonical Polyadic tensor factorization to build rank-one tensor series relying on three-dimensional latent factors for precisely representing an HDI tensor’s known data; b) treating the obtained rank-one tensors as neurons to form a novel neural tensor network model; and c) proposing a novel Backward Propagation algorithm for Latent factorization of tensors (BPL) to ensure high training efficiency. Experimental results on two large-scale DWDNs generated by a real TIPAS demonstrate that compared with state-of-the-art models, the proposed model achieves significant gain in prediction accuracy for missing links of a DWDN and achieves highly competitive computational efficiency.

A large-scale undirected weighted network (LUWN) is described by a symmetric high-dimensional and sparse (SHiDS) matrix. To correctly represent its symmetry, existing models adopt a strong symmetry assumption, i.e., reducing the solution space to adopt a unique latent factor matrix to describe an SHiDS matrix’s symmetry. Yet it may impair a resultant model’s representativeness to its numerical features. Aiming at addressing this issue, this work proposes a Relaxed Symmetric Non-negative latent factor analysis (RSN) model that adopts three-fold ideas: a) Introducing a triple equation constraint into its learning objective for relaxing the strong symmetry assumption, thereby greatly improving its representativeness to the numerical features of an SHiDS matrix; b) Adopting the framework of Alternating Direction Method of Multipliers to fast realize its learning objective subject to multiple constraints; and c) utilizing a data density-oriented principle during its modeling and optimization, thereby precisely representing an SHiDS matrix’s imbalanced data. Empirical studies on four industrial SHiDS matrices describing real LUWNs demonstrate that RSN outperforms state-of-the-art models in representing an SHiDS matrix precisely, as well as achieves highly competitive computational efficiency. Hence, this work greatly advances the area of LUWN analysis.

A nonnegative latent factorization of tensors (NLFT) model precisely represents the temporal patterns hidden in multichannel data emerging from various applications. It often adopts a single latent factor-dependent, nonnegative and multiplicative update on tensor (SLF-NMUT) algorithm. However, learning depth in this algorithm is not adjustable, resulting in frequent training fluctuation or poor model convergence caused by overshooting. To address this issue, this study carefully investigates the connections between the performance of an NLFT model and its learning depth via SLF-NMUT to present a joint learning-depth-adjusting scheme for it. Based on this scheme, a Depth-adjusted Multiplicative Update on tensor algorithm is innovatively proposed, thereby achieving a novel depth-adjusted nonnegative latent-factorization-of-tensors (DNL) model. Empirical studies on two industrial data sets demonstrate that compared with the state-of-the-art NLFT models, a DNL model achieves significant accuracy gain when performing missing data estimation on a high-dimensional and incomplete tensor with high efficiency.Note to Practitioners—Multichannel data are often encountered in various big-data-related applications. It is vital for a data analyzer to correctly capture the temporal patterns hidden in them for efficient knowledge acquisition and representation. This article focuses on analyzing temporal QoS data, which is a representative kind of multichannel data. To correctly extract their temporal patterns, an analyzer should correctly describe their nonnegativity. Such a purpose can be achieved by building a nonnegative latent factorization of tensors (NLFT) model relying on a single latent factor-dependent, nonnegative and multiplicative update on tensor (SLF-NMUT) algorithm. But its learning depth is not adjustable, making an NLFT model frequently suffer from severe fluctuations in its training error or even fail to converge. To address this issue, this study carefully investigates the learning rules for an NLFT model’s decision parameters using an SLF-NMUT and proposes a joint learning-depth-adjusting scheme. This scheme manipulates the multiplicative terms in SLF-NMUT-based learning rules linearly and exponentially, thereby making the learning depth adjustable. Based on it, this study builds a novel depth-adjusted nonnegative latent-factorization-of-tensors (DNL) model. Compared with the existing NLFT models, a DNL model better represents multichannel data. It meets industrial needs well and can be used to achieve high performance in data analysis tasks like temporal-aware missing data estimation

An opportunistic link between two mobile devices or nodes can be constructed when they are within each other’s communication range. Typically, cyber–physical environments consist of a number of mobile devices that are potentially able to establish opportunistic contacts and serve mobile applications in a cost-effective way. Opportunistic mobile service computing is a promising paradigm capable of utilizing the pervasive mobile computational resources around the users. Mobile users are thus allowed to exploit nearby mobile services to boost their computing capabilities without investment in their resource pool. Nevertheless, various challenges, especially its quality-of-service and reliability-aware scheduling, are yet to be addressed. Existing studies and related scheduling strategies consider mobile users to be fully stable and available. In this article, we propose a novel method for reliability-aware and deadline-constrained service composition over opportunistic networks. We leverage the Krill–Herd-based algorithm to yield a deadline-constrained, reliability-aware, and well-executable service composition schedule based on the estimation of completion time and reliability of schedule candidates. We carry out extensive case studies based on some well-known mobile service composition templates and a real-world opportunistic contact data set. The comparison results suggest that the proposed approach outperforms existing ones in terms of success rate and completion time of composed services.Note to Practitioners—Recently, the rapid development of mobile devices and mobile communication leads to the prosperity of mobile service computing. Services running on mobile devices within a limited range are allowed to be composed to coordinate through wireless communication technologies and perform complex tasks and business processes. Despite its great potential, mobile service compositions remains a challenge since the mobility of users and devices imposes high unpredictability on the execution of tasks. A careful investigation into existing methods has found their various limitations, e.g., assuming time-invariant availability of mobile services. This article presents a novel reliability-aware and deadline-constrained service composition method for mobile opportunistic networks. Instead of assuming time-invariant availability of mobile nodes, the proposed method is capable of estimating service availability at run-time and leveraging a Krill–Herd-based algorithm to yield the deadline-constrained, reliability-aware, and well-executable service composition schedules. Case studies based on well-known service composition templates and real-world data sets suggest that it outperforms traditional ones in terms of success and completion time of composed services. It can thus aid the design and optimization of composite services as well as their smooth execution in a mobile environment. It can help practitioners better manage the reliability and performance of real-world applications built upon mobile services.

Non-negativity is vital for a latent factor (LF)-based model to preserve the important feature of a high-dimensional and sparse (HiDS) matrix in recommender systems, i.e., none of its entries is negative. Current non-negative models rely on constraints-combined training schemes. However, they lack flexibility, scalability, or compatibility with general training schemes. This work aims to perform unconstrained non-negative latent factor analysis (UNLFA) on HiDS matrices. To do so, we innovatively transfer the non-negativity constraints from the decision parameters to the output LFs, and connect them through a single-element-dependent mapping function. Then we theoretically prove that by making a mapping function fulfill specific conditions, the resultant model is able to represent the original one precisely. We subsequently design highly efficient UNLFA algorithms for recommender systems. Experimental results on four industrial-size HiDS matrices demonstrate that compared with four state-of-the-art non-negative models, a UNLFA-based model obtains advantage in prediction accuracy for missing data and computational efficiency. Moreover, such high performance is achieved through its unconstrained training process which is compatible with various general training schemes, on the premise of fulfilling non-negativity constraints. Hence, UNLFA algorithms are highly valuable for industrial applications with the need of performing non-negative latent factor analysis on HiDS matrices.

A single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) learning algorithm is highly efficient in building a nonnegative LF (NLF) model defined on a high-dimensional and sparse (HiDS) matrix. However, convergence characteristics of such NLF models are never justified in theory. To address this issue, this study conducts rigorous convergence analysis for an SLF-NMU-based NLF model. The main idea is twofold: 1) proving that its learning objective keeps nonincreasing with its SLF-NMU-based learning rules via constructing specific auxiliary functions; and 2) proving that it converges to a stable equilibrium point with its SLF-NMU-based learning rules via analyzing the Karush-Kuhn-Tucker (KKT) conditions of its learning objective. Experimental results on ten HiDS matrices from real applications provide numerical evidence that indicates the correctness of the achieved proof.

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. A momentum method is frequently adopted to accelerate a learning algorithm, but it is incompatible with those implicitly adopting gradients like SLF-NMU. To build a fast NLF (FNLF) model, we propose a generalized momentum method compatible with SLF-NMU. With it, we further propose a single latent factor-dependent non-negative, multiplicative and momentum-incorporated update algorithm, thereby achieving an FNLF model. Empirical studies on six HiDS matrices from industrial application indicate that an FNLF model outperforms an NLF model in terms of both convergence rate and prediction accuracy for missing data. Hence, compared with an NLF model, an FNLF model is more practical in industrial applications.

High-dimensional and sparse (HiDS) data with non-negativity constraints are commonly seen in industrial applications, such as recommender systems. They can be modeled into an HiDS matrix, from which non-negative latent factor analysis (NLFA) is highly effective in extracting useful features. Preforming NLFA on an HiDS matrix is ill-posed, desiring an effective regularization scheme for avoiding overfitting. Current models mostly adopt a standard L2scheme, which does not consider the imbalanced distribution of known data in an HiDS matrix. From this point of view, this paper proposes an instance-frequency-weighted regularization (IR) scheme for NLFA on HiDS data. It specifies the regularization effects on each latent factors with its relevant instance count, i.e., instance-frequency, which clearly describes the known data distribution of an HiDS matrix. By doing so, it achieves finely grained modeling of regularization effects. The experimental results on HiDS matrices from industrial applications demonstrate that compared with an L2scheme, an IR scheme enables a resultant model to achieve higher accuracy in missing data estimation of an HiDS matrix.

High-dimensional and sparse (HiDS) matrices generated by recommender systems contain rich knowledge regarding various desired patterns like users' potential preferences and community tendency. Latent factor (LF) analysis proves to be highly efficient in extracting such knowledge from an HiDS matrix efficiently. Stochastic gradient descent (SGD) is a highly efficient algorithm for building an LF model. However, current LF models mostly adopt a standard SGD algorithm. Can SGD be extended from various aspects in order to improve the resultant models' convergence rate and prediction accuracy for missing data? Are such SGD extensions compatible with an LF model? To answer them, this paper carefully investigates eight extended SGD algorithms to propose eight novel LF models. Experimental results on two HiDS matrices generated by real recommender systems show that compared with an LF model with a standard SGD algorithm, an LF model with extended ones can achieve: 1) higher prediction accuracy for missing data; 2) faster convergence rate; and 3) model diversity.

Non-negative latent factor (NLF) models well represent high-dimensional and sparse (HiDS) matrices filled with non-negative data, which are frequently encountered in industrial applications like recommender systems. However, current NLF models mostly adopt Euclidean distance in their objective function, which represents a special case of a β-divergence function. Hence, it is highly desired to design a β-divergence-based NLF ( β-NLF) model that uses a β-divergence function, and investigate its performance in recommender systems as β varies. To do so, we first model β-NLF's learning objective with a β-divergence function. Subsequently, we deduce a general single latent factor-dependent, non-negative and multiplicative update scheme for β-NLF, and then design an efficient β-NLF algorithm. The experimental results on HiDS matrices from industrial applications indicate that by carefully choosing the value of β, β-NLF outperforms an NLF model with Euclidean distance in terms of accuracy for missing data prediction without increasing computational time. The research outcomes show the necessity of using an optimal β-divergence function in order to achieve the best performance of an NLF model on HiDS matrices. Hence, the proposed model has both theoretical and application significance.

Recommender systems (RSs) commonly adopt a user-item rating matrix to describe users' preferences on items. With users and items exploding, such a matrix is usually high-dimensional and sparse (HiDS). Recently, the idea of deep learning has been applied to RSs. However, current deep-structured RSs suffer from high computational complexity. Enlightened by the idea of deep forest, this paper proposes a deep latent factor model (DLFM) for building a deep-structured RS on an HiDS matrix efficiently. Its main idea is to construct a deep-structured model by sequentially connecting multiple latent factor (LF) models instead of multilayered neural networks through a nonlinear activation function. Thus, the computational complexity grows linearly with its layer count, which is easy to resolve in practice. The experimental results on four HiDS matrices from industrial RSs demonstrate that when compared with state-of-the-art LF models and deep-structured RSs, DLFM can well balance the prediction accuracy and computational efficiency, which well fits the desire of industrial RSs for fast and right recommendations.

With the rapid development of services computing in the past decade, automatic selection of QoS-aware Web service out from numerous candidates with similar functions is becoming a hot yet thorny issue. Conducting warming up tests on numerous candidate services for quality evaluation is extremely time-consuming and expensive, making it vital to generate highly accurate predictions for missing QoS data based on known ones. Since QoS data are time-dependent, it is vital to consider the temporal dynamic patterns hidden in historical ones when building a QoS-estimator. For addressing this issue, this study invents a Kalman filter-incorporated Latent Factor Analysis (KLFA)-based QoS-estimator, which precisely models the temporal patterns hidden in dynamic QoS data. Experimental results based on large-scale and real-world Web service QoS data demonstrate that compared with state-of-the-art temporal-aware QoS-estimators, a KLFA-based one achieves significantly higher prediction accuracy for missing QoS data.

Precisely understanding high-dimensional and sparse (HiDS) user-item interactions is the most important issue in a recommender system. A latent factor analysis (LFA)-based model has proven to be efficient in addressing it, while current models of this kind mostly rely on first-order optimizers. It is vital to implement an LFA-based model able to approach the second order stationary points efficiently for improving its representative learning ability. To do so, this work presents a Generalized-momentum-accelerated Hessian-vector Algorithm (GHA) for HiDS data. Its main idea includes a) adopting the principle of a Hessian-vector-product-based method to avoid operating a Hessian matrix directly, and b) incorporating a generalized momentum method into its parameter learning process for further enhancing its ability in approaching a stationary point. Experimental results on two industrial datasets demonstrate that when compared with state-of-the-art LFA-based models, a GHA-based LFA model achieves gains in accuracy and convergence rate. These positive outcomes also indicate that a generalized momentum method is compatible with algorithms implicitly relying on gradients like a second-order algorithm.

A latent factor (LF) model can implement efficient analysis for a high-dimensional and sparse (HiDS) matrix from recommender systems (RSs). However, an LF model's representation learning ability to a targeted HiDS matrix is heavily proportional to its known data density. Unfortunately, an HiDS matrix's known data are limited due to users' activity limitations in RSs. Motivated by this observation, this paper proposes a Prediction-sampling-based Multilayer-structured Latent Factor (PMLF) model. Following the principle of Deep Forest [1], PMLF implements a loosely-connected multilayered LF structure, where each layer generates synthetic ratings to enrich the input for the next layer. Such an injection process is carefully monitored through a random sampling process and nonlinear activations to avoid overfitting. Thus, PMLF's representation learning ability to an HiDS matrix is significantly enhanced owing to the carefully injected estimates and its generalized multilayer-structure. Experimental results on four HiDS matrices from industrial RSs indicate that compared with six state-of-the-art LF-based and deep neural networks-based models, PMLF well balances the prediction accuracy and computational efficiency, making it satisfy demands of fast and accurate industrial applications.

Edge computing is a promising paradigm that can boost the performance of novel mobile applications and energize the real-time governance of Internet-of-Things (IoT) big data. In edge computing, mobile application vendors are allowed to employ edge resources to speed up end-users' applications in an elastic and on-demand manner. However, due to the complex geographical distribution of edge servers and users, how to decide the most appropriate destination edge server to hire and how to decide the corresponding user-server allocation plan with as-low-as-possible monetary cost are the key problems for application vendors. Instead of assuming a simultaneous-batch-arrival pattern of incoming users and considering static optimization of the Edge User Allocation (EUA) problem by most existing studies, in this paper, we consider an online EUA problem where users' arrival and departure follow a general pattern. We take the long-term edge user allocation rate and edge server leasing cost as scheduling targets and propose a decentralized collaborative and fuzzy-control-based approach to yielding real-time user-edge-server allocation schedules. In this approach, edge users are allowed to independently make their own allocation decision only based on local information (i.e., the status of nearby edge servers). Experiments on real-world edge datasets demonstrate our approach outperforms state-of-the-art approaches in terms of long-term allocation rate and system cost.

Recommender systems (RSs) commonly describe its user-item preferences with a high-dimensional and sparse (HiDS) matrix filled with non-negative data. A non-negative latent factor (NLF) model relying on a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is frequently adopted to process such an HiDS matrix. However, an NLF model mostly adopts Euclidean distance for its objective function, which is naturally a special case of α-β-divergence. Moreover, it frequently suffers slow convergence. For addressing these issues, this study proposes a generalized and fast-converging non-negative latent factor (GFNLF) model. Its main idea is two-fold: a) adopting α-β-divergence for its objective function, thereby enhancing its representation ability for HiDS data; b) deducing its momentum-incorporated non-negative multiplicative update (MNMU) algorithm, thereby achieving its fast convergence. Empirical studies on two HiDS matrices emerging from real RSs demonstrate that with carefully-tuned hyperparameters, a GFNLF model outperforms state-of-the-art models in both computational efficiency and prediction accuracy for missing data of an HiDS matrix.

Human immunodeficiency virus 1 (HIV-1) protease (PR) plays a crucial role in the maturation of the virus. The study of substrate specificity of HIV-1 PR as a new endeavor strives to increase our ability to understand how HIV-1 PR recognizes its various cleavage sites. To predict HIV-1 PR cleavage sites, most of the existing approaches have been developed solely based on the homogeneity of substrate sequence information with supervised classification techniques. Although efficient, these approaches are found to be restricted to the ability of explaining their results and probably provide few insights into the mechanisms by which HIV-1 PR cleaves the substrates in a site-specific manner. In this work, a coevolutionary pattern-based prediction model for HIV-1 PR cleavage sites, namely EvoCleave, is proposed by integrating the coevolving information obtained from substrate sequences with a linear SVM classifier. The experiment results showed that EvoCleave yielded a very promising performance in terms of ROC analysis and f-measure. We also prospectively assessed the biological significance of coevolutionary patterns by applying them to study three fundamental issues of HIV-1 PR cleavage site. The analysis results demonstrated that the coevolutionary patterns offered valuable insights into the understanding of substrate specificity of HIV-1 PR.

Quality-of-service (QoS) data vary over time, making it vital to capture the temporal patterns hidden in such dynamic data for predicting missing ones with high accuracy. However, currently latent factor (LF) analysis-based QoS-predictors are mostly defined on static QoS data without the consideration of such temporal dynamics. To address this issue, this paper presents a biased non-negative latent factorization of tensors (BNLFTs) model for temporal pattern-aware QoS prediction. Its main idea is fourfold: 1) incorporating linear biases into the model for describing QoS fluctuations; 2) constraining the model to be non-negative for describing QoS non-negativity; 3) deducing a single LF-dependent, non-negative, and multiplicative update scheme for training the model; and 4) incorporating an alternating direction method into the model for faster convergence. The empirical studies on two dynamic QoS datasets from real applications show that compared with the state-of-the-art QoS-predictors, BNLFT represents temporal patterns more precisely with high computational efficiency, thereby achieving the most accurate predictions for missing QoS data.

High-dimensional and sparse (HiDS) matrices are commonly seen in big-data-related industrial applications like recommender systems. Latent factor (LF) models have proven to be accurate and efficient in extracting hidden knowledge from them. However, they mostly fail to fulfill the non-negativity constraints that describe the non-negative nature of many industrial data. Moreover, existing models suffer from slow convergence rate. An alternating-direction-method of multipliers-based non-negative LF (AMNLF) model decomposes the task of non-negative LF analysis on an HiDS matrix into small subtasks, where each task is solved based on the latest solutions to the previously solved ones, thereby achieving fast convergence and high prediction accuracy for its missing data. This paper theoretically analyzes the characteristics of an AMNLF model, and presents detailed empirical studies regarding its performance on nine HiDS matrices from industrial applications currently in use. Therefore, its capability of addressing HiDS matrices is justified in both theory and practice.

Due to the property of multiple solutions, redundant robot manipulators are usually required to simultaneously achieve multiple objectives in complex applications. The research of robustness for scheme formulation and optimization becomes an increasingly important issue for motion planning of redundant robot manipulators. From the perspective of optimization, a robust hybrid multiobjective (RHMO) scheme with a new disturbance rejection constraint is proposed in this article to achieve simultaneously four objectives together with the suppression of external time-varying disturbances. Theoretical results on the property of disturbance rejection are shown to confirm the effectiveness and robustness of the proposed RHMO scheme with a new disturbance rejection constraint. The RHMO scheme is then reformulated as dynamical quadratic programming with its solution found via the piecewise-linear projection equation neural network. Numerical experiments, tests, and comparisons on the basis of a PA10 manipulator verify the effectiveness, robustness, and superiority of the RHMO scheme with the new constraint for the motion planning and optimization of redundant robot manipulators against time-varying disturbances.

In this article, we present a metaheuristic-based control framework, called beetle antennae olfactory recurrent neural network, for simultaneous tracking control and obstacle avoidance of a redundant manipulator. The ability to avoid obstacles while tracking a predefined reference path is critical for any industrial manipulator. The formulated control framework unifies the tracking control and obstacle avoidance into a single constrained optimization problem by introducing a penalty term into the objective function, which actively rewards the optimizer for avoiding the obstacles. One of the significant features of the proposed framework is the way that the penalty term is formulated following a straightforward principle: maximize the minimum distance between a manipulator and an obstacle. The distance calculations are based on Gilbert–Johnson–Keerthi algorithm, which calculates the distance between a manipulator and an obstacle by directly using their three-dimensional geometries, which also implies that our algorithm works for a manipulator and an arbitrarily shaped obstacle. Theoretical treatment proves the stability and convergence, and simulations results using an LBR IIWA seven-DOF manipulator are presented to analyze the performance of the proposed framework.

Undirected, sparse and large-scaled networks existing ubiquitously in practical engineering are vitally important since they usually contain rich information in various patterns. Matrix factorization (MF) technique is an efficient method to extract the useful latent factors (LFs) from the LF model, which directly gives rise to the so-called MF model. However, most MF models cannot maintain some frequently encountered constraints such as nonnegativity of LFs and the symmetry of the target network. In addition, in spite of its potential capability of obtaining the effectiveness of both the computation and the storage, the currently developed double factorization (DF)-based model still suffers from the problem of the low prediction accuracy due to the limited amount of LFs. To address the above problems, a novel MF model is proposed in terms of the triple-factorization (TF) technique, thereby leading to TF-based symmetric and nonnegative latent factor (SNLF) models. Compared with the traditional DF-based SNLF model, the proposed TF-based SNLF model is equipped with: 1) constraints on symmetry and nonnegativity; 2) desirable performance with high accuracy; 3) the convergence of the algorithm; and 4) fairly low storage and computational complexity. Furthermore, in order to reduce overfitting so as to further improve the model performance, regularization is precisely considered into the proposed TF-based SNLF model. Experiments on real datasets show that the proposed TF-based SNLF model has a whelming ability of improving the estimation accuracy for the missing data as well as guaranteeing the symmetry of the target network and the nonnegativity of LFs at a little expense of the computation and storage burden. Moreover, it is easy to be implemented for the data analysis.

Online streaming feature selection has received extensive attention in the past few years. Existing approaches have a common assumption that the feature space of the fixed data instances increases dynamically without any missing entry. This assumption, however, does not always hold in many real-world applications. For example, in a credit evaluation system, we cannot collect the complete dynamic features for each person and/or enterprise. Motivated by this observation, this paper aims at conducting online feature selection from capricious streaming features, where features flow in one by one with some random missing entries while the number of data instances remains fixed. To do so, we propose a general framework named GF-CSF. The main idea of GF-CSF is to adopt latent factor analysis to preprocess capricious streaming features for completing their missing entries before conducting feature selection. Both theoretical and experimental analyses indicate that GF-CSF can efficiently improve any existing model of online streaming features selection to achieve online capricious streaming features selection.

The rapid development of mobile communication technologies prompts the emergence of mobile edge computing (MEC). As the key technology toward 5th generation (5G) wireless networks, it allows mobile users to offload their computational tasks to nearby servers deployed in base stations to alleviate the shortage of mobile resource. Nevertheless, various challenges, especially the edge-user-allocation problem, are yet to be properly addressed. Traditional studies consider this problem as a static global optimization problem where user positions are considered to be time-invariant and user-mobility-related information is not fully exploited. In reality, however, edge users are usually with high mobility and time-varying positions, which usually result in users reallocations among different base stations and impact on user-perceived quality-of-service (QoS). To overcome the above limitations, we consider the edge user allocation problem as an online decision-making and evolvable process and develop a mobility-aware and migration-enabled approach, named MobMig, for allocating users at real-time. Experiments based on real-world MEC dataset clearly demonstrate that our approach achieves higher user coverage rate and lower reallocations than traditional ones.

Non-negative latent factor (NLF) models have been frequently applied to information extraction, pattern recognition, and community detection. An NLF model can well represent a high-dimensional and sparse (HiDS) matrix of non-negative data and efficiently acquire useful knowledge from it. A single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is highly efficient to build NLF model. However, its convergence ability on such a matrix is still unveiled in theory. This paper presents the convergence property of an SLF-NMU algorithm. We theoretically prove that it can guarantee its convergence to a Karush-Kuhn-Tucher (KKT) stationary point. Empirical studies on two HiDS matrices from practical applications indicate that an SLF-NMU algorithm make an NLF model to converge at a relatively steady state.

Protein complexes are crucial in improving our understanding of the mechanisms employed by proteins. Various computational algorithms have thus been proposed to detect protein complexes from protein interaction networks. However, given massive protein interactome data obtained by high-throughput technologies, existing algorithms, especially those with additionally consideration of biological information of proteins, either have low efficiency in performing their tasks or suffer from limited effectiveness. For addressing this issue, this work proposes to detect protein complexes from a protein interaction network with high efficiency and effectiveness. To do so, the original detection task is first formulated into an optimization problem according to the intuitive properties of protein complexes. After that, the framework of alternating direction method of multipliers is applied to decompose this optimization problem into several subtasks, which can be subsequently solved in a separate and parallel manner. An algorithm for implementing this solution is then developed. Experimental results on five large protein interaction networks demonstrated that compared to state-of-the-art protein complex detection algorithms, our algorithm outperformed them in terms of both effectiveness and efficiency. Moreover, as number of parallel processes increases, one can expect an even higher computational efficiency for the proposed algorithm with no compromise on effectiveness.

Neural networks have been generally deemed as important tools to handle kinds of online computing problems in recent decades, which have plenty of applications in science and electronics fields. This paper proposes a novel recurrent neural network (RNN) to handle the perturbed time-varying underdetermined linear system with double bound limits on residual errors and state variables. Beyond that, the bound-limited underdetermined linear system is converted into a time-varying system that consists of linear and nonlinear formulas through constructing a nonnegative time-varying variable. Then, theoretical analyses are conducted to verify the superior convergence performance of the proposed RNN model. Furthermore, numerical experiment results and computer simulations demonstrate the superiority and effectiveness of the proposed RNN model for handling the time-varying underdetermined linear system with double bound limits. Finally, the proposed RNN model is applied to the physically limited PUMA560 robot to show its satisfactory applicabilities.

Accurately estimating unknown quality-of-service (QoS) data based on historical records of Web-service invocations is vital for automatic service selection. This work presents an effective scheme for addressing this issue via alternating direction method-based matrix factorization. Its main idea consists of a) adopting the principle of the alternating direction method to decompose the task of building a matrix factorization-based QoS-estimator into small subtasks, where each one trains a subset of desired parameters based on the latest status of the whole parameter set; b) building an ensemble of diversified single models with sophisticated diversifying and aggregating mechanism; and c) parallelizing the construction process of the ensemble to drastically reduce the time cost. Experimental results on two industrial QoS datasets demonstrate that with the proposed scheme, more accurate QoS estimates can be achieved than its peers with comparable computing time with the help of its practical parallelization.

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. On the other hand, a momentum method is frequently adopted to accelerate a learning algorithm explicitly depending on gradients, yet it is incompatible with learning algorithms implicitly depending on gradients, like SLF-NMU. To build a fast NLF model, we firstly propose a generalized momentum method compatible with SLF-NMU. With it, we propose the single latent factor-dependent, non-negative, multiplicative and momentum-integrated update (SLF-NM2U) algorithm for accelerating the building process of an NLF model, thereby achieving a fast non-negative latent factor (FNLF) model. Empirical studies on six HiDS matrices from industrial application indicate that with the incorporated momentum effects, FNLF outperforms NLF in terms of both convergence rate and prediction accuracy for missing data. Hence, compare with an NLF model, an FNLF model is more practical in industrial applications.

Generating highly accurate predictions for missing quality-of-service (QoS) data is an important issue. Latent factor (LF)-based QoS-predictors have proven to be effective in dealing with it. However, they are based on first-order solvers that cannot well address their target problem that is inherently bilinear and nonconvex, thereby leaving a significant opportunity for accuracy improvement. This paper proposes to incorporate an efficient second-order solver into them to raise their accuracy. To do so, we adopt the principle of Hessian-free optimization and successfully avoid the direct manipulation of a Hessian matrix, by employing the efficiently obtainable product between its Gauss-Newton approximation and an arbitrary vector. Thus, the second-order information is innovatively integrated into them. Experimental results on two industrial QoS datasets indicate that compared with the state-of-the-art predictors, the newly proposed one achieves significantly higher prediction accuracy at the expense of affordable computational burden. Hence, it is especially suitable for industrial applications requiring high prediction accuracy of unknown QoS data.

In this paper, a neural-dynamic distributed scheme is proposed for the cooperative control of multiple redundant manipulators with limited communications. It is guaranteed that, with the communication network being connected, all manipulators can jointly reach the same desired motion. The proposed distributed scheme is rearranged as a time-varying quadratic program and solved online by a Zhang neural network. Then, theoretical analyses show that, without noise, the proposed distributed scheme is able to execute a given task with exponentially convergent position errors. Moreover, an explicit bound relationship between the control input noise and the end-effector position error is analytically derived. Furthermore, numerical comparisons substantiate the superiority, effectiveness, and accuracy of the proposed distributed scheme.

High-dimensional and sparse (HiDS) matrices are commonly encountered in many big-data-related and industrial applications like recommender systems. When acquiring useful patterns from them, nonnegative matrix factorization (NMF) models have proven to be highly effective owing to their fine representativeness of the nonnegative data. However, current NMF techniques suffer from: 1) inefficiency in addressing HiDS matrices; and 2) constraints in their training schemes. To address these issues, this paper proposes to extract nonnegative latent factors (NLFs) from HiDS matrices via a novel inherently NLF (INLF) model. It bridges the output factors and decision variables via a single-element-dependent mapping function, thereby making the parameter training unconstrained and compatible with general training schemes on the premise of maintaining the nonnegativity constraints. Experimental results on six HiDS matrices arising from industrial applications indicate that INLF is able to acquire NLFs from them more efficiently than any existing method does.

Self-labeled technique, a paradigm of semisupervised classification (SSC), is highly effective in alleviating the shortage of labeled data in classification tasks via an iterative self-labeling process. Although existing self-labeled SSC models show great prospect in industrial applications, they suffer from performance degeneration caused by false-positive label-predictions of unlabeled data during the iterative self-labeling process. For addressing this issue, this paper proposes a novel SSC framework, which is highly compatible with most existing self-labeled SSC models. The main idea of this framework is to incorporate a differential-evolution-based positioning optimization algorithm for classification into the iterative self-labeling process, aiming at optimizing the positioning of newly labeled data. Specifically, five representative self-labeled SSC models with different characteristics are modified based on the proposed framework to check their performances. Experimental results on 45 benchmark datasets demonstrate that the proposed framework is highly compatible with tested self-labeled SSC models, and significantly effective in improving their performances.

Manipulators are subject to physical constraints at different levels, i.e., joint angle limits, joint velocity limits, and acceleration limits. Effective resolution of redundant manipulators with compliance to the physical constraints is a fundamental issue for safe operation. Existing results generally resolve the manipulator redundancy either at the velocity level or the acceleration level. On the one hand, the velocity-level redundancy resolution scheme is able to deal with the joint angle and joint velocity limits successfully but cannot address the joint acceleration limit. On the other hand, although the existing acceleration-level redundancy resolution scheme is able to overcome the failure of the velocity-level one in complying with acceleration constraints, it is at the cost of making the system equation more complicated, e.g., the dependence on the time derivative of the Jacobian matrix. Whether it is possible to conduct redundancy resolution at the velocity level but with the compliance to joint angle constraints, joint velocity constraints, and joint acceleration constraints remains an open problem in past decades. This paper gives a positive answer to this pending problem by providing a novel scheme. In the proposed scheme, the redundancy resolution problem is formulated as a quadratic program subject to joint angle, velocity, and acceleration constraints with the joint velocity being the decision variable and joint velocity norm as the performance index, which is widely adopted and closely related to the energy consumption. Then, a projection neural network is designed and proposed to online solve the problem with the joint acceleration constraint handled. Theoretical analysis is performed to guarantee the global convergence of the proposed projection neural network to the optimal solution to the redundancy resolution problem. Besides, simulation results based on a PUMA 560 industrial manipulator are presented and compared to verify the theoretical result and substantiate the efficacy and superiority of the proposed scheme.

In recent decades, primal-dual neural networks, as a special type of recurrent neural networks, have received great success in real-time manipulator control. However, noises are usually ignored when neural controllers are designed based on them, and thus, they may fail to perform well in the presence of intensive noises. Harmonic noises widely exist in real applications and can severely affect the control accuracy. This work proposes a novel primal-dual neural network design that directly takes noise control into account. By taking advantage of the fact that the unknown amplitude and phase information of a harmonic signal can be eliminated from its dynamics, our deliberately designed neural controller is able to reach the accurate tracking of reference trajectories in a noisy environment. Theoretical analysis and extensive simulations show that the proposed controller stabilizes the control system polluted by harmonic noises and converges the position tracking error to zero. Comparisons show that our proposed solution consistently and significantly outperforms the existing primal-dual neural solutions as well as feedforward neural one and adaptive neural one for redundancy resolution of manipulators.

Undirected, high dimensional and sparse networks are frequently encountered in industrial applications. They contain rich knowledge regarding various useful patterns. Non-negative latent factor (NLF) models have proven to be effective and efficient in acquiring useful knowledge from asymmetric networks. However, they cannot correctly describe the symmetry of an undirected network. For addressing this issue, this work analyzes the NLF extraction processes on asymmetric and symmetric matrices respectively, thereby innovatively achieving the symmetric and non-negative latent factor (SNLF) models for undirected, high dimensional and sparse networks. The proposed SNLF models are equipped with a) high efficiency, b) non-negativity, and c) symmetry. Experimental results on real networks show that they are able to a) represent the symmetry of the target network rigorously; b) maintain the non-negativity of resulting latent factors; and c) achieve high computational efficiency when performing data analysis tasks as missing data estimation.

This paper investigates the distributed motion planning of multiple robot arms with limited communications in the presence of noises. To do this, a nonlinearly-activated noise-tolerant zeroing neural network (NANTZNN) is designed and presented for the first time for solving the presented distributed scheme online. Theoretical analyses and simulation results show the effectiveness and accuracy of the presented distributed scheme with the aid of NANTZNN model.

The speed control of permanent magnet brushed (PMB) DC motors at low speeds is difficult due to the nonlinearity caused by various types of frictions. Under parameter uncertainty, the speed control becomes more difficult. In this paper, to handle the parameter uncertainty, we propose a dynamic neural network to adaptively reconstruct or learn the dynamics of PMB DC motors. Then, based on the parameters of the neural dynamic model, a near-optimal dynamic neural controller is designed and proposed for the speed control of PMB DC motors with frictions considered under parameter uncertainty. Simulations substantiate the efficacy of the proposed dynamic neural model and adaptive near-optimal controller for PMB DC motors with fully unknown parameters.

Protein-protein interactions (PPIs) play a central role in many biological processes. Although a large amount of human PPI data has been generated by high-throughput experimental techniques, they are very limited compared to the estimated 130 000 protein interactions in humans. Hence, automatic methods for human PPI-detection are highly desired. This work proposes a novel framework, i.e., Low-rank approximation-kernel Extreme Learning Machine (LELM), for detecting human PPI from a protein's primary sequences automatically. It has three main steps: 1) mapping each protein sequence into a matrix built on all kinds of adjacent amino acids; 2) applying the low-rank approximation model to the obtained matrix to solve its lowest rank representation, which reflects its true subspace structures; and 3) utilizing a powerful kernel extreme learning machine to predict the probability for PPI based on this lowest rank representation. Experimental results on a large-scale human PPI dataset demonstrate that the proposed LELM has significant advantages in accuracy and efficiency over the state-of-art approaches. Hence, this work establishes a new and effective way for the automatic detection of PPI.

Undirected, high-dimensional, and sparse (HiDS) networks are frequently encountered in industrial applications. They contain rich knowledge regarding various useful patterns. Nonnegative latent factor (NLF) models are effective and efficient in extracting useful knowledge from directed networks. However, they cannot describe the symmetry of an undirected network. For addressing this issue, this paper analyzes the extraction process of NLFs on asymmetric and symmetric matrices, respectively, thereby innovatively achieving the symmetric and nonnegative latent factor (SNLF) models for undirected, HiDS networks. The proposed SNLF models are equipped with: 1) high efficiency; 2) nonnegativity; and 3) symmetry. Experimental results on real networks show that the SNLF models are able to: 1) describe the symmetry of the target network rigorously; 2) ensure the nonnegativity of resultant latent factors; and 3) achieve high computational efficiency when addressing data analysis tasks like missing data estimation.

High-dimensional and sparse (HiDS) matrices are commonly encountered in many big data-related industrial applications like recommender systems. When acquiring useful patterns from them, non-negative matrix factorization (NMF) models have proven to be highly effective because of their fine representativeness of non-negative data. However, current NMF techniques suffer from a) inefficiency in addressing HiDS matrices, and b) constrained training schemes lack of flexibility, extensibility and adaptability. To address these issues, this work proposes to factorize industrial-size sparse matrices via a novel Inherently Non-negative Latent Factor (INLF) model. It connects the output factors and decision variables via a single-element-dependent sigmoid function, thereby innovatively removing the non-negativity constraints from its training process without impacting the solution accuracy. Hence, its training process is unconstrained, highly flexible and compatible with general learning schemes. Experimental results on five HiDS matrices generated by industrial applications indicate that INLF is able to acquire non-negative latent factors from them in a more efficient manner than any existing method does.

With the exploration of the World Wide Web, more and more entities are involved in various online applications, e.g., recommender systems and social network services. In such context, high-dimensional sparse matrices describing the relationships among them are frequently encountered. It is highly important to develop efficient non-negative latent factor (NLF) models for these high-dimensional sparse relationships because of a) their ability to extract useful knowledge from them; b) their fulfillment of the non-negativity constraints for representing most non-negative industrial data; and c) their high computational and storage efficiency on high-dimensional sparse matrices. However, due to the imbalanced distribution of known data in such a matrix, it is necessary to investigate the regularization effect in NLF models. We first review the NLF model briefly. Then we propose to integrate the frequency-weight on each involved entity into its Tikhonov regularization terms, for representing the imbalanced data from a high-dimensional sparse matrix. Experimental results on industrial-size matrices indicate that the proposed scheme is effective in improving the performance of the NLF model in missing-data-estimation.

BACKGROUND: Proteins are the important molecules which participate in virtually every aspect of cellular function within an organism in pairs. Although high-throughput technologies have generated considerable protein-protein interactions (PPIs) data for various species, the processes of experimental methods are both time-consuming and expensive. In addition, they are usually associated with high rates of both false positive and false negative results. Accordingly, a number of computational approaches have been developed to effectively and accurately predict protein interactions. However, most of these methods typically perform worse when other biological data sources (e.g., protein structure information, protein domains, or gene neighborhoods information) are not available. Therefore, it is very urgent to develop effective computational methods for prediction of PPIs solely using protein sequence information. RESULTS: In this study, we present a novel computational model combining weighted sparse representation based classifier (WSRC) and global encoding (GE) of amino acid sequence. Two kinds of protein descriptors, composition and transition, are extracted for representing each protein sequence. On the basis of such a feature representation, novel weighted sparse representation based classifier is introduced to predict protein interaction class. When the proposed method was evaluated with the PPIs data of S. cerevisiae, Human and H. pylori, it achieved high prediction accuracies of 96.82, 97.66 and 92.83 % respectively. Extensive experiments were performed for cross-species PPIs prediction and the prediction accuracies were also very promising. CONCLUSIONS: To further evaluate the performance of the proposed method, we then compared its performance with the method based on support vector machine (SVM). The results show that the proposed method achieved a significant improvement. Thus, the proposed method is a very efficient method to predict PPIs and may be a useful supplementary tool for future proteomics studies.

Summary Cloud computing is a novel paradigm capable of rationalizing the use of computational resources by means of outsourcing and virtualization. Elasticity is one of the most attractive features of cloud computing. Elastic clouds are able to adapt to workload changes by provisioning and de‐provisioning resources in an autonomic manner, such that at each point in time the available resources match the current demand as closely as possible. However, elasticity adds complexity, which makes quantitative analysis of cloud performance and power consumption difficult. Such analysis is required to evaluate and quantify the cost‐benefit of a strategy portfolio and the quantitative runtime performance and power consumption experienced by cloud‐users. In this study, we present a comprehensive analytical approach to performance and power consumption analysis of elastic clouds. Several metrics are defined and evaluated: expected task completion time, power consumption rate, and task rejection rate under different load conditions, elasticity intensities, and error intensities. To validate the proposed approach, we obtain experimental data through a real‐world cloud and conduct a confidence interval analysis. The analysis results suggest the perfect coverage of theoretical results by corresponding experimental confidence intervals. Copyright © 2016 John Wiley & Sons, Ltd.

Collaborative filtering (CF)-based recommenders are achieved by matrix factorization (MF) to obtain high prediction accuracy and scalability. Most current MF-based models, however, are static ones that cannot adapt to incremental user feedbacks. This work aims to develop a general, incremental- and-static-combined scheme for MF-based CF to obtain highly accurate and computationally affordable incremental recommenders. With it, a recommender is designed to consist of two components, i.e., a static one built on static rating data, and an incremental one built on a sub-matrix related to rating-variations only. Highly reliable predictions are thus generated by fusing their results. The experiments on large industrial datasets show that desired accuracy and acceptable computational complexity are achieved by the resulting recommender with the proposed scheme.

The extreme learning machine (ELM) has drawn insensitive research attentions due to its effectiveness in solving many machine learning problems. However, the matrix inversion operation involved in the algorithm is computational prohibitive and limits the wide applications of ELM in many scenarios. To overcome this problem, in this paper, we propose an inverse-free ELM to incrementally increase the number of hidden nodes, and update the connection weights progressively and optimally. Theoretical analysis proves the monotonic decrease of the training error with the proposed updating procedure and also proves the optimality in every updating step. Extensive numerical experiments show the effectiveness and accuracy of the proposed algorithm.

Nonnegative matrix factorization (NMF)-based models possess fine representativeness of a target matrix, which is critically important in collaborative filtering (CF)-based recommender systems. However, current NMF-based CF recommenders suffer from the problem of high computational and storage complexity, as well as slow convergence rate, which prevents them from industrial usage in context of big data. To address these issues, this paper proposes an alternating direction method (ADM)-based nonnegative latent factor (ANLF) model. The main idea is to implement the ADM-based optimization with regard to each single feature, to obtain high convergence rate as well as low complexity. Both computational and storage costs of ANLF are linear with the size of given data in the target matrix, which ensures high efficiency when dealing with extremely sparse matrices usually seen in CF problems. As demonstrated by the experiments on large, real data sets, ANLF also ensures fast convergence and high prediction accuracy, as well as the maintenance of nonnegativity constraints. Moreover, it is simple and easy to implement for real applications of learning systems.

Automatic Web-service selection is an important research topic in the domain of service computing. During this process, reliable predictions for quality of service (QoS) based on historical service invocations are vital to users. This work aims at making highly accurate predictions for missing QoS data via building an ensemble of nonnegative latent factor (NLF) models. Its motivations are: 1) the fulfillment of nonnegativity constraints can better represent the positive value nature of QoS data, thereby boosting the prediction accuracy and 2) since QoS prediction is a learning task, it is promising to further improve the prediction accuracy with a carefully designed ensemble model. To achieve this, we first implement an NLF model for QoS prediction. This model is then diversified through feature sampling and randomness injection to form a diversified NLF model, based on which an ensemble is built. Comparison results between the proposed ensemble and several widely employed and state-of-the-art QoS predictors on two large, real data sets demonstrate that the former can outperform the latter well in terms of prediction accuracy.

Latent-factor-based Quality-of-Service predictors can achieve high prediction accuracy and good scalability. However, most of them are based on first-order models that cannot well deal with their target problem that is inherently non-convex. Since second-order approaches have proven to be effective to such problems, this work proposes to implement a second-order predictor with an aim to achieve the high accuracy unlikely obtained by any existing methods. To do so, this work adopts the principle of Hessian-free optimization and successfully avoids the usage of a Hessian matrix by employing the efficiently obtainable product between its Gauss-Newton approximation and an arbitrary vector. Experimental results on two industrial QoS datasets indicate that the newly proposed predictor is highly accurate with fine computational efficiency.

SUMMARY Web services are emerging as a major technology for deploying automated interactions between distributed and heterogeneous applications. The accurate prediction of their quality of service (QoS) is important because their users rely on it to decide whether they meet the QoS requirement. The existing studies of QoS prediction usually assume that QoS of service activities follows certain distributions. These distributions are used as static model inputs into stochastic process models to obtain analytical QoS results. Instead, we consider the QoS activities to be fluctuating and introduce a dynamic framework to predict the runtime QoS by employing an Autoregressive Moving Average Model and QoS reduction rules. In the case study of a real‐world composite service sample, a comparison between existing approaches and the proposed one is presented, and results suggest that the proposed one achieves higher prediction accuracy.Copyright © 2014 John Wiley & Sons, Ltd.

Cloud computing is a recently developed new technology for complex systems with massive service sharing, which is different from the resource sharing of the grid computing systems. In a cloud environment, service requests from users go through numerous provider-specific steps from the instant it is submitted to when the requested service is fully delivered. Quality modeling and analysis of clouds are not easy tasks because of the complexity of the automated provisioning mechanism and dynamically changing cloud environment. This work proposes an analytical model-based approach for quality evaluation of Infrastructure-as-a-Service cloud by considering expected request completion time, rejection probability, and system overhead rate as key quality metrics. It also features with the modeling of different warm-up and cool-down strategies of machines and the ability to identify the optimal balance between system overhead and performance. To validate the correctness of the proposed model, we obtain simulative quality-of-service (QoS) data and conduct a confidence interval analysis. The result can be used to help design and optimize industrial cloud computing systems.

Recommender systems are an important kind of learning systems, which can be achieved by latent-factor (LF)-based collaborative filtering (CF) with high efficiency and scalability. LF-based CF models rely on an optimization process with respect to some desired latent features; however, most of them employ first-order optimization algorithms, e.g., gradient decent schemes, to conduct their optimization task, thereby failing in discovering patterns reflected by higher order information. This work proposes to build a new LF-based CF model via second-order optimization to achieve higher accuracy. We first investigate a Hessian-free optimization framework, and employ its principle to avoid direct usage of the Hessian matrix by computing its product with an arbitrary vector. We then propose the Hessian-free optimization-based LF model, which is able to extract latent factors from the given incomplete matrices via a second-order optimization process. Compared with LF models based on first-order optimization algorithms, experimental results on two industrial datasets show that the proposed one can offer higher prediction accuracy with reasonable computational efficiency. Hence, it is a promising model for implementing high-performance recommenders.

Cloud computing is a promising paradigm capable of rationalizing the use of computational resources by means of outsourcing and virtualization. Virtualization allows to instantiate virtual machines (VMs) on top of fewer physical systems managed by a VM manager. Performance evaluation of clouds is required to evaluate and quantify the cost-benefit of a strategy portfolio and the quality of service (QoS) experienced by end-users. Such evaluation is not feasible by means of simulation or on-the-field measurement, due to the great scale of parameter spaces that have to be traversed. In this study, we present a stochastic-queuing-network-based approach to performance analysis of migration-enabled clouds in error-prone environment. Several performance metrics are defined and evaluated: utilization, expected task completion time, and task rejection rate under different load conditions and error intensities. To validate the proposed approach, we obtain experimental performance data through a real-world cloud and conduct a confidence-interval analysis. The analysis results suggest the perfect coverage of theoretical performance results by corresponding experimental confidence intervals.

With the increasing call for green cloud, reducing energy consumption has been an important requirement for cloud resource providers not only to reduce operating costs, but also to improve system reliability. Dynamic voltage scaling (DVS) has been a key technique in exploiting the hardware characteristics of cloud datacenters to save energy by lowering the supply voltage and operating frequency. This paper presents a novel stochastic framework for energy efficiency and performance analysis of DVS-enabled cloud. This framework uses virtual machine request arrival rate, failure rate, repair rate, and service rate of datacenter servers as model inputs. Based on a queuing-network-based analysis, this paper gives analytic solutions of three metrics. The proposed framework can be used to help the design and optimization of energy-aware high performance cloud systems.

Matrix-factorization (MF)-based approaches prove to be highly accurate and scalable in addressing collaborative filtering (CF) problems. During the MF process, the non-negativity, which ensures good representativeness of the learnt model, is critically important. However, current non-negative MF (NMF) models are mostly designed for problems in computer vision, while CF problems differ from them due to their extreme sparsity of the target rating-matrix. Currently available NMF-based CF models are based on matrix manipulation and lack practicability for industrial use. In this work, we focus on developing an NMF-based CF model with a single-element-based approach. The idea is to investigate the non-negative update process depending on each involved feature rather than on the whole feature matrices. With the non-negative single-element-based update rules, we subsequently integrate the Tikhonov regularizing terms, and propose the regularized single-element-based NMF (RSNMF) model. RSNMF is especially suitable for solving CF problems subject to the constraint of non-negativity. The experiments on large industrial datasets show high accuracy and low-computational complexity achieved by RSNMF.

Ontology Web Language for Services (OWL-S), one of the most significant semantic Web service ontologies proposed to date, provides a core ontological framework and guidelines for describing the properties and capabilities of services in an unambiguous computer-interpretable form. Analysis of the quality of service of composite service processes specified in OWL-S enables service users to decide whether the process meets nonfunctional requirements. In this paper, we propose a probabilistic approach for reliability analysis of OWL-S processes, employing the non-Markovian stochastic Petri net (NMSPN) as the fundamental model. Based on the NMSPN representations of the OWL-S elements, we introduce an analytical method for the calculation of the process-normal-completion probability as the reliability estimate. This method takes the probabilistic parameters of service invocations and messages as model inputs. To validate the feasibility and accuracy of our approach, we obtain runtime experimental data and conduct a confidence interval analysis in a case study. A sensitivity analysis is also performed to determine the impact of model parameters on reliability and to help identify the reliability bottlenecks.

As an application of cloud storage, cloud backup is effective, reliable, extensible, cost-effective and usable, which makes cloud backup having a broad prospect of application. We proposed a reliable cloud backup scheduling algorithm that showing good performance. In this algorithm, data access time and data reliability are introduced. With the cloud state table and cloud resources table, the algorithm determines which cloud storage node will save the data. The simulated experiment shows that the proposed algorithm, guaranteeing the data reliability, gets good data access performance.

SUMMARY The Ontology Web Language (OWL)‐based web service ontology is one of the major standards for modeling and description of distributed service composition. To analyze the performance of composite service, processes specified in OWL‐S give way to tell whether the process meets the performance requirements and to choose the process with higher efficiency from those with similar function. In this paper, we propose a stochastic Petri net‐based approach for performance analysis of OWL‐S processes, which employs non‐Markovian stochastic Petri nets as the intermediate representation. The main innovation of this research includes a translation from OWL‐S to non‐Markovian stochastic Petri nets and a performance (using expected‐process‐normal‐completion‐time as the metric) analysis method. We also validate the correctness of the approach in the case study by showing 90% CI obtained from experimental results that cover corresponding theoretical prediction values. Copyright © 2012 John Wiley & Sons, Ltd.

Latent Factor Model (LFM) based approaches are becoming popular when implementing Collaborative Filtering (CF) recommenders, due to their high recommendation accuracy. However, current LFM approaches address the accuracy issue only based on the rating data, whereas early research indicates that integrating information from additional data sources is helpful to the recommendation accuracy. In this work we focus on improving the recommendation accuracy of a LFM based CF recommender by integrating folksonomy information. To implement this approach, we first propose a novel model named Item Folsonomy Relevance (IFR) to analyze the item relevance inside the folksonomy; we subsequently integrate the latent factors of the IFR model and rating data through probabilistic matrix factorization (PMF), a state-of-the-art matrix factorization technique, to produce recommendations based on information from both the ratings and folksonomy simultaneously. The experiments on MovieLens dataset showed that compared to two state-of-the-art LFM approaches and another folksonomy-augmented recommder, our approach could obtain advantage in recommendation accuracy.