Predicting temporal Quality of Service (QoS) data is critical for optimizing network services and rationalizing resource allocation in cloud computing and service-oriented systems. Existing mainstream methods have achieved promising predictive performance. However, their purely data-driven manner limits their ability to capture non-stationary temporal patterns, thereby leading to accuracy degradation when temporal QoS data exhibits fluctuations. To tackle this limitation, we propose a novelExtendedKalman Filter-EnhancedLatent Feature Analysis (EKL) model to perform efficient and accurate temporal QoS prediction from the perspective of bidirectional model-data-driven. Its main idea is three-fold: a) designing a model-driven feature producer to obtain the temporal latent features to capture the intricate temporal pattern following the principle of an Extended Kalman Filter; b) building a data-driven feature producer based on the alternating least squares algorithm to identify time-invariant latent features describing intrinsic user-service characteristics; c) exploiting a density-oriented parallel strategy that achieves workload balancing by sorting users in accordance with their service invocation density, which effectively elevates computational efficiency. In addition, we provide a rigorous theoretical analysis to formally prove the convergence of the proposed EKL. Experimental evaluations conducted on real-world temporal QoS datasets reveal that our proposed EKL surpasses existing state-of-the-art models with respect to both computational efficiency and prediction accuracy for missing temporal QoS data.

High-dimensional and incomplete (HDI) data are ubiquitous for representing the intricate interactions among a vast number of nodes arising from diverse real application scenarios. Nonnegative latent factor (NLF) models have demonstrated their effectiveness in extracting critical latent features from HDI data by leveraging a single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) algorithm. However, the SLF-NMU algorithm always leads NLF models to converge sluggishly as it updates an LF based only on the current updated information. To address this critical issue, we present APNLF, which innovatively adopts an adaptive proportional–integral–derivative (PID) controller to enable the learning process of SLF-NMU to be more efficient. The proposed APNLF encompasses the following twofold ideas: 1) establishing a PID-increment-based SLF-NMU (PSN) algorithm, which updates an LF by comprehensively modeling the current, past, and future update increment information guided by the principle of a PID controller; and 2) designing an effective fuzzy reasoning rule to implement all hyperparameters adaptation, which boosts model’s applicability in practical applications. Moreover, APNLF’s convergence analysis is provided in theory. Experiments on five HDI datasets demonstrate that the APNLF model outperforms the state-of-the-art models in efficiency and accuracy on an HDI matrix. The source code is available athttps://github.com/Aaaaapplege/APNLF-Development

A dynamic graph (DG) is frequently encountered in numerous real-world scenarios. Consequently, A dynamic graph convolutional network (DGCN) has been successfully applied to perform precise representation learning on a DG. However, conventional DGCNs typically consist of a static GCN coupled with a sequence neural network (SNN) to model spatial and temporal patterns separately. This decoupled modeling mechanism inherently disrupts the intricate spatio-temporal dependencies. To address the issue, this study proposes a novel Tensorized Lightweight Graph Convolutional Network (TLGCN) for accurate dynamic graph learning. It mainly contains the following two key concepts: a) designing a novel spatio-temporal information propagation method for joint propagation of spatio-temporal information based on the tensor M-product framework; b) proposing a tensorized lightweight graph convolutional network based on the above method, which significantly reduces the memory occupation of the model by omitting complex feature transformation and nonlinear activation. Numerical experiments on four real-world datasets demonstrate that the proposed TLGCN outperforms the state-of-the-art models in the weight estimation task on DGs.

Dynamic graphs offer more precise modeling of real-world applications compared to static graphs. Therefore, learning on dynamic graphs has garnered significant research attention in recent years. Unfortunately, the current approaches for learning dynamic graphs remain inadequate in capturing global spatial-temporal dependency. This issue arises from capturing biased spatial and temporal dependencies, thereby weakening the coupling between spatial and temporal dimensions. To overcome it, we propose a Self-Reliant Global Dependency Apprehending Framework on Dynamic Graphs, namely SGD-DyG. Specifically, we first design a frequency self-enhanced learning module that examines the global inherent interactions of the node features hidden in the frequency domain. Furthermore, we propose a global-local-mixed self-supervised learning module that maximizes spatial-temporal mutual information between local node and global graph embeddings. Extensive experiments on seven real-world dynamic graph datasets validate that the proposed SGD-DyG consistently exceeds state-of-the-art models.

As modern manufacturing advances, suppressing residual vibrations in flexible structures and underactuated systems becomes crucial. Input shaping has garnered attention for its effectiveness in mitigating vibrations and enhancing motion performance. However, existing input shapers typically suffer from unavoidable time delays, modeling inaccuracies, and limited adaptability to uncertain systems, leading to suboptimal performance. To address these critical issues, this paper proposes a Particle Swarm Optimization-based Residual Negative Magnitude (PRV) vibration control scheme with two innovations: a) utilizing the zero vibration shaper to collect system data and employing a data-driven particle swarm optimization (PSO) algorithm to estimate system errors; and b) designing a robust PSO-based Residual negative magnitude (PR) shaper to reduce time delays, address modeling errors, and adapt to diverse system configurations. To validate its performance, two real-world datasets from two laboratory platforms have been established and made publicly available. Empirical results demonstrate that the proposed PR shaper outperforms state-of-the-art methods, and the proposed PRV scheme achieves significant vibration control effects.

An undirected weighted graph (UWG) is the fundamental data representation in various real applications. A graph convolution network is frequently utilized for representation learning to a UWG. Nevertheless, existing graph convolutional networks (GCNs) only consider a node's neighborhood during the embedding propagation, which regrettably decreases its representation learning capability due to the information loss in the modeling phase. Motivated by this discovery, this study proposes an adaptive neighborhood-resonated graph convolution network (ANR-GCN) with the following ideas: 1) establishing the weighted embedding propagation with the consideration of link weights in a UWG, thereby incorporating the interaction strength of each node pair into the ANR-GCN model; 2) building the neighborhood-regularization (NR) to make each node resonate with its neighborhoods, thus reinforcing the informative neighborhood information for improving the ANR-GCN's representation capability to the complex topology of the target UWG; and 3) diversifying the NR effects following the attention principle for guaranteeing the ANR-GCN's learning capacity. The proposed ANR-GCN's representation learning ability to a UWG is theoretically guaranteed from the perspectives of bounded generalization error and uniform stability. Extensive experiments on four UWG datasets illustrate that the proposed ANR-GCN significantly outperforms state-of-the-art GCNs in missing edge detection in a UWG, which evidently demonstrates its superior performance.

An undirected weighted graph (UWG) is regularly adopted to portray the interactions among a solo set of nodes from big data-connected applications such as the interactive confidence between proteins in a protein network. A graph convolutional network (GCN) is able to represent a UWG for subsequent pattern analysis tasks such as missing link estimation. However, existing GCNs mostly neglect the local collaborative information hidden in connected node pairs, which leads to severe information loss. To address this issue, this study proposes a node-collaboration-informed graph convolutional network (NGCN) model for implementing the precise UWG representation learning with threefold ideas: 1) extracting the nodes' global graph characteristics via incorporating the residual connection and weighted representation propagation into the GCN module; 2) learning the nodes' local collaborative information from the observed interactive node pairs via a symmetric latent factor analysis (SLFA) module; and 3) designing an effective strategy to fuse the nodes' global graph characteristics and local collaborative information adaptively for highly accurate representation to the target UWG. Its high representation ability to target UWG is proved in theory. Empirical studies on six UWGs generated by real-world applications indicate that owing to its elegant modeling for the node collaborations, the proposed NGCN significantly outperforms several leading-edge models in estimation accuracy to the missing links of a UWG. Its high scalability ensures its compatibility with other GCN extensions, which will be investigated in the future.

A dynamic weighted directed graph (DWDG) is commonly encountered in various application scenarios. It involves extensive dynamic interactions among numerous nodes. Most existing approaches explore the intricate temporal patterns hidden in a DWDG from the purely data-driven perspective, which suffers from accuracy loss when a DWDG exhibits strong fluctuations over time. To address this issue, this study proposes a novel Extended-Kalman-Filter-Incorporated Latent, Feature (EKLF) model to represent a DWDG from the model-driven perspective. Its main idea is divided into the following two-fold ideas: a) adopting a control model, i.e., the Extended Kalman Filter (EKF), to track the complex temporal patterns precisely with its nonlinear state-transition and observation functions; and b) introducing an alternating least squares (ALS) algorithm to train the latent features (LFs) alternatively for precisely representing a DWDG. Empirical studies on DWDG datasets demonstrate that the proposed EKLF model outperforms state-of-the-art models in prediction accuracy and computational efficiency for missing edge weights of a DWDG. It unveils the potential for precisely representing a DWDG by incorporating a control model.

High-dimensional and incomplete (HDI) data are commonly encountered in various big data-related applications concerning the complex interactions among numerous nodes, such as the user-item iterations in a recommender system. A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can perform efficient representation learning to such HDI data, thereby extracting useful knowledge from them. However, a standard SGD algorithm updates a latent factor based on the current stochastic gradient only, without the considerations on the past information, making a resultant model suffer from slow convergence. To address this critical issue, this paper proposes an Adaptive Non-linear PID-incorporated SGD (ANPS) algorithm with two-fold ideas: 1) rebuilding the instant learning error when computing the stochastic gradient following the principle of a nonlinear PID controller to incorporate past update information into the learning scheme efficiently, and 2) implementing gain parameter adaptation following the principle of particle swarm optimization (PSO). Experiments on six widely-adopted HDI datasets demonstrate that compared with state-of-the-art LFA models, an ANPS-based LFA model achieves significant advantage in both efficiency and accuracy. Moreover, its flexible gain parameter adaptation mechanism greatly boosts its practicability for real issues.Note to Practitioners—In many industrial applications like recommender systems, social network systems, and cloud service systems, people usually encounter numerous nodes and their highly-incomplete relationships. An HDI matrix is commonly adopted to describe such specific relationships. One of the major challenges is to acquire useful knowledge from an HDI matrix efficiently and accurately for various data analysis tasks, e.g., accurate recommendation, community detection, and web service selection. An SGD-based LFA model has been widely adopted to tackle this issue. However, it suffers from slow convergence that leads to considerable time cost on large-scale datasets. This study proposes an ANPS algorithm following the principle of a nonlinear PID controller. With it, an ANPS-based LFA model is achieved, which possesses fast convergence rate on an industrial HDI matrix. The proposed ANPS algorithm can be leveraged for different types of various machine learning models, thereby improving their utility and scalability in practice.

A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can obtain superior performance when performing representation to a high-dimensional and incomplete (HDI) matrix, which is encountered in various big data-related applications cause of the great demand for describing the highly complex interactions among tremendous nodes. However, an SGD-based LFA model is often stacked by slow convergence since a standard SGD algorithm updates a single latent factor depending on the stochastic gradient of current instance learning error, which disregards the past learning information. To address this critical issue, this paper innovatively proposes a Fuzzy PID-incorporated SGD (FPS) algorithm with the following two-fold ideas: a) refining the instance learning error by modeling the past update information guided by the principle of a PID controller efficiently, and b) designing a fuzzy reasoning process to implement the gain parameter adaptation in a PID controller effectively. With it, an FPS-based LFA model is further built for fast and accurate latent factor analysis on an HDI matrix. Experiments conducted on six HDI datasets reveal that the proposed FPS-based LFA model surpasses state-of-the-art LFA models in computational efficiency and accuracy when estimating missing data within an HDI matrix.

A latent factor analysis (LFA) model can be efficiently built via the stochastic gradient descent (SGD) algorithm to address high-dimensional and incomplete (HDI) data generated by various big data-related applications. However, its performance depends hugely on its hyperparameters, which is conventionally decided through the grid-search that yields extremely high-computational costs. Particle swarm optimization (PSO)-based hyperparameter adaptation provides a potential solution to this severe problem, while it leads to accuracy loss due to the untimely convergence of the PSO algorithm. Moreover, existing PSO extensions mostly perplex the evolution scheme to make the hyperparameter adaptation time-consuming and computationally expensive. Aiming at implementing an accurate and adaptive LFA model, a pseudo gradient-adjusted PSO (PGA-PSO) algorithm with twofold ideas is proposed: 1) modeling the position transitions in a PSO algorithm as the pseudo gradients for optimizing each particle’s position and 2) incorporating the principle of adaptive moment estimation into the pseudo gradient estimation to make it consider previous pseudo gradient information for addressing the untimely issue. With it, a PGA-PSO-incorporated LFA (PPL) model is successfully constructed. The empirical studies on six HDI datasets demonstrate that owing to the efficient hyperparameter adaptation implemented by the proposed PGA-PSO algorithm, the obtained PPL model surpasses state-of-the-art LFA models in accuracy when predicting an HDI matrix’s missing data. Hence, it satisfies the scalability and efficiency demands emerging from practical applications.

Precise representation learning to an undirected weighted network (UWN) is the foundation of understanding its connection patterns and functional mechanisms. A graph convolution network (GCN) is frequently utilized to tackle this issue. However, it only considers the neighbor information in the forward propagation process, which unfortunately impairs its representation learning ability. Motivated by this discovery, this work proposes a novel Neighbor-induced Graph Convolution Network (N-GCN) that adopts two-fold ideas: a) employing the weighted forward propagation process, which aggregates the neighbor information by considering the interaction strength of node pair; b) incorporating a neighbor-regularizer into the loss function, which induces the neighbor information to illustrate a UWN's intrinsic symmetry, thereby boosting the representation learning ability. Experimental results on four UWNs validate the proposed N-GCN outperforms the state-of-the-art models in achieving a highly accurate representation of UWNs emerging from real applications.

With the rapid development of services computing in the past decade, Quality-of-Service (QoS)-aware selection of Web services has become a hot yet thorny issue. Conducting warming-up tests on a large set of candidate services for QoS evaluation is time consuming and expensive, making it vital to implement accurate QoS-estimators. Existing QoS-estimators barely consider the temporal patterns hidden in QoS data. However, such data are naturally time dependent. For addressing this critical issue, this study presents a Kalman-filter-incorporated latent factor analysis (KLFA)-based QoS-estimator for accurate representation to temporally dynamic QoS data. Its main idea is to make the user latent features (LFs) time dependent, while the service ones time consistent. A novel iterative training scheme is designed, where the user LFs are learned through a Kalman filter for precisely modeling the temporal patterns, and the service ones are alternatively trained via an alternating least squares algorithm for precisely representing the historical QoS data. Empirical studies on large-scale and real Web service QoS datasets demonstrate that the proposed KLFA model significantly outperforms state-of-the-art QoS-estimators in estimation accuracy for dynamic QoS data.

A High-dimensional and incomplete (HDI) matrix is regularly adopted to portray the inherent non-negativity of interactions among numerous nodes, which is involved in countless industrial applications driven by big data. An inherently non-negative latent factor (LF) model can take out the intrinsical features from such data conveniently and effectually due to its unimpeded training process. However, it constructs the learning objective relying on a standard Euclidean distance, thereby seriously restricting its representative ability to HDI data generated by different domains. To address this issue, this work proposes an adaptive divergence-based non-negative LF (ADNLF) model following: 1) constructing a generalized objective function based on$\alpha - \beta $-divergence to inflate its ability to represent various HDI data; 2) connecting the optimization variables with output LFs by a smooth and single LF-dependent bridging function to satisfy the non-negativity constraints constantly; and 3) facilitating adaptive divergence in the learning objective through particle swarm optimization for high scalability. Empirical studies on eight HDI matrices validate that an ADNLF model evidently outstrips state-of-the-art models in terms of estimation accuracy as well as computational efficiency for missing data of an HDI dataset.

How to extract useful knowledge from a high-dimensional and sparse (HiDS) matrix efficiently is critical for many big data-related applications. A latent factor (LF) model has been widely adopted to address this problem. It commonly relies on an iterative learning algorithm like stochastic gradient descent. However, an algorithm of this kind commonly consumes many iterations to converge, resulting in considerable time cost on large-scale datasets. How to accelerate an LF model's training process without accuracy loss becomes a vital issue. To address it, this study innovatively proposes amultilayered-and-randomizedlatentfactor (MLF) model. Its main idea is two-fold: a) adopting randomized-learning to train LFs for implementing a ‘one-iteration’ training process for saving time; and 2) adopting the principle of a generally multilayered structure as in a deep forest or multilayered extreme learning machine to structure its LFs, thereby enhancing its representative learning ability. Empirical studies on six HiDS matrices from real applications demonstrate that compared with state-of-the-art LF models, an MLF model achieves significantly higher computational efficiency with satisfactory prediction accuracy. It has the potential to handle LF analysis on a large scale HiDS matrix with real-time requirements.

To quantify user-item preferences, a recommender system (RS) commonly adopts a high-dimensional and sparse (HiDS) matrix. Such a matrix can be represented by a non-negative latent factor analysis model relying on a single latent factor (LF)-dependent, non-negative, and multiplicative update algorithm. However, existing models' representative abilities are limited due to their specialized learning objective. To address this issue, this study proposes an α- β -divergence-generalized model that enjoys fast convergence. Its ideas are three-fold: 1) generalizing its learning objective with α- β -divergence to achieve highly accurate representation of HiDS data; 2) incorporating a generalized momentum method into parameter learning for fast convergence; and 3) implementing self-adaptation of controllable hyperparameters for excellent practicability. Empirical studies on six HiDS matrices from real RSs demonstrate that compared with state-of-the-art LF models, the proposed one achieves significant accuracy and efficiency gain to estimate huge missing data in an HiDS matrix.

High-dimensional and sparse (HiDS) matrices are frequently found in various industrial applications. A latent factor analysis (LFA) model is commonly adopted to extract useful knowledge from an HiDS matrix, whose parameter training mostly relies on a stochastic gradient descent (SGD) algorithm. However, an SGD-based LFA model's learning rate is hard to tune in real applications, making it vital to implement its self-adaptation. To address this critical issue, this study firstly investigates the evolution process of a particle swarm optimization algorithm with care, and then proposes to incorporate more dynamic information into it for avoiding accuracy loss caused by premature convergence without extra computation burden, thereby innovatively achieving a novel position-transitional particle swarm optimization (P2SO) algorithm. It is subsequently adopted to implement a P2SO-based LFA (PLFA) model that builds a learning rate swarm applied to the same group of LFs. Thus, a PLFA model implements highly efficient learning rate adaptation as well as represents an HiDS matrix precisely. Experimental results on four HiDS matrices emerging from real applications demonstrate that compared with an SGD-based LFA model, a PLFA model no longer suffers from a tedious and expensive tuning process of its learning rate, and it can achieve even higher prediction accuracy for missing data of an HiDS matrix. On the other hand, compared with state-of-the-art adaptive LFA models, a PLFA model's prediction accuracy and computational efficiency are highly competitive. Hence, it has high potential in addressing real industrial issues.

Non-negative latent factor (NLF) models well represent high-dimensional and sparse (HiDS) matrices filled with non-negative data, which are frequently encountered in industrial applications like recommender systems. However, current NLF models mostly adopt Euclidean distance in their objective function, which represents a special case of a β-divergence function. Hence, it is highly desired to design a β-divergence-based NLF ( β-NLF) model that uses a β-divergence function, and investigate its performance in recommender systems as β varies. To do so, we first model β-NLF's learning objective with a β-divergence function. Subsequently, we deduce a general single latent factor-dependent, non-negative and multiplicative update scheme for β-NLF, and then design an efficient β-NLF algorithm. The experimental results on HiDS matrices from industrial applications indicate that by carefully choosing the value of β, β-NLF outperforms an NLF model with Euclidean distance in terms of accuracy for missing data prediction without increasing computational time. The research outcomes show the necessity of using an optimal β-divergence function in order to achieve the best performance of an NLF model on HiDS matrices. Hence, the proposed model has both theoretical and application significance.

With the rapid development of services computing in the past decade, automatic selection of QoS-aware Web service out from numerous candidates with similar functions is becoming a hot yet thorny issue. Conducting warming up tests on numerous candidate services for quality evaluation is extremely time-consuming and expensive, making it vital to generate highly accurate predictions for missing QoS data based on known ones. Since QoS data are time-dependent, it is vital to consider the temporal dynamic patterns hidden in historical ones when building a QoS-estimator. For addressing this issue, this study invents a Kalman filter-incorporated Latent Factor Analysis (KLFA)-based QoS-estimator, which precisely models the temporal patterns hidden in dynamic QoS data. Experimental results based on large-scale and real-world Web service QoS data demonstrate that compared with state-of-the-art temporal-aware QoS-estimators, a KLFA-based one achieves significantly higher prediction accuracy for missing QoS data.

Recommender system (RS) commonly describes its user-item preferences with a high-dimensional and sparse (HiDS) matrix. A latent factor (LF) model relying on stochastic gradient descent (SGD) is frequently adopted to extract useful information from such an HiDS matrix. In spite of its efficiency, an SGD-based LF model commonly takes many iterations to converge. When processing a large-scale HiDS matrix, its computational efficiency should be further improved by further accelerating its convergence rate as well as maintaining its learning ability. To address this issue, this paper innovatively proposes novel SGD algorithm which incorporates a nonlinear proportional integral derivative (NPID) controller into its learning scheme for building an LF model. The main idea is to adopt an NPID controller to model the learning residual achieved in the past iterations, thereby adjusting the learning direction and step size of the current iteration, thereby making a resultant model converge fast. With the NPID-incorporated SGD algorithm, this study proposes an NPID-SGD-based LF (NSLF) model. Experimental results on two HiDS matrices demonstrate that compared with a standard SGD-based LF model, the proposed model achieves higher computational efficiency and prediction accuracy for missing data of an HiDS matrix.

Recommender systems (RSs) commonly describe its user-item preferences with a high-dimensional and sparse (HiDS) matrix filled with non-negative data. A non-negative latent factor (NLF) model relying on a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is frequently adopted to process such an HiDS matrix. However, an NLF model mostly adopts Euclidean distance for its objective function, which is naturally a special case of α-β-divergence. Moreover, it frequently suffers slow convergence. For addressing these issues, this study proposes a generalized and fast-converging non-negative latent factor (GFNLF) model. Its main idea is two-fold: a) adopting α-β-divergence for its objective function, thereby enhancing its representation ability for HiDS data; b) deducing its momentum-incorporated non-negative multiplicative update (MNMU) algorithm, thereby achieving its fast convergence. Empirical studies on two HiDS matrices emerging from real RSs demonstrate that with carefully-tuned hyperparameters, a GFNLF model outperforms state-of-the-art models in both computational efficiency and prediction accuracy for missing data of an HiDS matrix.

Self-labeled technique, a paradigm of semisupervised classification (SSC), is highly effective in alleviating the shortage of labeled data in classification tasks via an iterative self-labeling process. Although existing self-labeled SSC models show great prospect in industrial applications, they suffer from performance degeneration caused by false-positive label-predictions of unlabeled data during the iterative self-labeling process. For addressing this issue, this paper proposes a novel SSC framework, which is highly compatible with most existing self-labeled SSC models. The main idea of this framework is to incorporate a differential-evolution-based positioning optimization algorithm for classification into the iterative self-labeling process, aiming at optimizing the positioning of newly labeled data. Specifically, five representative self-labeled SSC models with different characteristics are modified based on the proposed framework to check their performances. Experimental results on 45 benchmark datasets demonstrate that the proposed framework is highly compatible with tested self-labeled SSC models, and significantly effective in improving their performances.