Community detection sheds light on various graph mining tasks such as social recommendation, which is becoming a long-standing issue in the realm of complex network analysis. A non-negative matrix factorization (NMF) is frequently used to tackle this task. However, it builds based on the principle of linear representation and has difficulty in capturing non-linear features from irregularly non-Euclidean data. To address this issue, this study boosts an NMF-based community detector by combining with a graph convolution module, and a novel Graph Convolution and Graph-Laplacian bi-regularized, Symmetric non-negative matrix factorization (GCGS) model is proposed relying on two main ideas: a) taking a graph convolution network (GCN) as a non-linear constraint module on the feature matrix to ensure its non-linearity; and b) adopting graph regularization to preserve the local geometric features of the network topology. A non-negative and multiplicative update (NMU) algorithm is then derived to solve the unified objective function. Extensive experimental results on six real networks indicate that GCGS achieves higher precision in community detection than its peers.

Community is a fundamental and highly desired pattern in a Large-scale Undirected Network (LUN). Community detection is a vital issue when LUN representation learning is performed. Owing to its good scalability and interpretability, a Symmetric and Non-negative Matrix Factorization model is frequently utilized to tackle this issue. It adopts a unique Latent Factor (LF) matrix for precisely representing LUN’s symmetry, which, unfortunately, leads to a reduced LF space that decreases its representation learning ability to a target LUN. Motivated by this discovery, this study proposes a Symmetry and Graph Bi-regularized Non-negative Matrix Factorization (B-NMF) method that: a) leverages multiple LF matrices when representing LUN, thereby boosting the representation learning ability; b) constructs a symmetry regularization term that implies the equality constraint among its multiple LF matrices, thereby illustrating LUN’s intrinsic symmetry; and c) incorporates graph regularization into its learning objective, thereby illustrating LUN’s local geometry. A theoretical proof is given to theoretically validate B-NMF’s convergence ability. The regularization hyperparameters are selected by validating model modularity, thereby guaranteeing B-NMF’s practicability in addressing real application issues. Extensive experimental results on ten LUNs from real applications demonstrate that the proposed B-NMF-based community detector significantly outperforms several baseline and state-of-the-art models in achieving highly-accurate community detection results. Note to Practitioners—LUNs are very-commonly seen in real applications like a social network system. Communities in LUNs are vital for various knowledge discovery-related applications. For accurately detecting them, a detector should guarantee its high representation learning ability to a target LUN. To do so, this paper presents a B-NMF model that is able to perform precise representation learning to LUNs, thereby achieving accurate community detection results. In comparison with conventional Symmetric and Non-negative Matrix Factorization-based community detectors, a B-NMF-based community detector enjoys its enlarged latent feature space, which ensures its higher representation ability to a target LUN. It depends on two regularization hyperparameters, which can be selected by performing grid-search on the target LUN via its modularity evaluation. This paper gives the empirical values of B-NMF’s regularization hyperparameters based on the parametersensitivity tests on the involved experimental datasets. The proposed B-NMF model is shown to be highly suitable for addressing community detection and clustering tasks on LUNs from real applications.

A fast non-negative latent factor (FNLF) model for a high-dimensional and sparse (HiDS) matrix adopts a Single Latent Factor-dependent, Non-negative, Multiplicative and Momentum-incorporated Update (SLF-NM2U) algorithm, which enables its fast convergence. It is crucial to achieve a rigorously theoretical proof regarding its fast convergence, which has not been provided in prior research. Aiming at addressing this critical issue, this work theoretically proves that with an appropriately chosen momentum coefficient, SLF-NM2U enables the fast convergence of an FNLF model in both continuous and discrete time cases. Empirical analysis of HiDS matrices generated by representative industrial applications provides empirical evidences for the theoretical proof. Hence, this study represents an important milestone in the field of HiDS matrix analysis.

Community detection is a popular yet thorny issue in social network analysis. A symmetric and nonnegative matrix factorization (SNMF) model based on a nonnegative multiplicative update (NMU) scheme is frequently adopted to address it. Current research mainly focuses on integrating additional information into it without considering the effects of a learning scheme. This study aims to implement highly accurate community detectors via the connections between an SNMF-based community detector's detection accuracy and an NMU scheme's scaling factor. The main idea is to adjust such scaling factor via a linear or nonlinear strategy, thereby innovatively implementing several scaling-factor-adjusted NMU schemes. They are applied to SNMF and graph-regularized SNMF models to achieve four novel SNMF-based community detectors. Theoretical studies indicate that with the proposed schemes and proper hyperparameter settings, each model can: 1) keep its loss function nonincreasing during its training process and 2) converge to a stationary point. Empirical studies on eight social networks show that they achieve significant accuracy gain in community detection over the state-of-the-art community detectors.

A non-negative latent factor (NLF) model with a single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is frequently adopted to extract useful knowledge from non-negative data represented by high-dimensional and sparse (HiDS) matrices arising from various service-oriented applications. However, its convergence rate is slow. To address this issue, this study proposes aGeneralized Nesterov's acceleration-incorporated,Non-negative andAdaptiveLatentFactor (GNALF) model. It results from a) incorporating a generalized Nesterov's accelerated gradient (NAG) method into an SLF-NMU algorithm, thereby achieving anNAG-incorporated andelement-orientednon-negative (NEN) algorithm to perform efficient parameter update; and b) making its regularization and acceleration parameters self-adaptive via incorporating the principle of a particle swarm optimization algorithm into the training process, thereby implementing a highly adaptive and practical model. Empirical studies on six large sparse matrices from different recommendation service applications show that a GNALF model achieves very high convergence rate without the need of hyper-parameter tuning, making its computational efficiency significantly higher than state-of-the-art models. Meanwhile, such efficiency gain does not result in accuracy loss, since its prediction accuracy is comparable with its peers. Hence, it can better serve practical service applications with real-time demands.

A nonnegative latent factorization of tensors (NLFT) model precisely represents the temporal patterns hidden in multichannel data emerging from various applications. It often adopts a single latent factor-dependent, nonnegative and multiplicative update on tensor (SLF-NMUT) algorithm. However, learning depth in this algorithm is not adjustable, resulting in frequent training fluctuation or poor model convergence caused by overshooting. To address this issue, this study carefully investigates the connections between the performance of an NLFT model and its learning depth via SLF-NMUT to present a joint learning-depth-adjusting scheme for it. Based on this scheme, a Depth-adjusted Multiplicative Update on tensor algorithm is innovatively proposed, thereby achieving a novel depth-adjusted nonnegative latent-factorization-of-tensors (DNL) model. Empirical studies on two industrial data sets demonstrate that compared with the state-of-the-art NLFT models, a DNL model achieves significant accuracy gain when performing missing data estimation on a high-dimensional and incomplete tensor with high efficiency.Note to Practitioners—Multichannel data are often encountered in various big-data-related applications. It is vital for a data analyzer to correctly capture the temporal patterns hidden in them for efficient knowledge acquisition and representation. This article focuses on analyzing temporal QoS data, which is a representative kind of multichannel data. To correctly extract their temporal patterns, an analyzer should correctly describe their nonnegativity. Such a purpose can be achieved by building a nonnegative latent factorization of tensors (NLFT) model relying on a single latent factor-dependent, nonnegative and multiplicative update on tensor (SLF-NMUT) algorithm. But its learning depth is not adjustable, making an NLFT model frequently suffer from severe fluctuations in its training error or even fail to converge. To address this issue, this study carefully investigates the learning rules for an NLFT model’s decision parameters using an SLF-NMUT and proposes a joint learning-depth-adjusting scheme. This scheme manipulates the multiplicative terms in SLF-NMUT-based learning rules linearly and exponentially, thereby making the learning depth adjustable. Based on it, this study builds a novel depth-adjusted nonnegative latent-factorization-of-tensors (DNL) model. Compared with the existing NLFT models, a DNL model better represents multichannel data. It meets industrial needs well and can be used to achieve high performance in data analysis tasks like temporal-aware missing data estimation

Non-negativity is vital for a latent factor (LF)-based model to preserve the important feature of a high-dimensional and sparse (HiDS) matrix in recommender systems, i.e., none of its entries is negative. Current non-negative models rely on constraints-combined training schemes. However, they lack flexibility, scalability, or compatibility with general training schemes. This work aims to perform unconstrained non-negative latent factor analysis (UNLFA) on HiDS matrices. To do so, we innovatively transfer the non-negativity constraints from the decision parameters to the output LFs, and connect them through a single-element-dependent mapping function. Then we theoretically prove that by making a mapping function fulfill specific conditions, the resultant model is able to represent the original one precisely. We subsequently design highly efficient UNLFA algorithms for recommender systems. Experimental results on four industrial-size HiDS matrices demonstrate that compared with four state-of-the-art non-negative models, a UNLFA-based model obtains advantage in prediction accuracy for missing data and computational efficiency. Moreover, such high performance is achieved through its unconstrained training process which is compatible with various general training schemes, on the premise of fulfilling non-negativity constraints. Hence, UNLFA algorithms are highly valuable for industrial applications with the need of performing non-negative latent factor analysis on HiDS matrices.

A single latent factor (LF)-dependent, nonnegative, and multiplicative update (SLF-NMU) learning algorithm is highly efficient in building a nonnegative LF (NLF) model defined on a high-dimensional and sparse (HiDS) matrix. However, convergence characteristics of such NLF models are never justified in theory. To address this issue, this study conducts rigorous convergence analysis for an SLF-NMU-based NLF model. The main idea is twofold: 1) proving that its learning objective keeps nonincreasing with its SLF-NMU-based learning rules via constructing specific auxiliary functions; and 2) proving that it converges to a stable equilibrium point with its SLF-NMU-based learning rules via analyzing the Karush-Kuhn-Tucker (KKT) conditions of its learning objective. Experimental results on ten HiDS matrices from real applications provide numerical evidence that indicates the correctness of the achieved proof.

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. A momentum method is frequently adopted to accelerate a learning algorithm, but it is incompatible with those implicitly adopting gradients like SLF-NMU. To build a fast NLF (FNLF) model, we propose a generalized momentum method compatible with SLF-NMU. With it, we further propose a single latent factor-dependent non-negative, multiplicative and momentum-incorporated update algorithm, thereby achieving an FNLF model. Empirical studies on six HiDS matrices from industrial application indicate that an FNLF model outperforms an NLF model in terms of both convergence rate and prediction accuracy for missing data. Hence, compared with an NLF model, an FNLF model is more practical in industrial applications.

Non-negative latent factor (NLF) models well represent high-dimensional and sparse (HiDS) matrices filled with non-negative data, which are frequently encountered in industrial applications like recommender systems. However, current NLF models mostly adopt Euclidean distance in their objective function, which represents a special case of a β-divergence function. Hence, it is highly desired to design a β-divergence-based NLF ( β-NLF) model that uses a β-divergence function, and investigate its performance in recommender systems as β varies. To do so, we first model β-NLF's learning objective with a β-divergence function. Subsequently, we deduce a general single latent factor-dependent, non-negative and multiplicative update scheme for β-NLF, and then design an efficient β-NLF algorithm. The experimental results on HiDS matrices from industrial applications indicate that by carefully choosing the value of β, β-NLF outperforms an NLF model with Euclidean distance in terms of accuracy for missing data prediction without increasing computational time. The research outcomes show the necessity of using an optimal β-divergence function in order to achieve the best performance of an NLF model on HiDS matrices. Hence, the proposed model has both theoretical and application significance.

Non-negative latent factor (NLF) models have been frequently applied to information extraction, pattern recognition, and community detection. An NLF model can well represent a high-dimensional and sparse (HiDS) matrix of non-negative data and efficiently acquire useful knowledge from it. A single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) algorithm is highly efficient to build NLF model. However, its convergence ability on such a matrix is still unveiled in theory. This paper presents the convergence property of an SLF-NMU algorithm. We theoretically prove that it can guarantee its convergence to a Karush-Kuhn-Tucher (KKT) stationary point. Empirical studies on two HiDS matrices from practical applications indicate that an SLF-NMU algorithm make an NLF model to converge at a relatively steady state.

A fast non-negative latent factor (FNLF) model adopts a single latent factor-dependent, non-negative, multiplicative and momentum-incorporated update (SLF-NM2U) algorithm, which can ensure fast convergence on a high-dimensional and sparse (HiDS) matrix according to empirical studies in prior researches. However, it is crucial to investigate the theoretical proof regarding the reason why incorporation of a generalized momentum method into an SLF-NM2U algorithm can ensure the fast convergence of an FNLF model, which has not been addressed in previous work. Therefore, this paper aims to unveil how a generalized momentum method improves the convergence rate of an FNLF model in the discrete time case by combining physical analysis. The FNLF model is superior to the NLF model in terms of the convergence rate and the prediction accuracy of missing data. This conclusion is obtained by empirically research on the HiDS matrix in industrial applications, which also provides empirical basis for theoretical proof.

Non-negative latent factor (NLF) models can efficiently acquire useful knowledge from high-dimensional and sparse (HiDS) matrices filled with non-negative data. Single latent factor-dependent, non-negative and multiplicative update (SLF-NMU) is an efficient algorithm for building an NLF model on an HiDS matrix, yet it suffers slow convergence. On the other hand, a momentum method is frequently adopted to accelerate a learning algorithm explicitly depending on gradients, yet it is incompatible with learning algorithms implicitly depending on gradients, like SLF-NMU. To build a fast NLF model, we firstly propose a generalized momentum method compatible with SLF-NMU. With it, we propose the single latent factor-dependent, non-negative, multiplicative and momentum-integrated update (SLF-NM2U) algorithm for accelerating the building process of an NLF model, thereby achieving a fast non-negative latent factor (FNLF) model. Empirical studies on six HiDS matrices from industrial application indicate that with the incorporated momentum effects, FNLF outperforms NLF in terms of both convergence rate and prediction accuracy for missing data. Hence, compare with an NLF model, an FNLF model is more practical in industrial applications.