Current vision-language models (VLMs) understand complex vision-text tasks by extracting overall semantic information from largescale cross-modal associations.However, extracting from large-scale cross-modal associations often smooths out semantic details and requires large computations, limiting multimodal fine-grained understanding performance and efficiency.To address this issue, this paper proposes a detail-oriented prompt learning (DoPL) method for vision-language models to implement fine-grained multi-modal semantic alignment with merely 0.25M trainable parameters.According to the low-entropy information concentration theory, DoPL explores shared interest tokens from text-vision correlations and transforms them into alignment weights to enhance text prompt and vision prompt via detail-oriented prompt generation.It effectively guides the current frozen layer to extract fine-grained text-vision alignment cues.Furthermore, DoPL constructs detail-oriented prompt generation for each frozen layer to implement layer-by-layer localization of finegrained semantic alignment, achieving precise understanding in complex vision-text tasks.DoPL performs well in parameter-efficient finegrained semantic alignment with only 0.12% tunable parameters for vision-language models.The state-of-the-art results over the previous parameter-efficient fine-tuning methods and full fine-tuning approaches on six benchmarks demonstrate the effectiveness and efficiency of DoPL in complex multi-modal tasks.

Low-rank adaptation (LoRA) efficiently adapts LLMs to downstream tasks by decomposing LLMs' weight update into trainable low-rank matrices for fine-tuning.However, the random low-rank matrices may introduce massive taskirrelevant information, while their recomposed form suffers from limited representation spaces under low-rank operations.Such dense and choked adaptation in LoRA impairs the adaptation performance of LLMs on downstream tasks.To address these challenges, this paper proposes OHoRA, an orthogonal high-rank adaptation for parameter-efficient fine-tuning on LLMs.According to the information bottleneck theory, OHoRA decomposes LLMs' pre-trained weight matrices into orthogonal basis vectors via QR decomposition and splits them into two low-redundancy high-rank components to suppress task-irrelevant information.It then performs dynamic rank-elevated recomposition through Kronecker product to generate expansive task-tailored representation spaces, enabling precise LLM adaptation and enhanced generalization.OHoRA effectively operationalizes the information bottleneck theory to decompose LLMs' weight matrices into low-redundancy high-rank components and recompose them in rank-elevated manner for more task-tailored representation spaces and precise LLM adaptation.Empirical evaluation shows OHoRA's effectiveness by outperforming LoRA and its variants and achieving comparable performance to full fine-tuning with only 0.0371% trainable parameters.

In this work, we focus on enhancing the realism of face age transformation. Previous methods often relied on style transfer strategies or text-attention manipulation, which frequently result in undesirable artifacts or distorted facial defects. We propose DMDM, an age transformation method by Dual-Modal collaborative attention using Diffusion Models. Specifically, we introduce a collaborative text-image attention based editing method that balances age semantic control and visual harmony of generated face. To further refine fidelity, we propose the Softer Image Attention Injection Mechanism, which dynamically integrates image guidance. Additionally, we design a query image set to retrieve relevant images accroding to source face and other attributes for image guidance. Finally, Age-aware Face Restoration module is proposed to enhance high-frequency details according to age through a cascaded refinement pipeline. Extensive experiment demonstrates that DMDM achieves state-of-the-art performance, especially in visual quality.

Data-Knowledge dual-driven fashion can enhance model performance by complementing data-driven basis with expert knowledge. However, cutting-edge works in speech emotion recognition (SER) primarily evolve in data-driven training, failing to incorporate prior knowledge to form a closed loop and posing an obstacle to capture task-specific details when used independently. In this paper, we propose a Dual-Drive Broad Attentive Network (DiBAN) to achieve comprehensive emotional learning for SER. Specifically, the Dual-Drive Emotional Modeling module incorporates handcrafted extractor, pre-trained model and tailored base models, to conduct integral emotional modeling. Subsequently, the Multi-Model Attention-Aware Learning module is designed to refine the data-knowledge emotional disparities based on the attention-enhanced entropy loss. Finally, the Broad Adaptive Decision Fusion module performs adaptive fusion of emotional decisions from different drives. Extensive experiments on seven SER corpora demonstrate that DiBAN achieves significant improvements over the base models and outperforms comparative methods, fully showcasing its superiority.

The dynamic neural network (DNN), in contrast to the static counterpart, offers numerous advantages, such as improved accuracy, efficiency, and interpretability. These benefits stem from the network's flexible structures and parameters, making it highly attractive and applicable across various domains. As the broad learning system (BLS) continues to evolve, DNNs have expanded beyond deep learning (DL), orienting a more comprehensive range of domains. Therefore, this comprehensive review article focuses on two prominent areas where DNN structures have rapidly developed: 1) DL and 2) broad learning. This article provides an in-depth exploration of the techniques related to dynamic construction and inference. Furthermore, it discusses the applications of DNNs in diverse domains while also addressing open issues and highlighting promising research directions. By offering a comprehensive understanding of DNNs, this article serves as a valuable resource for researchers, guiding them toward future investigations.

The broad learning system (BLS) has recently been applied in numerous fields. However, it is mainly a supervised learning system and thus not suitable for specific practical applications with a mixture of labeled and unlabeled data. Despite a manifold regularization-based semi-supervised BLS, its performance still requires improvement, because its assumption is not always applicable. Therefore, this article proposes an incremental-self-training-guided semi-supervised BLS (ISTSS-BLS). Distinctive to traditional self-training, where all unlabeled data are labeled simultaneously, incremental self-training (IST) obtains unlabeled data incrementally from an established sorted list based on the distance between the data and their cluster center. During iterative learning, a small portion of labeled data is first used to train BLS. The system recursively self-updates its structure and meta-parameters using: 1) the double-restricted mechanism and 2) the dynamic neuron-incremental mechanism. The double-restricted mechanism is beneficial to preventing the introduction of incorrect pseudo-labeled samples, and the dynamic neuron-incremental mechanism guides the self-updating of the network structure effectively based on the training accuracy of the labeled data. These strategies guarantee a parsimonious model during the update. Besides, a novel metric, the accuracy-time ratio (A/T), is proposed to evaluate the model's performance comprehensively regarding time and accuracy. In experimental verifications, ISTSS-BLS performs outstandingly on 11 datasets. Specifically, the IST is compared with the traditional one on three scales data, saving up to 52.02% learning time. In addition, ISTSS-BLS is compared with different state-of-the-art alternatives, and all results indicate that it possesses significant advantages in performance.

This study investigates the feasibility of incorporating the broad learning (BL) model in federated learning. Traditional deep learning-based federated learning encounters challenges such as excessive communication volume and extended training time. To address these issues, federated learning with the broad learning model has attracted much attention. We provide a detailed discussion on server-side aggregation for BL models, including the initialization process and three feasible aggregation approaches for single-round aggregation. Then, ablation studies are conducted to assess the suitable BL model architectures within the context of federated learning. Furthermore, we perform a comparative analysis between these approaches and existing federated learning schemes to evaluate their advantages and limitations. Our research suggests that federated learning with BL models is highly feasible in certain scenarios and can effectively tackle the issues of transmission efficiency. Finally, we analyze our findings and propose further investigation to improve algorithms for more efficient performance of federated broad learning systems in the future.

Kernel clustering has the ability to get the inherent nonlinear structure of the data. But the high computational complexity and the unknown representation of the kernel space make it unavailable for the data clustering in distributed peer-to-peer (P2P) networks. To solve this issue, we propose a new series of random feature-based collaborative kernel clustering algorithms in this article. In the most basic algorithm, each node in a distributed P2P network first maps its data into a low-dimensional random feature space with the approximation of the given kernel by using the random Fourier feature mapping method. Then, each node independently searches the clusters with its local data and the collaborative knowledge from its neighbor nodes, and the distributed clustering is performed among all network nodes until reaching the global consensus result, i.e., all nodes have the same cluster centers. In addition, an improved version is designed with assignment of feature weights, which is optimized by the maximum-entropy technique to extract important features for the cluster identification. What’s more, to relief the impact of different kernel functions and related parameters on clustering results, the combination of multiple kernels rather than a single kernel is adopted for the low-dimensional approximation, and the optimized weights are assigned to provide the guidance on the choice of the kernels and their parameters and discover significant features at the same time. Experiments on synthetic and real-world datasets show that the proposed methods achieve similar and even better results than the traditional kernel clustering methods on various performance metrics, including the average classification rate, the average normalized mutual information, and the average adjusted rand index. More importantly, the low-dimensional random features approximated to kernels and the distributed clustering mechanism adopted in these methods bring the greatly lower temporal complexity.

It is hard to construct an optimal classifier for high-dimensional imbalanced data, on which the performance of classifiers is seriously affected and becomes poor. Although many approaches, such as resampling, cost-sensitive, and ensemble learning methods, have been proposed to deal with the skewed data, they are constrained by high-dimensional data with noise and redundancy. In this study, we propose an adaptive subspace optimization ensemble method (ASOEM) for high-dimensional imbalanced data classification to overcome the above limitations. To construct accurate and diverse base classifiers, a novel adaptive subspace optimization (ASO) method based on adaptive subspace generation (ASG) process and rotated subspace optimization (RSO) process is designed to generate multiple robust and discriminative subspaces. Then a resampling scheme is applied on the optimized subspace to build a class-balanced data for each base classifier. To verify the effectiveness, our ASOEM is implemented based on different resampling strategies on 24 real-world high-dimensional imbalanced datasets. Experimental results demonstrate that our proposed methods outperform other mainstream imbalance learning approaches and classifier ensemble methods.

In recent years, the appearance of the broad learning system (BLS) is poised to revolutionize conventional artificial intelligence methods. It represents a step toward building more efficient and effective machine-learning methods that can be extended to a broader range of necessary research fields. In this survey, we provide a comprehensive overview of the BLS in data mining and neural networks for the first time, focusing on summarizing various BLS methods from the aspects of its algorithms, theories, applications, and future open research questions. First, we introduce the basic pattern of BLS manifestation, the universal approximation capability, and essence from the theoretical perspective. Furthermore, we focus on BLS's various improvements based on the current state of the theoretical research, which further improves its flexibility, stability, and accuracy under general or specific conditions, including classification, regression, semisupervised, and unsupervised tasks. Due to its remarkable efficiency, impressive generalization performance, and easy extendibility, BLS has been applied in different domains. Next, we illustrate BLS's practical advances, such as computer vision, biomedical engineering, control, and natural language processing. Finally, the future open research problems and promising directions for BLSs are pointed out.

The broad learning system (BLS) is an algorithm that facilitates feature representation learning and data classification. Although weights of BLS are obtained by analytical computation, which brings better generalization and higher efficiency, BLS suffers from two drawbacks: 1) the performance depends on the number of hidden nodes, which requires manual tuning, and 2) double random mappings bring about the uncertainty, which leads to poor resistance to noise data, as well as unpredictable effects on performance. To address these issues, a kernel-based BLS (KBLS) method is proposed by projecting feature nodes obtained from the first random mapping into kernel space. This manipulation reduces the uncertainty, which contributes to performance improvements with the fixed number of hidden nodes, and indicates that manually tuning is no longer needed. Moreover, to further improve the stability and noise resistance of KBLS, a progressive ensemble framework is proposed, in which the residual of the previous base classifiers is used to train the following base classifier. We conduct comparative experiments against the existing state-of-the-art hierarchical learning methods on multiple noisy real-world datasets. The experimental results indicate our approaches achieve the best or at least comparable performance in terms of accuracy.

Cement has been widely used in civil engineering, whose quality directly affects the safety of buildings. Cement compressive strength, as an important quality indicator, its accurate estimation is of great significance in quality inspections and the design of high-performance products. However, existing measurement technology remains traditional and destructive. Except for high time-consuming and the waste of various resources, it requires significant improvement since the unprofessional operations will give rise to large errors. In this article, an efficient system is proposed to estimate the cement compressive strength based on the broad learning and internal factors, in which the index system describes the internal factors affecting the compressive strength, and the broad learning system distills the potential correlation between the compressive strength and those factors. It can nondestructively estimate the strength directly with the internal factors, e.g., clinker composition and physical properties. In addition, to verify its practicability and to assist the formula optimization in the application, the robustness test and factorial analysis are designed. The experimental results prove that this model can accurately estimate the strength with excellent generalization ability, which saves labor power and material, avoids large errors caused by unprofessional operations, and aids high-performance cement production. Especially, its ability to rapidly build an accurate estimation model is beneficial for the production of various cement in industry.

The newly proposed Broad Learning System (BLS) offers an alternative way to deep learning which saves a time-consuming training process and powerful computing resources. However, the randomly generated feature nodes and lots of enhancement nodes in BLS may reduce the performance of the final classifier. Aiming at the problems in randomly generated feature nodes which suffer from unpredictability and need guidance, this paper proposes an adaptive feature nodes evolutionary algorithm (AFNE) to extract better features; While broad learning neural network often requires a large number of enhancement nodes parameters to ensure its performance, which easily leads to the redundancy or dependency between features, as well as the performance degradation of the final model. Therefore, this article also proposes a new criterion based on node sensitivity to prune the network enhancement layer nodes to remove redundant nodes, reduce the network scale, and increase the generalization ability. The proposed algorithm ADP-BLS can improve the accuracy and generalization performance of the final classifier through the evolution of feature nodes and the pruning of enhancement nodes. Extensive comparative experiments on real world data sets verify the effectiveness of the proposed ADP-BLS. At the same time, when verifying the module validity of the innovative algorithms added in this article, experiments also show that the AFNE and pruning method integrated into BLS can improve the model to a certain extent.

The fuzzy broad learning system (FBLS) is a recently proposed neuro-fuzzy model that shares the similar structure of a broad learning system (BLS). It shows high accuracy in both classification and regression tasks and inherits the fast computational nature of a BLS. However, the ensemble of several fuzzy subsystems in an FBLS decreases the possibility of understanding the fuzzy model since the fuzzy rules from different fuzzy systems are difficult to combine together while keeping the consistence. To balance the model accuracy and complexity, a synthetically simplified FBLS with better interpretability, named compact FBLS (CFBLS), is developed in this article, which can generate much fewer and more explainable fuzzy rules for understanding. In such a way, only one traditional Takagi–Sugeno–Kang fuzzy system is employed in the feature layer of a CFBLS, and the input universe of discourse is equally partitioned to obtain the fuzzy sets with proper linguistic labels accordingly. The random feature selection matrix and rule combination matrix are employed to reduce the total number of fuzzy rules and to avoid the “curse of dimensionality.” The enhancement layer is kept in the CFBLS that helps to add a nonlinear transformation of input features to the traditional first-order polynomial used in the consequent part of a fuzzy rule. The pseudoinverse is also used to determine the parameters of CFBLS guaranteeing its fast computational nature. The experiments on the popular UCI and KEEL datasets indicate that the CFBLS can generate a smaller set of comprehensible fuzzy rules and achieve much higher accuracy than some state-of-the-art neuro-fuzzy models. Moreover, the advantage of CFBLS is also verified in a real-world application.

The broad learning system (BLS) has been proved to be effective and efficient lately. In this article, several deep variants of BLS are reviewed, and a new adaptive incremental structure, Stacked BLS, is proposed. The proposed model is a novel incremental stacking of BLS. This invariant inherits the efficiency and effectiveness of BLS that the structure and weights of lower layers of BLS are fixed when the new blocks are added. The incremental stacking algorithm computes not only the connection weights between the newly stacking blocks but also the connection weights of the enhancement nodes within the BLS block. The Stacked BLS is considered as the increment of “layers” and “neurons” dynamically during the training for multilayer neural networks. The proposed architecture along with the training algorithms that utilizes the residual characteristic is very versatile in comparison with traditional fixed architecture. Finally, experimental results on UCI datasets, MNIST dataset, NORB dataset, CIFAR-10 dataset, SVHN dataset, and CIFAR-100 dataset indicate that the proposed method outperforms the selected state-of-the-art methods on both accuracy and training speed, such as deep residual networks. The results also imply that the proposed structure could highly reduce the number of nodes and the training time of the original BLS in the classification task of some datasets.

Automatic real-time emotion recognition based on multi-channel EEG signals is a significant and challenging task in neurology and psychiatry. In recent years, deep learning has been used in EEG emotion recognition. However, many existing deep learning based methods still require complex pre-processing or additional feature extraction, which make it difficult to achieve real-time emotion recognition. In this paper, an end-to-end model named Temporal Convolutional Broad Learning System (TCBLS) was designed for multi-channel EEG based emotion recognition. The TCBLS takes one-dimensional EEG signals as input, then extracts emotion-related features of EEG automatically. In this model, the Temporal Convolutional Network (TCN) is designed to extract EEG temporal features and deep abstract features simultaneously, then Broad Learning System (BLS) is used to map the features to a more discriminative space and further enhance the features. We evaluated our method on DEAP database, performing 10-fold cross-validation on each subject to obtain the classification accuracy. Experimental results indicate that the performance of TCBLS is better than other comparison methods, and the mean accuracy of TCBLS is 99.5755% and 99.5781% on valence and arousal classification task respectively. The results demonstrate the effectiveness and robustness of TCBLS in EEG emotion recognition.

The Broad Learning System has been proved to be effective and efficient. However, the associated feature nodes in the system are mainly based on linear mappings. Although such kind of features has been successful in various datasets and applications, more general features (especially for the nonlinear features) are necessary for specific applications. Motivated by the powerful capability of the kernel methods, a novel expansion of broad learning system based on multiple kernels is proposed in this paper. Firstly, the nonlinear feature mappings in the form of multiple kernels are merged into the feature nodes of broad learning system. After that, the resulted features are further enhanced through nonlinear activation functions. The experimental results on UCI datasets indicate that the proposed method outperforms the other methods.

After a very fast and efficient discriminative broad learning system (BLS) that takes advantage of flatted structure and incremental learning has been developed, here, a mathematical proof of the universal approximation property of BLS is provided. In addition, the framework of several BLS variants with their mathematical modeling is given. The variations include cascade, recurrent, and broad-deep combination structures. From the experimental results, the BLS and its variations outperform several exist learning algorithms on regression performance over function approximation, time series prediction, and face recognition databases. In addition, experiments on the extremely challenging data set, such as MS-Celeb-1M, are given. Compared with other convolutional networks, the effectiveness and efficiency of the variants of BLS are demonstrated.

In recent years, research on facial expression recognition (FER) has become an increasingly active research topic. Deep learning is a new area, which gives a new way to classify images of human faces into emotion categories. However, it faces many difficulties caused by poor robustness and real-time performance. This paper designs a new architecture network based on Broad Learning System (BLS) for facial expressions recognition. It is established as a flat network. The original inputs are transferred and placed as mapped features in feature nodes, while the structure is expanded in wide sense in the enhancement nodes. To evaluate our architecture we tested the proposed method with the Extended Cohn-Kanade Dataset (CK+). The experimental results show that the BLS approach is very effective in facial expression recognition to compare with convolutional neural networks.

Broad Learning System (BLS) that aims to offer an alternative way of learning in deep structure is proposed in this paper. Deep structure and learning suffer from a time-consuming training process because of a large number of connecting parameters in filters and layers. Moreover, it encounters a complete retraining process if the structure is not sufficient to model the system. The BLS is established in the form of a flat network, where the original inputs are transferred and placed as "mapped features" in feature nodes and the structure is expanded in wide sense in the "enhancement nodes." The incremental learning algorithms are developed for fast remodeling in broad expansion without a retraining process if the network deems to be expanded. Two incremental learning algorithms are given for both the increment of the feature nodes (or filters in deep structure) and the increment of the enhancement nodes. The designed model and algorithms are very versatile for selecting a model rapidly. In addition, another incremental learning is developed for a system that has been modeled encounters a new incoming input. Specifically, the system can be remodeled in an incremental way without the entire retraining from the beginning. Satisfactory result for model reduction using singular value decomposition is conducted to simplify the final structure. Compared with existing deep neural networks, experimental results on the Modified National Institute of Standards and Technology database and NYU NORB object recognition dataset benchmark data demonstrate the effectiveness of the proposed BLS.