The profitability and survival of ship-owning companies in the global shipping market are deeply intertwined with accurate forecasts of ship prices and charter rates. Effective detection of market shortfalls and capitalization on temporal arbitrage opportunities are essential for maintaining a competitive edge. Traditional forecasting models, while adept at handling various multivariate time series tasks, predominantly focus on embedding synchronous time lags, often neglecting asynchronous dependencies. This paper introduces the Shipping Transformer (SFormer), a novel forecasting model designed to address this gap by integrating a discontinuous and parsimonious embedding strategy. This approach effectively captures lead-lag relationships between explanatory and target series. To further enhance forecasting performance, we introduce a cross-dimension attention module that uncovers cross-series dependencies. The SFormer sets a new benchmark for accuracy in predicting twelve time series of prices and charter rates for four ship types across multiple forecasting horizons. This research marks a significant advancement in the field of ship pricing and charter rate forecasting, providing ship-owning companies with critical insights to optimize their operations and enhance their strategic decision-making processes within the engineering management framework of the shipping industry.

With the advancement of satellite communication technology, the maritime Internet of Things (IoT) has made significant progress. As a result, vast amounts of Automatic Identification System (AIS) data from global vessels are transmitted to various maritime stakeholders through Maritime IoT systems. AIS data contains a large amount of dynamic and static information that requires effective and intuitive visualization for comprehensive analysis. However, two major deficiencies challenge current visualization models: a lack of consideration for interactions between distant pixels and low efficiency. To address these issues, we developed a large-scale vessel trajectories visualization algorithm, called the Non-local Kernel Density Estimation (NLKDE) algorithm, which incorporates a non-local convolution process. It accurately calculates the density distribution of vessel trajectories by considering correlations between distant pixels. Additionally, we implemented the NLKDE algorithm under a Graphics Processing Unit (GPU) framework to enable parallel computing and improve operational efficiency. Comprehensive experiments using multiple vessel trajectory datasets show that the NLKDE algorithm excels in vessel trajectory density visualization tasks, and the GPU-accelerated framework significantly shortens the execution time to achieve real-time results. From both theoretical and practical perspectives, GPU-accelerated NLKDE provides technical support for real-time monitoring of vessel dynamics in complex water areas and contributes to constructing maritime intelligent transportation systems. The code for this paper can be accessed at: https://github.com/maohliang/GPU-NLKDE.

The deep random vector functional link (dRVFL) and ensemble dRVFL (edRVFL) succeed in various tasks and achieve state-of-the-art performance compared with other randomized neural networks (NNs). However, existing edRVFL structures need more diversity and error correction ability in an independent network. Our work fills the gap by combining stacked deep blocks and residual learning with the edRVFL. Subsequently, we propose a novel dRVFL combined with residual learning, ResdRVFL, whose deep layers calibrate the wrong estimations from shallow layers. Additionally, we propose incorporating a scaling parameter to control the scaling of residuals from shallow layers, thus mitigating the risk of overfitting. Finally, we present an ensemble deep stacking network, SResdRVFL, based on ResdRVFL. SResdRVFL aggregates multiple blocks into a cohesive network, leveraging the benefits of deep learning and ensemble learning. We evaluate the proposed model on 28 datasets and compare it with the state-of-the-art methods. The comparative study demonstrates that the SResdRVFL is the best-performing approach in terms of average ranking and errors based on 28 datasets.

Aircraft engines operate under more demanding and unique environments, which require the inner components to be able to withstand extreme conditions. Intershaft bearings serve as the critical part of power transmission. Therefore, their accurate and reliable fault diagnosis is of paramount importance to ensure secure and dependable functioning of the engine. In this field, scarcity of labeled fault data owing to high collection costs is a common challenge. To address this, this article proposes a semi-supervised cross-domain diagnostic method for aircraft engine intershaft bearings, utilizing a virtual domain-driven approach to achieve high accuracy with limited labeled data. Specifically, a dynamics-based simulation model is developed to generate source domain data, reducing the dependency of deep learning models on experimental platforms and lowering platform construction costs. Additionally, a hyperbolic geometric metric learning strategy is designed to capture hierarchical features in high-dimensional data, which handles the correlation between different fault types and enhancing classification accuracy. Furthermore, a domain-class adversarial decoupling mechanism is developed to mitigate the domain bias, enabling the precise representation of fault modes and maximizing the utility of unlabeled virtual domain data. Using datasets from both real-world aircraft engine scenarios and public resource experiments validate the proposed method, illustrating its superior performance compared to state-of-the-art techniques on public domain benchmark datasets.

The electricity generation through wind energy is rapidly expanding, primarily due to its priorities of lower carbon emissions and sustainability. Precise wind speed forecasting is essential for renewable energy conversions as it mitigates the randomness of wind power, therefore aiding in more effective control and strategic planning for power system dispatch. However, the inherent fluctuation of wind speed challenges accurate and consistent time series forecasting. In this paper, we develop a novel parsimonious channel mixing ensemble deep random vector functional link (pcm-edRVFL) network to anticipate future wind speeds. The ensemble deep random vector functional link network (edRVFL) utilizes deep feature extraction and ensemble learning to improve forecasting performance. We refined the standard edRVFL model by incorporating a parsimonious channel mixing selection approach for input data, focusing on crucial historical observations, and strengthening the representation of each explanatory variable. We conduct extensive evaluations on four wind speed datasets using the proposed model, and the comparative experiment results demonstrate its superiority over other baseline models. Our proposed pcm-edRVFL network provides a practical approach for precise and efficient wind speed forecasting, proving to be an instrumental resource in wind energy design and operation systems.

The Random Vector Functional Link Network (RVFL) is a single-layer feed-forward network characterized by randomised weights in its hidden layers. However, the randomness can introduce detrimental neurons, potentially impairing the network’s performance. In response, this paper introduces multiple strategies to mitigate the noise from these randomised weights in RVFL networks. We first present a neuron normalization method that enhances latent space diversity and the network’s resilience to input features. Additionally, we develop improved approaches incorporating various feature selection and elimination techniques. Furthermore, Bayesian Optimization is utilized to optimize hyperparameters within a defined space. The efficacy of these methods is demonstrated through results from UCI classification tasks, highlighting the statistically superior performance of our Noise Eliminated edRVFL (NE-edRVFL) with neuron normalization.

Knowledge distillation (KD) is a conventional method in the field of deep learning that enables the transfer of dark knowledge from a teacher model to a student model, consequently improving the performance of the student model. In randomized neural networks, due to the simple topology of network architecture and the insignificant relationship between model performance and model size, KD is not able to improve model performance. In this work, we propose a self-distillation pipeline for randomized neural networks: the predictions of the network itself are regarded as the additional target, which are mixed with the weighted original target as a distillation target containing dark knowledge to supervise the training of the model. All the predictions during multi-generation self-distillation process can be integrated by a multi-teacher method. By induction, we have additionally arrived at the methods for infinite self-distillation (ISD) of randomized neural networks. We then provide relevant theoretical analysis about the self-distillation method for randomized neural networks. Furthermore, we demonstrated the effectiveness of the proposed method in practical applications on several benchmark datasets.

Active upper limb assistive robots have the potential to improve the quality of life for patients with limb disabilities and assist those who require rehabilitation. However, patients often have difficulty accepting these robots due to the lack of intuitive human-robot interaction. One of the key challenges is accurately predicting human motion intention throughout the movement trajectory. To address this issue, we propose a dynamic online ensemble deep random vector functional link (DOedRVFL) network that relies solely on data from wear-able inertial measurement units (IMU) for online joint angle prediction. The DOedRVFL employs multiple hidden layers to extract rich features from the IMU data. The random nature of these layers enables real-time applications. Additionally, we use recursive least squares to optimize each output layer's weights in real-time. Finally, we designed a dynamic ensemble module to aggregate all outputs while considering real-time performance. Comparative results demonstrate the superiority and suitability of DOedRVFL for predicting human joint angles. Furthermore, online learning and randomized feature extraction make it well-suited for real-time control of assistive robots.

This work investigated the use of an ensemble deep random vector functional link (edRVFL) network for electroencephalogram (EEG)-based driver fatigue recognition. Against the low feature learning capability of the edRVFL network from raw EEG signals, two strategies were exploited in this work. Specifically, the first one was to exploit the advantages of the feature extractor module in CNNs, i.e., use CNN features as the input of the edRVFL network. The second one was to improve the feature learning capability of the edRVFL network. An enhanced edRFVL network named FGloWD-edRVFL was proposed, in which four enhancements were implemented, including random forest-based Feature selection, Global output layer, Weighting and entropy-based Dynamic ensemble. The proposed FGloWD-edRVFL network was evaluated on the challenging cross-subject driver fatigue recognition tasks. The results indicated that the proposed model could boost the recognition performance, significantly outperforming all strong baselines. The step-wise analysis further demonstrated the effectiveness of the proposed enhancements in the edRVFL network.

Randomized neural networks (RNNs) have shown outstanding performance in many different fields. The superiority of having fewer training parameters and closed-form solutions makes them popular in small datasets analysis. However, automatically decoding raw electroencephalogram (EEG) data using RNNs is still challenging in EEG-based passive brain–computer interface (pBCI) classification tasks. Models with the high-dimension input of EEG may suffer from overfitting and the intrinsic characteristics of non-stationary, high-level noises and subject variability could limit the generation of distinctive features in the hidden layers. To address these problems in EEG-based pBCI tasks, this work proposes a spectral-ensemble deep random vector functional link (SedRVFL) network that focuses on feature learning in the frequency domain. Specifically, an unsupervised feature-refining (FR) block is proposed to improve the low feature learning capability in RNNs. Moreover, a dynamic direct link (DDL) is performed to further complement the frequency information. The proposed model has been evaluated on a self-collected dataset as well as a public driving dataset. The cross-subject classification results obtained demonstrated its effectiveness. This work offers a new solution for EEG decoding, i.e., using optimized RNNs for decoding complex raw EEG data and boosting the classification performance of EEG-based pBCI tasks.

Transformer architectures have been applied to graph-specific data such as protein structure and shopper lists, and they perform accurately on graph/node classification and prediction tasks. Researchers have proved that the attention matrix in Transformers has low-rank properties, and the self-attention plays a scoring role in the aggregation function of the Transformers. However, it can not solve the issues such as heterophily and over-smoothing. The low-rank properties and the limitations of Transformers inspire this work to propose a Global Representation (GR) based attention mechanism to alleviate the two heterophily and over-smoothing issues. First, this GR-based model integrates geometric information of the nodes of interest that conveys the structural properties of the graph. Unlike a typical Transformer where a node feature forms a Key, we propose to use GR to construct the Key, which discovers the relation between the nodes and the structural representation of the graph. Next, we present various compositions of GR emanating from nodes of interest and α-hop neighbors. Then, we explore this attention property with an extensive experimental test to assess the performance and the possible direction of improvements for future works. Additionally, we provide mathematical proof showing the efficient feature update in our proposed method. Finally, we verify and validate the performance of the model on eight benchmark datasets that show the effectiveness of the proposed method.

Electroencephalogram (EEG) has become increasingly popular in driver fatigue monitoring systems. Several decomposition methods have been attempted to analyze the EEG signals that are complex, nonlinear and non-stationary and improve the EEG decoding performance in different applications. However, it remains challenging to extract more distinguishable features from different decomposed components for driver fatigue recognition. In this work, we propose a novel decomposition-based hybrid ensemble convolutional neural network (CNN) framework to enhance the capability of decoding EEG signals. Four decomposition methods are employed to disassemble the EEG signals into components of different complexity. Instead of handcraft features, the CNNs in this framework directly learn from the decomposed components. In addition, a component-specific batch normalization layer is employed to reduce subject variability. Moreover, we employ two ensemble modes to integrate the outputs of all CNNs, comprehensively exploiting the diverse information of the decomposed components. Against the challenging cross-subject driver fatigue recognition task, the models under the framework all showed superior performance to the strong baselines. Specifically, the performance of different decomposition methods and ensemble modes was further compared. The results indicated that discrete wavelet transform-based ensemble CNN achieved the highest average classification accuracy of 83.48% among the compared methods. The proposed framework can be extended to any CNN architecture and be applied to any EEG-related tasks, opening the possibility of extracting more beneficial features from complex EEG data.

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series.

Since situation awareness (SA) plays an important role in many fields, the measure of SA is one of the most concerning problems. Using physiological signals to evaluate SA is becoming a popular research topic because of their advantages of non-intrusiveness and objectivity. However, previous studies mainly exploited the use of single physiological signals such as electroencephalogram (EEG) or eye tracking. The multi-modal SA recognition is still a research gap. Therefore, this work conducts a pilot study to investigate SA recognition by using two modalities: EEG and eye tracking data. Specifically, an optimized Stroop test that is more compatible with the definition of SA was used to induce different states of SA and collect physiological data. Furthermore, a random vector functional link-based stacking (RVFL-S) model was proposed to perform the multi-modal SA recognition. Experiment results showed that using the combination of EEG and eye tracking data can boost the performance of SA recognition. Moreover, the proposed RVFL-S model can effectively integrate the classification information from two modalities. It showed better performance than baseline methods, achieving 77.62% leave-one-subject-out (LOSO) average accuracy. This was around 5% improvement compared with the baseline classification models with input of only one modality. This pilot study demonstrated that the use of multi-modality is a potential strategy for SA recognition.

Time series forecasting remains a challenging task in data science while it is of great relevance to decision-making in various industries such as transportation, finance, electricity resource management, meteorology. Traditional forecasting models based on statistics fail in challenging tasks with high non-linearity and complicated characteristics. Due to its architecture bias, deep learning-based models overfit randomness and noise. This paper proposes a novel online performance-based ensemble deep random vector functional link neural network model for the time series forecasting tasks. The proposed model supports the non-iterative online learning and dynamic ensemble method, which keeps adjusting the parameters and the weights of each output layer based on the dynamic evaluation of the latest prediction performance. Extensive experiments show that our proposed method outperforms the state-of-the-art statistical, machine learning-based, and deep learning-based models.

Reservoir Computing (RC) is well-suited for simpler sequential tasks which require inexpensive, rapid training, and the Echo State Network (ESN) plays a significant role in RC. In this article, we proposed variations of the Random Vector Functional Link (RVFL) network based on reservoir computing for non-sequential tasks. To commence, we present a plain echo state-based RVFL (esRVFL) that is distinguished from randomly generated input weights by the fact that esRVFL generates sparse matrices randomly to complete the initialization of the neuron weights. Following that, we extended it to a deep structure and introduced several network topologies. We also follow esRVFL and replace the single layer of echo state with a multi-layer stacked echo state network, where the entire network only needs to compute a set of output weights, which is called deep esRVFL (desRVFL). We evaluated our method on several public datasets and compared it with related methods. Experiments have shown that the proposed method can handle the classification tasks for tabular data and outperform some state-of-the-art randomized neural networks.

Active upper limb assistive robots can improve patients' quality of life with limb impairment and facilitate the rehabilitation of patients in need. To provide intuitive and simultaneous active assistance, continuous joint angle prediction of the upper limb is crucial for promoting the interactive control between the robot and subjects. The model-free approach is gradually becoming mainstream in predicting joint angles, especially those based on deep neural networks. However, the current applied approach can not provide competitive predictive performance, fast computation speed, and learning efficiency. This paper implements random vector functional link networks (RVFL) and extreme learning machine networks (ELM) for the continuous joint angle prediction using only data from wearable inertial measurement units (IMU). The input features are five joint angles of the upper limb in the time domain, derived from the forearm attached IMU and upper arm attached IMU. Five joint angles are fed into the models to predict every joint angle. Multiple RVFL networks (Shallow RVFL, deep RVFL, and deep ensemble RVFL) and ELM networks (Shallow ELM, deep ELM, and deep ensemble ELM) were evaluated over a comprehensive experimental framework. On the one hand, the results show that the prediction performance of RVFL approaches is better than the variant of ELM networks, which are precise for practical robot-assisted application scenarios. On the other hand, the fast computation of RVFL networks offers significant practicability to the real-time control of assistive robots.

The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework’s capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures.

Hospitals can predetermine the admission rate and facilitate resource allocation based on valid emergency requests and bed capacity estimation. The excess unoccupied beds can be determined with the help of forecasting the number of discharged patients. Extracting predictive features and mining the temporal patterns from historical observations are crucial for accurate and reliable forecasts. Machine learning algorithms have demonstrated the ability to learn temporal knowledge and make predictions for unseen inputs. This paper utilizes several machine learning algorithms to forecast the inpatient discharges of Singapore hospitals and compare them with statistical methods. A novel ensemble deep learning algorithm based on random vector functional links is established to predict inpatient discharges. The ensemble deep learning framework is optimized in a greedy layer-wise fashion. Several forecasting metrics and statistical tests are utilized to demonstrate the proposed method's superiority. The proposed algorithm statistically outperforms the benchmark with a ranking of 1.875. Finally, practical implications and future directions are discussed.