Recently, contrastive learning approaches have achieved significant empirical success in representation learning for multivariate time series classification (MTSC). A key component of contrastive learning is to select appropriate augmentations. However, it remains an open question to find the desired augmentations of time series data for given datasets. Meanwhile, the existing augmentation methods focus on the design of feasible positive pairs while neglecting the construction of discriminative negative pairs. To address these issues, we propose a mixed supervised contrastive learning framework, called NI-MTSC, based on the neighborhood interpolation for MTSC. It integrates a novel neighborhood interpolation time series data augmentation and mixed supervised contrastive loss (MixCon). Firstly, the proposed neighborhood interpolation data augmentation method is used to generate more standardized and feasible positive and negative pairs for contrastive learning and form more distinct class boundaries in the representation space. The main steps include: i) we first use the distance metrics to explore the similarity of each sample data in different degrees and find the nearest-neighbor samples; ii) we adopt the linear interpolation to create new augmented samples between the original sample and the neighbor samples; iii) we introduce the label information of the original sample and the neighbor samples to construct positive and negative pairs. Secondly, to maximize the use of labels, we introduce the designed loss function MixCon, which combines the inter-class supervised and intra-class self-supervised contrastive loss to capture time series information hierarchically at the timestamp level for high-quality representations learning. The experiments on the 18 public datasets from the UEA MTSC archive and show that our proposed NI-MTSC achieves the best performance in classification accuracy compared with the state-of-the-art methods, indicating the effectiveness of our proposed method for the MTSC task.

Accurate multi-energy load forecasting is crucial for the optimal operation of Integrated Energy Systems (IES). This study innovatively proposes a hybrid prediction model with two core innovations: (1) We design a periodic Huber loss function that dynamically adjusts penalty weights, effectively balancing load periodicity characteristics with outlier robustness; (2) We propose a stacked prediction architecture that combines the dilated convolution properties of Temporal Convolutional Network (TCN) with the bidirectional temporal modeling capabilities of Bidirectional Gated Recurrent Unit (BiGRU). By connecting multiple fundamental modules in series through residual connections, the proposed model achieves progressive extraction and prediction of multi-scale temporal features in multi-energy load sequences through gradual residual output optimization. Experiments conducted on the actual operational data from Arizona State University’s campus energy system demonstrate that the proposed model exhibits better performance than the advanced methods (Informer and FEDformer). The experimental results indicate the proposed model is effective for real-time operational scheduling of integrated energy systems.

In controlled settings, facial age estimation models have achieved remarkable accuracy. However, in the uncontrolled or "in the wild" scenarios, the performance of these models faces significant challenges. Factors such as varying image resolutions, environmental lighting conditions, and obstructions substantially impact the effectiveness of age estimation in these less constrained environments. This complexity highlights the ongoing need for advancements in the field to address the unique challenges presented by the real-world conditions. To solve this problem, we propose a novel model that combines Convolutional Neural Networks (CNN) and Vision Transformer architectures. We have made a strategic modification to the ResNet architecture, which forms the backbone of our model. A significant enhancement in our model is the replacement of traditional convolutions with depth-wise separable convolutions, which offers improved efficiency and reduced model complexity. In addition, we replace the last three bottleneck layers with the components of the Vision Transformer. This alteration allows for a more sophisticated and nuanced feature extraction process, particularly beneficial for complex image scenarios encountered in the uncontrolled settings. Additionally, our model incorporates a feature pooling pyramid structure to facilitate the multi-scale feature fusion. This innovative combination not only leverages the strengths of both CNN and Transformers, but also enhances the feature extraction and representation capabilities across various scales, making it particularly effective for diverse vision-based tasks. We evaluate our model on three uncontrolled popular datasets and validate the effectiveness of the proposed method.

Student performance prediction is vital for identifying at-risk students and providing support to help them succeed academically. In this paper, we propose a feature importance-based multi-layer CatBoost approach to predict the students' grade in the period exam. The idea is to construct a multi-layer structure with increasingly important features layer by layer. Specifically, the feature importance are first calculated and sorted in ascending order. In each layer, features with the least importance are accumulated until reaching a given threshold. Then, these selected features are used to construct the first layer by training the CatBoost. Next, this trained CatBoost is utilized to generate a feature that adds to the feature set with their importance within a threshold. After that, all these feature are used to train the CatBoost in the next layer. This process is repeated until all the features are used. The results show that the proposed model has the best performance. Moreover, the statistical test conducted based on 20-runs of experiments validates the significant superiority of our proposed model over the compared models and demonstrates the efficacy of the multi-layer structure in enhancing the proposed model. This indicates our proposed model can help decision makers in enhancing educational quality.

To address the limitations of existing house price prediction methods that struggle with handling mixed data types, we developed a novel model utilizing a reinforcement learning-based weight fusion algorithm. First, we reconstruct the feature fusion vectors from categorical feature variables. Then, we create individual house price prediction models using a variety of algorithms: Multiple Linear Regression, Regression Decision Tree, Extreme Random Regression Tree, and Multilayer Perceptron. These models' weights are dynamically adjusted using Q-Learning to minimize the Root Mean Square Error (RMSE). Our experiments demonstrate substantial gains in RMSE performance using our weight fusion approach. Specifically, on the small-scale Shenzhen dataset, RMSE improvements vary from 8.65% to 64.61%. On the large-scale Boston dataset, the improvements range between 53.06% and 76.40% in comparison to the standalone prediction algorithms.

Although machine learning classifiers have been successfully used in the medical and engineering fields, there is still room for improving the predictive accuracy of model classification. The higher the accuracy of the classifier, the better suggestions can be provided for the decision makers. Therefore, in this study, we propose an ensemble machine learning approach, called Feature generation-based Ensemble Support Vector Machine (FESVM), for classification tasks. We first apply the feature selection technique to select the most related features. Next, we introduce an ensemble strategy to aggregate multiple base estimators for the final prediction using the meta-classifier SVM. During this stage, we use the classification probabilities obtained from the base classifier to generate new features. After that, the generated features are added to the original data set to form a new data set. Finally, this new data set is utilised to train the meta-classifier SVM to obtain the final classification results. For example, for a binary classification task, each base classifier has two probabilities (p for one class and 1−p for the other class). In this case, two new features are generated from the combination of probabilities based on these base classifiers. One is the sum of p as new feature 1, and the other is the sum of 1−p as new feature 2. These two new features are then added to the original data set to form the new data set. In the same way, our feature generation method can be easily extended for a multi-class task for generating new features, where the number of features depends on the number of classes. Those generated features from the base estimators (first layer) are added to the original data set to form a new data set. This new data set is used as the input to the second layer (meta-classifier) to obtain the final model. Experiments based on the 20 data sets show that our proposed model FESVM has the best performance compared to the other machine learning classifiers under comparison. In addition, our FESVM has better performance than the original stacking method in the multi-class classification tasks. Statistical results based on the Wilcoxon–Holm method also confirms that our FESVM can significantly outperform the other models. These indicate that our FESVM can be a useful tool for classification tasks, especially multi-classification tasks.

Financial news disclosures provide valuable information for traders and investors while making stock market investment decisions. Essential but challenging, the stock market prediction problem has attracted significant attention from both researchers and practitioners. Conventional machine learning models often fail to interpret the content of financial news due to the complexity and ambiguity of natural language used in the news. Inspired by the success of recurrent neural networks (RNNs) in sequential data processing, we propose an ensemble RNN approach (long short-term memory, gated recurrent unit, and SimpleRNN) to predict stock market movements. To avoid extracting tens of thousands of features using traditional natural language processing methods, we apply sentiment analysis and the sliding window method to extract only the most representative features. Our experimental results confirm the effectiveness of these two methods for feature extraction and show that the proposed ensemble approach is able to outperform other models under comparison.

Transformer-based and interaction point-based methods have demonstrated promising performance and potential in human-object interaction detection. However, due to differences in structure and properties, direct integration of these two types of models is not feasible. Recent Transformer-based methods divide the decoder into two branches: an instance decoder for human-object pair detection and a classification decoder for interaction recognition. While the attention mechanism within the Transformer enhances the connection between localization and classification, this paper focuses on further improving HOI detection performance by increasing the intrinsic correlation between instance and action features. To address these challenges, this paper proposes a novel Transformer-based HOI Detection framework. In the proposed method, the decoder contains three parts: learnable query generator, instance decoder, and interaction classifier. The learnable query generator aims to build an effective query to guide the instance decoder and interaction classifier to learn more accurate instance and interaction features. These features are then applied to update the query generator for the next layer. Especially, inspired by the interaction point-based HOI and object detection methods, this paper introduces the prior bounding boxes, keypoints detection and spatial relation feature to build the novel learnable query generator. Finally, the proposed method is verified on HICO-DET and V-COCO datasets. The experimental results show that the proposed method has the better performance compared with the state-of-the-art methods.

DBSCAN is a classical clustering algorithm, which can identify different shapes and isolate noisy patterns from a dataset. Despite the above advantages, the bottleneck of DBSCAN is its computation time for high dimensional datasets. This work, thus, presents a simple and fast method to improve the efficiency of DBSCAN algorithm. We reduce the execution time in two aspects. The first one is to reduce the number of points presented to DBSCAN and the second one is to apply the HNSW technique instead of the linear search structure for improving its efficiency. The experimental results show that our proposed algorithm can greatly improve the clustering speed without losing or even obtaining better accuracy, especially for large-scale datasets.