As an essential component of fine-tuning, warm-up plays a crucial role in promoting stability and generalization. Many studies have examined its underlying mechanisms from different aspects. However, most of the studies focus on incorporating these insights into optimizers to reduce the reliance on warm-up. Little attention has been paid to addressing the inherent limitations of the warm-up itself, which restricts its effectiveness. In this work, we revisit warm-up from a loss landscape perspective and identify several limitations with existing warm-up, including: (1) susceptibility to nearby suboptimal traps, (2) sensitivity to hyperparameters and random seeds, and (3) inefficiency during the early stages of training. To overcome these limitations, we propose Sensitivity-Aware Warm-Up (SAWU), a lightweight and adaptive strategy that dynamically leverages learning sensitivity during warm-up to guide updates toward better and more stable basins. In addition, SAWU also introduces an adaptive scheduling mechanism and phase transition strategy across warm-up, stable, and decay phases to further enhance robustness and efficiency. Extensive experiments on various downstream tasks show that SAWU significantly outperforms the vanilla method (e.g., average 3.43% improvement on RoBerta). Moreover, SAWU can be easily combined with various optimizers and remains effective even when warm-up-based methods fail (e.g, it lifts RAdam from 49.46% to 91.78% on qnli. Thanks to its lightweight nature, SAWU introduces minimal overhead and even reduces training time by over 5% compared to other methods.

Feature engineering plays a critical role in machine learning (ML), but existing methods often struggle with high computational cost and limited scalability when applied to large-scale and sparse datasets. In this article, we propose EaSFE, an efficient and scalable feature engineering framework that unifies feature generation, filtering, and evaluation in an end-to-end manner. EaSFE is designed to efficiently construct and select informative features while explicitly considering computational and memory constraints. To achieve scalability, EaSFE incorporates parallel and distributed execution mechanisms, as well as a chunk-based data processing strategy that enables memory-efficient feature engineering on large datasets. In addition, EaSFE adopts tailored storage and execution strategies to handle high-dimensional sparse data effectively. Extensive experiments on multiple real-world datasets demonstrate that EaSFE consistently improves predictive performance (e.g., 5% accuracy improvement in poker ) while substantially enhancing efficiency (i.e., over 10x speedup) compared to existing feature engineering methods. In addition, EaSFE is demonstrated to scale to large and sparse datasets, successfully handling datasets with over 119 million training instances and 54 million features.

The task of dynamic graph link prediction is to forecast the evolution of complex systems. Empirical observations reveal that interactions within these systems exhibit an Entangled Spatio-Temporal Pattern, which manifests through three interrelated phenomena, namely Latent High-Order Bridges, Multi-Frequency Temporal Dynamics, and Spatio-Temporal Entanglement, with stronger structural ties facilitating tolerance for longer temporal gaps. However, limited by computationally prohibitive multi-hop sampling or inefficient long-sequence modeling, existing methods struggle to capture this complex pattern. Inspired by State-Space Models (SSMs) like Mamba for efficient long-range modeling yet aiming to address their native agnosticism to structural and multi-frequency dynamics, we propose a framework named DyGHydra, which couples a tailored Continuous-Time Hierarchical Mamba (CT-HMamba) backbone with a multi-hop structural encoder. The framework first employs the multi-hop structural encoder to reveal latent high-order interactions, extracting interaction-level cross-hop features. Subsequently, the CT-HMamba backbone utilizes these features to address multi-frequency dynamics through a hierarchical architecture, decomposing interaction history to simultaneously model high-frequency bursts and long-term trends. To capture the spatio-temporal entanglement, CT-HMamba further tailors its core state-space mechanism to be co-driven by physical time and structural context. Specifically, physical time governs the state transition decay to reflect temporal forgetting, while structural context modulates the input-output projections to prioritize topologically significant events. Extensive experiments on eleven real-world datasets show that DyGHydra achieves state-of-the-art performance across most settings for both transductive and inductive link prediction, validating its effectiveness in modeling complex temporal dynamics with superior efficiency.

Creating photorealistic and animatable 3D head avatars across multiple views is still an ongoing challenge in AR/VR applications. Although previous studies adopt Neural Radiance Fields (NeRF) with 3D Morphable Model (3DMM) as a prior to produce impressive results in generating 3D heads, they incur considerable time costs and lack rendering quality. In this paper, we propose a novel approach called MGHead that synthesizes high-fidelity dynamic 3D head avatar with realistic appearance and complex deformation. It exploits anchor-based 3D Gaussians to model geometric shape and extends the fundamental deformation structure of a parametric morphable face model to each fixed point, providing greater robustness in responding to expressive variations. Notably, we introduce an expression and pose-dependent attribute adapter that injects driving signals into anchor features to refine neural Gaussian attributes. This adjustment compensates for the deficiencies of linear blend skinning in capturing high-frequency dynamics effectively, further improving the expressive realism and natural appearance of head avatar. Extensive experiments demonstrate the superiority of our model in visual detail quality and quantitative evaluations.

Data science aims to solve real-world problems with the knowledge derived from data. Successfully tackling a data science problem requires practitioners to choose an appropriate solution, which potentially comprises various components such as pre-processing techniques, learning algorithms, hyper-parameters, and so on. Therefore, a problem-driven recommendation for the promising solution is invaluable, as it facilitates efficient and convenient problem-solving. However, existing solution recommendation approaches confront notable challenges when dealing with limited and sparse prior experience in practical applications. Learning from such prior easily leads to overfitting and poor generalization in solution recommendations. To address this issue, we propose a novel solution recommendation method that can predict a good-quality data science solution, including the pre-processing, the learning algorithm, and hyper-parameters, for a given problem. The foundation of our method is a carefully designed ranking model that exploits a weight-sharing structure and a newly proposed loss. The ranking model focuses on incorporating relative ranking information into the predicted performance score of each solution. With these techniques, our method can recommend the solution with the highest score and effectively mitigate the limitations of using sparse prior experience. Our experiments demonstrate the superiority of our method in predicting solutions with higher accuracy and rank, even trained on highly sparse historical performance records. It also reduces recommendation time significantly compared to the baselines, offering remarkable efficiency and convenience for practitioners.

Knowledge tracing defines the task of predicting future performance of students based on their historical interactions. Recently, some graph-based methods try to capture correspondence between questions and concepts by constructing the question-concept bipartite graph to tackle the knowledge tracing problem. However, they fail to explicitly integrate such intrinsic relations into the final answer predictor due to the sparse data. In this paper, we propose a novel Multi-relation Enhanced Graph-based Model for Knowledge Tracing (MGKT) to tackle the above problem. More specifically, MGKT constructs graph structure to explore multiple relations such as the high-order association among questions and the similarity of question’s attributes. In addition, two self-supervised training strategies, namely hypergraph contrast learning and hypergraph reconstruction, are proposed to incorporate these special correlations into question representations. Extensive experiments demonstrate that MGKT outperforms state-of-the-art knowledge tracing methods on three benchmark datasets.

Super-resolution combined with novel image synthesis is an advanced image processing method to synthesize low-resolution images into new high-resolution images. NeRF-SR is the first model to obtain decent multi-view super-resolution results with only low-resolution input images, but the super-sampling method implemented using the original Nerf’s MLP network cannot represent the complex details of the scene well. We consider that the volume density and color features obtained by the MLP network do not take into account the global geometry along the ray and the color relationship between the sampling points. To tackle this challenge, we introduce an attention-based model and auto-encoding network to synthesize high-fidelity views from low-resolution input to high-resolution output. The attention-based model mixes the pixel color information of the sampling points on each ray and supervises using ground-truth colors. At the same time, the auto-encoding network learns the global geometry along the ray. Experimental results demonstrate that our model can produce high-quality results for high-resolution new view synthesis, both on synthetic and real-world datasets.

Monitoring corn crop growth status is of great significance to crop production, breeding, and seed production. The Unmanned Aerial Vehicles’ (UAVs) technology makes it possible to use computer vision technology to identify corn growth stage intelligently. A model customized for corn growth status monitoring based on a dense convolutional neural network (CM-CNN) was proposed, including a two-way dense module and a new activation function ELU. The two-way dense module enlarges the receptive field, while the ELU alleviates gradient disappearance and speeds up learning in deep neural networks. Dense architecture concatenates all the previous layer features to enhance feature reuse. The proposed CM-CNN performs well in classifying corn growth stages. Experimental results show that CM-CNN is a state-of-the-art method, with an accuracy of its relevant data up to 99.3%. Compared with other CNN models, viz. AlexNet, ZFNet, VGG, InceptionV3, Xception and ResNet, fewer parameters are in CM-CNN.

Community detection is a well-established problem and nontrivial task in complex network analysis. The goal of community detection is to discover community structures in complex networks. In recent years, many existing works have been proposed to handle this task, particularly nonnegative matrix factorization-based method, e.g. HNMF, BNMF, which is interpretable and can learn latent features of complex data. These methods usually decompose the original matrix into two matrixes, in one matrix, each column corresponds to a representation of community and each column of another matrix indicates the membership between overall pairs of communities and nodes. Then they discover the community by updating the two matrices iteratively and learn the shallow feature of the community. However, these methods either ignore the topological structure characteristics of the community or ignore the microscopic community structure properties. In this paper, we propose a novel model, named Modularized Deep NonNegative Matrix Factorization (MDNMF) for community detection, which preserves both the topology information and the instinct community structure properties of the community. The experimental results show that our proposed models can significantly outperform state-of-the-art approaches on several well-known dataset.

Cancer diseases are caused by combination of genetic, environmental, and lifestyle factors. Therefore, it is difficult for health organization to treat this disease. This study focuses on identifying the landscape of Cancer research by using bibliometric methods and social network analysis methods based on a number of research articles related to Cancer retrieved from PubMed. To deeply understand the landscape of research on this disease, we adopt productivity analysis which consists of author, university/institution, country and frequent MeSH terms analysis. We specifically perform the concept graph-based network analysis by applying four centrality measures and analyzing co-occurrence of MeSH terms. In the end, we propose a method to predict the Rising Star that may be the active researcher in the field of Cancer disease in the next few years. With this method, we can possibly find more academic cooperation via academic social networks. The encouraging results show that our work is highly feasible.