Split Learning (SL) is a distributed learning framework that has gained popularity for its privacy-preserving nature and low computational demands. However, recent studies have the potential that a server adversary to carry out inference attacks, compromising the privacy of victim clients. Nevertheless, upon re-evaluating prior studies, we found that existing methods rely on overly strong assumptions to enhance their performance, resulting in a significant decline in effectiveness under more realistic scenarios. In this work, we provide new insights into the inherent vulnerabilities of SL. Specifically, we discover that both the smashed data and the server model contain the client's representation preference, which the server adversary can exploit to build a substitute client that approximates the target client's unique feature extraction behavior. With a well-trained substitute client, the server can perfectly steal the target client's functionality, training data, and labels. Building on this observation, we introduce Split Leakage (SLeak), a new threat that targets multiple privacy stealing objectives against SL. Notably, SLeak does not depend on strong privacy priors and only requires partial same-domain auxiliary public data to conduct the attacks. Experimental results on diverse datasets and target models show that SLeak surpasses the state-of-the-art method across multiple metrics. Moreover, ablation studies further confirm its robustness and applicability under various scenarios and assumptions.

Model Inversion Attacks (MIAs) can recover private training data by accessing model weights or outputs, posing significant threats to user privacy. Existing defenses cannot provide comprehensive protection against attackers with varying levels of knowledge and often lack evaluation against the most advanced attacks. Moreover, current defenses primarily focus on regularizing latent representations or labels. While prior work has explored input-level defenses (e.g., Random Erasing), these approaches are typically limited to simple transformations, and more complex or systematically combined input-level defenses remain underexplored. As the most common input-level perturbation technique, data augmentation applies transformations like cropping directly to input images. However, finding augmentations or their combinations that achieve a good privacy-utility trade-off is challenging, as it is impossible to evaluate every augmentation exhaustively through attacks. To address this, we design privacy and utility assessments for efficient evaluation and propose a Defense via Auto-Augmentation Search (DAAS). DAAS can automatically assess and identify candidates with strong privacy-utility trade-offs from a large augmentation pool. The final search results can then be leveraged for privacy-preserving training against MIAs. We evaluate DAAS across various models, datasets, and attacks, demonstrating superior defense performance compared to existing methods. Extensive ablation studies further demonstrate the effectiveness of DAAS.

The rapid growth of video platforms has transformed information dissemination and led to an explosion of multimedia content. However, this widespread reach also introduces risks, as some users exploit these platforms to spread hate speech, which is often concealed through complex rhetoric, making hateful video detection a critical challenge. Existing detection methods rely heavily on unimodal analysis or simple feature fusion, struggling to capture cross-modal interactions and reason through implicit hate in sarcasm and metaphor. To address these limitations, we propose HVGuard, the first reasoning-based hateful video detection framework with multimodal large language models (MLLMs). Our approach integrates Chain-of-Thought (CoT) reasoning to enhance multimodal interaction modeling and implicit hate interpretation. Additionally, we design a Mixture-of-Experts (MoE) network for efficient multimodal fusion and final decision-making. The framework is modular and extensible, allowing flexible integration of different MLLMs and encoders. Experimental results demonstrate that HVGuard outperforms all existing advanced detection tools, achieving an improvement of 6.88% to 13.13% in accuracy and 9.21% to 34.37% in M-F1 on two public datasets covering both English and Chinese.

Split Learning (SL) is a widely adopted distributed privacy-preserving training paradigm with minimal computational overhead for clients. However, Feature-Space Hijacking Attack (FSHA) poses a significant threat against SL, where the server manipulates the client's optimization process, compromising input privacy. Some studies propose that clients can detect potential hijacking by monitoring the gradients returned by the server. However, these gradient-based methods are vulnerable to adversarial anti-detection and lack robustness to changes in model architecture. In this paper, we propose a novel detection method named Bidirectional Feature Discrepancy Defense (BiFD), which leverages features to capture richer semantic information. We also observe that hijacked features are easier to reconstruct and harder to classify, providing a key distinction between malicious and honest servers—an aspect overlooked in previous works. Extensive results across multiple datasets and model architectures demonstrate the excellent and robust performance of BiFD.

Chaotic dynamics are ubiquitous in many real-world systems, ranging from biological and industrial processes to climate dynamics and the spread of viruses. These systems are characterized by high sensitivity to initial conditions, making it challenging to predict their future behavior confidently. In this study, we propose a novel deep-learning framework that addresses this challenge by directly exploiting the long-term compounding of local prediction errors during model training, aiming to extend the time horizon for reliable predictions of chaotic systems. Our approach observes the future trajectories of initial errors at a time horizon, modeling the evolution of the loss to that point through the use of two major components: (1) a recurrent architecture (Error Trajectory Tracing) designed to trace the trajectories of predictive errors through phase space, and (2) a training regime, Horizon Forcing, that pushes the model’s focus out to a predetermined time horizon. We validate our method on three classic chaotic systems and six real-world time series prediction tasks with chaotic characteristics. The results show that our approach outperforms the state-of-the-art methods.

The transition from 2D to 3D medical imaging presents unprecedented challenges in data analysis and interpretation, particularly due to domain shifts and the complex temporal nature of patient data. We introduce 3DLVR, a groundbreaking framework that leverages advanced domain adaptation techniques to bridge this dimensional gap in automated radiology report generation. At the heart of 3DLVR lies a novel cross-attention-based multi-modal fusion module, 3DLR, which seamlessly integrates longitudinal patient data to adapt to evolving medical conditions and imaging characteristics. Our approach utilizes optimal transport theory to formulate the domain adaptation problem, ensuring robust performance across diverse patient populations and imaging protocols. Experiments on the CT-RATE dataset demonstrate 3DLVR’s superior adaptability, with a great improvement in cross-domain performance compared to existing methods. Furthermore, our framework exhibits remarkable generalization capabilities. By advancing the frontier of domain adaptation in medical AI, 3DLVR not only enhances the accuracy and relevance of automated reports but also paves the way for more personalized and adaptive healthcare systems. This work represents a significant step towards realizing the potential of 3D medical imaging in the era of precision medicine, with broad implications for improving diagnostic accuracy, treatment planning, and patient outcomes.

In the rapidly advancing area of semi-supervised learning (SSL), especially concerning large-scale computer vision models and medical imaging, pseudo-labeling has become a central strategy. Nevertheless, this technique faces significant hurdles, including the possibility of integrating erroneously pseudo-labeled data due to improper threshold settings and the underutilization of data with low-confidence pseudo-labels. To overcome these challenges, we introduce a novel SSL framework for training large models, termed PE-DAT, which fuses Pseudo-loss Estimation with Domain-Adaptive Training. This integration substantially enhances the accuracy of medical image classification tasks, addressing both multiclass and multi-label problems. A pivotal aspect of our approach is a cutting-edge data selection method based on the concept that genuine data exhibit lower pseudo-loss values compared to their noisy counterparts. The PE-DAT framework’s effectiveness is demonstrated through its superior performance relative to existing benchmarks, validated by extensive testing on a wide array of medical and natural image datasets.

Diabetic Retinopathy (DR) is a leading cause of vision impairment globally, making accurate and efficient detection of DR stages from ophthalmological images critical for early diagnosis and treatment. In this study, we develop and evaluate the efficacy of four state-of-the-art deep learning models for classifying DR stages: a custom Convolutional Neural Network (CNN), VGG19, ResNet50, and Vision Transformer (ViT). These models are systematically applied to the Brazilian Multi-label Ophthalmological Dataset (BRSET), which consists of 16,266 retinal fundus images categorized into different DR severity levels. Each model’s performance is assessed based on metrics such as accuracy, F1-score, and AUC-ROC. Our results provide a comprehensive comparison of these architectures, highlighting their strengths and limitations in DR stage estimation, and offer insights into the suitability of these models for large-scale DR screening.

Split Learning (SL) is a distributed learning framework renowned for its privacy-preserving features and minimal computational requirements. Previous research consistently highlights the potential privacy breaches in SL systems by server adversaries reconstructing training data. However, these studies often rely on strong assumptions or compromise system utility to enhance attack performance. This paper introduces a new semi-honest Data Reconstruction Attack on SL, named Feature-Oriented Reconstruction Attack (FORA). In contrast to prior works, FORA relies on limited prior knowledge, specifically that the server utilizes auxiliary samples from the public without knowing any client's private information. This allows FORA to conduct the attack stealthily and achieve robust performance. The key vulnerability exploited by FORA is the revelation of the model representation preference in the smashed data output by victim client. FORA constructs a substitute client through feature-level transfer learning, aiming to closely mimic the victim client's representation preference. Leveraging this substitute client, the server trains the attack model to effectively reconstruct private data. Extensive experiments showcase FORA's superior performance compared to state-of-the-art methods. Furthermore, the paper systematically evaluates the proposed method's applicability across diverse settings and advanced defense strategies.

Effective early warning systems for extreme flood events in large river basins necessitate reliable long-lead streamflow forecasts. However, the inherent uncertainty within each phase of the weather system-rainfall prediction, runoff generation, and streamflow prediction-amplifies with each stage, rendering accurate long-lead streamflow estimations challenging. In response to this, our study introduces a novel deep-learning-based model, the Cascaded Spatio-Temporal Learning Deep Network (CASTLE). CASTLE synergistically integrates observed upstream precipitation, recent streamflow data, and short-term precipitation forecasts derived from a selection of quantitative climate models to produce an accurate streamflow estimate. Specifically, we employ deep residual architectures on both observed and forecasted precipitation data to model the cascading spatio-temporal processes, which begin with upstream rainfall, move to rainfall-runoff, and finally conclude with downstream discharge. Our aim is to identify hidden space-time patterns that can be used to forecast future downstream flow over extended periods. We assess CASTLE’s efficacy by forecasting the downstream discharge of the Ganges River over a long lead time. Results show that our approach outperforms the current state-of-the-art streamflow forecasting models.

Paraphrase generation is a fundamental and long-standing task in natural language processing. In this paper, we concentrate on two contributions to the task: (1) we propose Retrieval Augmented Prompt Tuning (RAPT) as a parameter-efficient method to adapt large pre-trained language models for paraphrase generation; (2) we propose Novelty Conditioned RAPT (NC-RAPT) as a simple model-agnostic method of using specialized prompt tokens for controlled paraphrase generation with varying levels of lexical novelty. By conducting extensive experiments on four datasets, we demonstrate the effectiveness of the proposed approaches for retaining the semantic content of the original text while inducing lexical novelty in the generation.

The understanding of chaotic systems is challenging not only for theoretical research but also for many important applications. Chaotic behavior is found in many nonlinear dynamical systems, such as those found in climate dynamics, weather, the stock market, and the space-time dynamics of virus spread. A reliable solution for these systems must handle their complex space-time dynamics and sensitive dependence on initial conditions. We develop a deep learning framework to push the time horizon at which reliable predictions can be made further into the future by better evaluating the consequences of local errors when modeling nonlinear systems. Our approach observes the future trajectories of initial errors at a time horizon to model the evolution of the loss to that point with two major components: 1) a recurrent architecture, Error Trajectory Tracing, that is designed to trace the trajectories of predictive errors through phase space, and 2) a training regime, Horizon Forcing, that pushes the model’s focus out to a predetermined time horizon. We validate our method on classic chaotic systems and real-world time series prediction tasks with chaotic characteristics, and show that our approach outperforms the current state-of-the-art methods.

Existing interactive learning systems usually train models on simulators as surrogates for real users. Due to the limited amount of user data, trained simulators may lead to biased results as it fails to well represent real users. One solution is to model users as agents, and then simultaneously train the interactive system and user agents by multi-agent reinforcement learning (MARL) frameworks. However, developing efficient MARL frameworks for modern interactive multimodal systems is still challenging. First, given the existence of multimodal data, how to develop accurate multimodal fusion within and between agents in each interaction is challenging and unclear. Second, interactions between users and systems are complex and it is challenging to track and synchronize the interactions over time. The above multimodal fusion between agents and synchronization over time becomes even more challenging, when the amount of user data is limited. To jointly address these challenges and achieve more sample-efficient learning, we propose a novel spatial-temporal aligned sta multi-agent reinforcement learning framework to better align the multimodal data within and between agents over time. Based on our framework, we develop sample-efficient visual dialog systems. Through extensive experiments and analysis, we validate the effectiveness of our spatial-temporal aligned sta multi-agent reinforcement learning framework in visual dialog systems.

The study of model bias and variance with respect to decision boundaries is critically important in supervised learning and artificial intelligence. There is generally a tradeoff between the two, as fine-tuning of the decision boundary of a classification model to accommodate more boundary training samples (i.e., higher model complexity) may improve training accuracy (i.e., lower bias) but hurt generalization against unseen data (i.e., higher variance). By focusing on just classification boundary fine-tuning and model complexity, it is difficult to reduce both bias and variance. To overcome this dilemma, we take a different perspective and investigate a new approach to handle inaccuracy and uncertainty in the training data labels, which are inevitable in many applications where labels are conceptual entities and labeling is performed by human annotators. The process of classification can be undermined by uncertainty in the labels of the training data; extending a boundary to accommodate an inaccurately labeled point will increase both bias and variance. Our novel method can reduce both bias and variance by estimating the pointwise label uncertainty of the training set and accordingly adjusting the training sample weights such that those samples with high uncertainty are weighted down and those with low uncertainty are weighted up. In this way, uncertain samples have a smaller contribution to the objective function of the model’s learning algorithm and exert less pull on the decision boundary. In a real-world physical activity recognition case study, the data present many labeling challenges, and we show that this new approach improves model performance and reduces model variance.

Complex contagion models have been developed to understand a wide range of social phenomena such as adoption of cultural fads, the diffusion of belief, norms, and innovations in social networks, and the rise of collective action to join a riot. Most existing works focus on contagions where individuals’ states are represented by binary variables, and propagation takes place over a single isolated network. However, characterization of an individual’s standing on a given matter as a binary state might be overly simplistic as most of our opinions, feelings, and perceptions vary over more than two states. Also, most real-world contagions take place over multiple networks (e.g., Twitter and Facebook) or involve multiplex networks where individuals engage in different types of relationships (e.g., co-worker, family, etc.). To this end, this paper studies multi-stage complex contagions that take place over multi-layer or multiplex networks. Under a linear threshold based contagion model, we first give analytic results for the expected size of global cascades, i.e., cases where a randomly chosen node can initiate a propagation that eventually reaches a positive fraction of the whole population. Then, analytic results are confirmed by an extensive numerical study. In addition, we demonstrate how the dynamics of complex contagions is affected by the structural properties of the networks. In particular, we reveal an interesting connection between the assortativity of a network and the impact of hyper-active nodes on the cascade size.

Spread of influence is one of the most widely studied propagation processes in the literature on complex networks. Examples include the rise of collective action to join a riot and diffusion of beliefs, norms, and cultural fads, to name a few. Most existing works on modeling influence propagation consider a single content (e.g., an opinion, decision, product, political view, etc.) spreading over a network independent from everything else. However, most real-life examples involve multiple correlated contents spreading simultaneously and exhibiting positive (e.g., opinions on same-sex marriage and gun control) or negative (e.g., opinions on universal health care and tax-relief for the “rich”) correlation. To accommodate these cases, this paper proposes the vector threshold model, as an extension of the widely used Watts threshold model for complex contagions. Here, the state of a node is represented by a binary vector representing their opinion on a number of content items. Nodes switch their states based on the influence they receive from their neighbors in the network. The influence is represented by a vector containing the proportion of neighbors who support each content; both positively and negatively correlated contents can be captured in this formulation by using different rules for switching node states. Our main result is concerned with the expected size of global cascades, i.e., cases where a randomly chosen node can initiate a propagation that eventually reaches a positive fraction of the whole population. We also derive conditions on network structure for global cascades to be possible. Analytic results are supported by a numerical study.

It is well known that a direct parallelization of sequential optimization methods (e.g., coordinate descent and stochastic gradient methods) is often not effective. The reason is that at each iteration, the number of operations may be too small. We point out that this common understanding may not be true if the algorithm sequentially accesses the data in a feature-wise manner. For almost all real-world sparse sets we have examined, some features are much denser than others. Thus a direct parallelization of loops in a sequential method may result in excellent speedup. This approach possesses an advantage of retaining all convergence results because the algorithm is not changed at all. We apply this idea on coordinate descent (CD) methods, which are effective single-thread technique for L1-regularized classification. Further, an investigation on the shrinking technique commonly used to remove some features in the training process shows that this technique helps the parallelization of CD methods. Experiments indicate that a naive parallelization achieves better speedup than existing methods that laboriously modify the algorithm to achieve parallelism. Though a bit ironic, we conclude that the naive parallelization of the CD method is a highly competitive and robust multi-core implementation for L1-regularized classification.

Click-through rate (CTR) prediction plays an important role in computational advertising. Models based on degree-2 polynomial mappings and factorization machines (FMs) are widely used for this task. Recently, a variant of FMs, field-aware factorization machines (FFMs), outperforms existing models in some world-wide CTR-prediction competitions. Based on our experiences in winning two of them, in this paper we establish FFMs as an effective method for classifying large sparse data including those from CTR prediction. First, we propose efficient implementations for training FFMs. Then we comprehensively analyze FFMs and compare this approach with competing models. Experiments show that FFMs are very useful for certain classification problems. Finally, we have released a package of FFMs for public use.

Matrix factorization (MF) plays a key role in many applications such as recommender systems and computer vision, but MF may take long running time for handling large matrices commonly seen in the big data era. Many parallel techniques have been proposed to reduce the running time, but few parallel MF packages are available. Therefore, we present an open source library, LIBMF, based on recent advances of parallel MF for shared-memory systems. LIBMF includes easy-to-use command-line tools, interfaces to C/C++ languages, and comprehensive documentation. Our experiments demonstrate that LIBMF outperforms state of the art packages. LIBMF is BSD-licensed, so users can freely use, modify, and redistribute the code.

Matrix factorization is known to be an effective method for recommender systems that are given only the ratings from users to items. Currently, stochastic gradient (SG) method is one of the most popular algorithms for matrix factorization. However, as a sequential approach, SG is difficult to be parallelized for handling web-scale problems. In this article, we develop a fast parallel SG method, FPSG, for shared memory systems. By dramatically reducing the cache-miss rate and carefully addressing the load balance of threads, FPSG is more efficient than state-of-the-art parallel algorithms for matrix factorization.

Matrix factorization is known to be an effective method for recommender systems that are given only the ratings from users to items. Currently, stochastic gradient descent (SGD) is one of the most popular algorithms for matrix factorization. However, as a sequential approach, SGD is difficult to be parallelized for handling web-scale problems. In this paper, we develop a fast parallel SGD method, FPSGD, for shared memory systems. By dramatically reducing the cache-miss rate and carefully addressing the load balance of threads, FPSGD is more efficient than state-of-the-art parallel algorithms for matrix factorization.