Multimodal sarcasm detection (MSD) aims to identify sarcasm polarity from diverse modalities (i.e., image–text pairs), a task that has received increasing attention. While significant progress has been made, existing approaches still face two major issues: lack of explainability and weak generalizability. In this paper, we introduce a new large vision–language model (LVLM) dubbed S³-MSD for explainable and generalizable MSD through three key components. For explainability, we develop (1) a self-training paradigm that automatically bootstraps answers with explanations, and (2) a self-calibrating mechanism that rectifies flawed explanations. For generalizability, we design (3) a self-focusing module that amplifies visual semantic entities through preference optimization, thereby mitigating textual over-reliance. Experimental results on both in-distribution and out-of-distribution (OOD) benchmarks demonstrate that S³-MSD consistently outperforms state-of-the-art methods in detection performance. Furthermore, the proposed S³-MSD provides persuasive explanations, as verified by both quantitative metrics and human evaluations.

Low light conditions in endoscopic imaging would lead to poor visibility, reduced contrast, and increased noise, which may hinder accurate diagnosis and surgical guidance. Against this low-light endoscopic image enhancement (LLEIE) task, inspired by the remarkable performance of pretrained CLIP in downstream vision tasks, in this paper, we carefully investigate the pretrained priors of CLIP and embed them into a text-modulated semantic-aware discriminator (TMSD). Through the adversarial learning mechanism, the discriminator can be easily integrated into different low-light enhancement baselines for helping them accomplish better visual restoration effects without incurring any extra inference cost. Specifically, to make the foundation model CLIP suitable for the LLEIE task, we initially propose a prompt learning procedure to obtain the text embedding and image semantics corresponding to the normal-light endoscopic imaging scenario. Building upon the acquired text prior and image semantic priors, we devise a text modulator to synergize these two priors, yielding a richer semantic representation. Leveraging the convolutional modulation and cross-attention mechanisms, we blend this semantic guidance information into the discriminator, thereby fostering the fine-grained distribution learning of normal-light endoscopic images in visual semantics and guiding different enhancement baselines achieving higher visual quality. Based on five public benchmark datasets, including three synthetic datasets, one real clinical dataset, and one clinical downstream segmentation dataset, we comprehensively evaluate the effectiveness of our proposed TMSD. Extensive experiments substantiate that the integration of the proposed TMSD enables seven representative baselines to obtain better perceptual quality, especially in the cross-domain clinical generalization scenario. Besides, the downstream segmentation accuracy can be evidently improved, showing the favorable application potential of the proposed TMSD. Moreover, to comprehensively evaluate the generality of our TMSD framework, we successfully apply it to a new and classic metal artifact reduction task. It is worth mentioning that our TMSD does not incur any extra computational cost during inference.

Semi-supervised learning (SSL) has greatly advanced 3D medical image segmentation by alleviating the need for intensive labeling by radiologists. While previous efforts focused on model-centric advancements, the emergence of foundational generalist models like the Segment Anything Model (SAM) is expected to reshape the SSL landscape. Although these generalists usually show performance gaps relative to previous specialists in medical imaging, they possess impressive zero-shot segmentation abilities with manual prompts. Thus, this capability could serve as "free lunch" for training specialists, offering future SSL a promising data-centric perspective, especially revolutionizing both pseudo and expert labeling strategies to enhance the data pool. In this regard, we propose the Generalist Model-driven Active Barely Supervised (GM-ABS) learning paradigm, for developing specialized 3D segmentation models under extremely limited (barely) annotation budgets, e.g., merely cross-labeling three slices per selected scan. In specific, building upon a basic mean-teacher SSL framework, GM-ABS modernizes the SSL paradigm with two key data-centric designs: (i) Specialist-generalist collaboration, where the in-training specialist leverages class-specific positional prompts derived from class prototypes to interact with the frozen class-agnostic generalist across multiple views to achieve noisy-yet-effective label augmentation. Then, the specialist robustly assimilates the augmented knowledge via noise-tolerant collaborative learning. (ii) Expert-model collaboration that promotes active cross-labeling with notably low labeling efforts. This design progressively furnishes the specialist with informative and efficient supervision via a human-in-the-loop manner, which in turn benefits the quality of class-specific prompts. Extensive experiments on three benchmark datasets highlight the promising performance of GM-ABS over recent SSL approaches under extremely constrained labeling resources.

Cloud platforms have been virtualizing storage devices like flash-based solid-state drives (SSDs) to make effective use of storage resources. They enable either software-isolated instance or hardware-isolated instance for facilitating the storage sharing between multi-tenant applications. However, for decades, they have to combat the fundamental tussle between the performance isolation and resource utilization. They suffer from either long tail latency caused by weak isolation or low storage utilization caused by strong isolation.

Amid the rapid expansion of the Non-Fungible Token (NFT) market, X (formerly Twitter) has emerged as a crucial channel for communication between project creators and their communities. This study investigates the short-term effects of NFT project tweets on trading behaviors and price dynamics. Guided by Media Richness Theory (MRT), we con-ducted a quantitative analysis of tweets from nine leading NFT projects, categorizing them into three distinct clusters. Our findings reveal heterogeneous correlations between tweet content, NFT categories, and price fluctuations. The differing roles and functions of NFTs across categories shape both the distribution of tweets and their short-term pricing impacts. Furthermore, we employed three machine learning models using media richness as a predictive feature, achieving approximately 60 % accuracy in forecasting NFT price movements. Overall, this research highlights the predictive potential of social media for NFT price trends and its contribution to the NFT ecosystems sustainability.

Empowered by blockchain technology, smart contracts have attracted considerable interest from Web3 users due to their distinct advantages. Nevertheless, it is challenging to address problems caused by the dramatic expansion of the Web3 ecosystem. This article introduces the smart contract-as-a-service (SCaaS) paradigm to mitigate smart contracts’ redundant deployment via their composability and reusability. Moreover, we design trust and incentive schemes to ensure project security and developer engagement in SCaaS. Specifically, we first introduce a reputation filter by leveraging the authentic on-chain data, aiming to eliminate high-risk contracts. We then design a contract-based incentive mechanism to help the foundation attract heterogeneous developers with multidimensional private information, and maximize the foundation’s utility by inducing developers to undertake projects of differing complexities based on their ability. We further differentiate between veteran and newcome developers and examine their influences on foundational strategies. Finally, extensive experimental results demonstrate that our proposed contracts can efficiently remove high-risk smart contracts, maximize the foundation’s utility, and ensure that developers select contracts honestly and participate in the SCaaS ecosystem actively.

Anatomical abnormality detection and report generation of chest X-ray (CXR) are two essential tasks in clinical practice. The former aims at localizing and characterizing cardiopulmonary radiological findings in CXRs, while the latter summarizes the findings in a detailed report for further diagnosis and treatment. Existing methods often focused on either task separately, ignoring their correlation. This work proposes a co-evolutionary abnormality detection and report generation (CoE-DG) framework. The framework utilizes both fully labeled (with bounding box annotations and clinical reports) and weakly labeled (with reports only) data to achieve mutual promotion between the abnormality detection and report generation tasks. Specifically, we introduce a bi-directional information interaction strategy with generator-guided information propagation (GIP) and detector-guided information propagation (DIP). For semi-supervised abnormality detection, GIP takes the informative feature extracted by the generator as an auxiliary input to the detector and uses the generator's prediction to refine the detector's pseudo labels. We further propose an intra-image-modal self-adaptive non-maximum suppression module (SA-NMS). This module dynamically rectifies pseudo detection labels generated by the teacher detection model with high-confidence predictions by the student. Inversely, for report generation, DIP takes the abnormalities' categories and locations predicted by the detector as input and guidance for the generator to improve the generated reports. Finally, a co-evolutionary training strategy is implemented to iteratively conduct GIP and DIP and consistently improve both tasks' performance. Experimental results on two public CXR datasets demonstrate CoE-DG's superior performance to several up-to-date object detection, report generation, and unified models. Our code is available at https://github.com/jinghanSunn/CoE-DG.

Most existing federated learning (FL) methods for medical image analysis only considered intramodal heterogeneity, limiting their applicability to multimodal imaging applications. In practice, it is not uncommon that some FL participants only possess a subset of the complete imaging modalities, posing inter-modal heterogeneity as a challenge to effectively training a global model on all participants’ data. In addition, each participant would expect to obtain a personalized model tailored for its local data characteristics from the FL in such a scenario. In this work, we propose a new FL framework with federated modality-specific encoders and multimodal anchors (FedMEMA) to simultaneously address the two concurrent issues. Above all, FedMEMA employs an exclusive encoder for each modality to account for the inter-modal heterogeneity in the first place. In the meantime, while the encoders are shared by the participants, the decoders are personalized to meet individual needs. Specifically, a server with full-modal data employs a fusion decoder to aggregate and fuse representations from all modality-specific encoders, thus bridging the modalities to optimize the encoders via backpropagation reversely. Meanwhile, multiple anchors are extracted from the fused multimodal representations and distributed to the clients in addition to the encoder parameters. On the other end, the clients with incomplete modalities calibrate their missing-modal representations toward the global full-modal anchors via scaled dot-product cross-attention, making up the information loss due to absent modalities while adapting the representations of present ones. FedMEMA is validated on the BraTS 2020 benchmark for multimodal brain tumor segmentation. Results show that it outperforms various up-to-date methods for multimodal and personalized FL and that its novel designs are effective. Our code is available.

Modular data centers (MDCs) that can be placed right at the energy farms and powered mostly by renewable energy, is a flexible and effective approach to lowering the carbon footprint of data centers. However, the main challenge of using renewable energy is the high variability of power produced, which implies large volatility in powering computing resources at MDCs, and degraded application performance due to the task evictions and migrations. This causes challenges for platform operators to decide the MDC deployment.

In the rapidly evolving landscape of blockchain technology, security emerges as a paramount concern. This paper introduces an innovative blockchain security threat awareness platform, designed to comprehensively address the multifaceted security challenges within blockchain networks, particularly focusing on Ethereum contracts. Central to the platform is a dual-database architecture, blending a NoSQL database with a graph database, enhancing data management, and enabling intricate transaction network visualizations. The platform's Threat Detection module, utilizing Large Language Models (LLMs) in conjunction with traditional methods, offers a novel approach to identifying and categorizing vulnerabilities in Ethereum smart contracts. Complementing this, the Threat Evidence Collection module provides detailed post-attack analysis, tracing transactions to their sources and evaluating address risks. This module's capabilities extend to producing statistical reports, including the transactional history and risk evaluation of individual addresses. Demonstrated on the Ethereum blockchain, the platform showcases its proficiency in handling complex data, rapid threat detection, and extensive forensic analysis, presenting a robust solution to fortifying blockchain security and offering a proactive defense mechanism for users and developers in the blockchain environment.

As a bridge connecting the Web3 financial ecosystem and digital games, smart contracts empowered blockchain games have attracted significant attention from the Web3 community in recent years. By providing players ownership over assets and interoperable Non-Fungible Tokens (NFTs), blockchain games enable the reuse of in-game assets beyond the original games, thereby overturning the “walled garden” among traditional games. Nonetheless, blockchain games diminish the monopolistic edge previously held by traditional game providers, forcing them to compete with players by token distribution. Therefore, this paper explores the duopoly competition within the blockchain game market, emphasizing the role of interoperable NFTs together with NFT wear and tear. Specifically, we propose a three-stage game to formulate the interactions between game providers and players. Besides, we revealed the relationship between game providers' code disclosure strategies for NFT interoperability and token retention strategies. Finally, the experimental results demonstrate how the token distribution, players' preferences, and the NFT wear level affect the profits of game providers.

Multimodal magnetic resonance imaging (MRI) provides complementary information for sub-region analysis of brain tumors. Plenty of methods have been proposed for automatic brain tumor segmentation using four common MRI modalities and achieved remarkable performance. In practice, however, it is common to have one or more modalities missing due to image corruption, artifacts, acquisition protocols, allergy to contrast agents, or simply cost. In this work, we propose a novel two-stage framework for brain tumor segmentation with missing modalities. In the first stage, a multimodal masked autoencoder (M3AE) is proposed, where both random modalities (i.e., modality dropout) and random patches of the remaining modalities are masked for a reconstruction task, for self-supervised learning of robust multimodal representations against missing modalities. To this end, we name our framework M3AE. Meanwhile, we employ model inversion to optimize a representative full-modal image at marginal extra cost, which will be used to substitute for the missing modalities and boost performance during inference. Then in the second stage, a memory-efficient self distillation is proposed to distill knowledge between heterogenous missing-modal situations while fine-tuning the model for supervised segmentation. Our M3AE belongs to the ‘catch-all’ genre where a single model can be applied to all possible subsets of modalities, thus is economic for both training and deployment. Extensive experiments on BraTS 2018 and 2020 datasets demonstrate its superior performance to existing state-of-the-art methods with missing modalities, as well as the efficacy of its components. Our code is available at: https://github.com/ccarliu/m3ae.

In modern solid-state drives (SSDs), the indexing of flash pages is a critical component in their storage controllers. It not only affects the data access performance, but also determines the efficiency of the precious in-device DRAM resource. A variety of address mapping schemes and optimizations have been proposed. However, most of them were developed with human-driven heuristics.

Thanks to the mature manufacturing techniques, flash-based solid-state drives (SSDs) are highly customizable for applications today, which brings opportunities to further improve their storage performance and resource utilization. However, the SSD efficiency is usually determined by many hardware parameters, making it hard for developers to manually tune them and determine the optimized SSD hardware configurations.

This article presents a systematic study on the security of modern file systems, following a vulnerability-centric perspective. Specifically, we collected 377 file system vulnerabilities committed to the CVE database in the past 20 years. We characterize them from four dimensions: why the vulnerabilities appear, how the vulnerabilities can be exploited, what consequences can arise, and how the vulnerabilities are fixed. This way, we build a deep understanding of the attack surfaces faced by file systems, the threats imposed by the attack surfaces, and the good and bad practices in mitigating the attacks in file systems. We envision that our study will bring insights towards the future development of file systems, the enhancement of file system security, and the relevant vulnerability-mitigating solutions.

Unsupervised pretraining based on contrastive learning has made significant progress recently and showed comparable or even superior transfer learning performance to traditional supervised pretraining on various tasks. In this work, we first empirically investigate when and why unsupervised pretraining surpasses supervised counterparts for image classification tasks with a series of control experiments. Besides the commonly used accuracy, we further analyze the results qualitatively with the class activation maps and assess the learned representations quantitatively with the representation entropy and uniformity. Our core finding is that it is the amount of information effectively perceived by the learning model that is crucial to transfer learning, instead of absolute size of the dataset. Based on this finding, we propose Classification Activation Map guided contrastive (CAMtrast) learning which better utilizes the label supervsion to strengthen supervised pretraining, by making the networks perceive more information from the training images. CAMtrast is evaluated with three fundamental visual learning tasks: image recognition, object detection, and semantic segmentation, on various public datasets. Experimental results show that our CAMtrast effectively improves the performance of supervised pretraining, and that its performance is superior to both unsupervised counterparts and a recent related work which similarly attempted improving supervised pretraining.

Variations in the performance of parallel and distributed systems are becoming increasingly challenging. The runtimes of different executions can vary greatly even with a fixed number of computing nodes. Many HPC applications on supercomputers exhibit such variance. This not only leads to unpredictable execution times, but also renders the system’s behavior unintuitive. The efficient online detection of variations in performance is an open problem in HPC research. To solve it, we propose an approach, calledvSensor, to detect variations in the performance of systems. The key finding of this study is that the source code of programs can better represent performance at runtime than an external detector. Specifically, many HPC applications contain code snippets that are fixed workload patterns of execution, e.g., the workload of an invariant quantity and a linearly growing workload. This observation allows us to automatically identify these snippets of workload-related code and use them to detect variations in performance. We evaluatevSensoron the Tianhe-2A system with a large number of parallel applications, and the results indicate that it can efficiently identify variations in system performance. The average overhead of 4,096 processes is less than 6% for fixed-workload v-sensors. We identify a problematic node with slow memory by usingvSensorthat degrades the performance of the program by 21%. A serious issue with network performance is also detected that slows down the Tianhe-2A system by 3.37 times for an HPC kernel.

Renewable energy is becoming an important power source for data centers, especially with the zero-carbon waste pledges made by big cloud providers. However, one of the main challenges of renewable energy sources is the high variability of power produced. Traditional approaches such as batteries or transmitting to the grid fall short on scale, overhead, or "green-ness". We propose Virtual Battery: instead of adapting the availability of power to match the computation demand we shift computational demand to meet the availability of power. Virtual batteries shift demand by requiring applications to either be flexible and delay-tolerant or proactively migrating to where power is (going to be) available. We show that using multiple virtual battery sites in combination can meet the needs of modern applications. Moreover, we show how an intelligent network and power aware co-scheduler can not only provide availability despite variability but also help mitigate migration related network overhead by over 30% in total and 4.2x at peak.

We present ParaCrash, a testing framework for studying crash recovery in a typical HPC I/O stack, and demonstrate its use by identifying 15 new crash-consistency bugs in various parallel file systems (PFS) and I/O libraries. ParaCrash uses a "golden version" approach to test the entire HPC I/O stack: storage state after recovery from a crash is correct if it matches the state that can be achieved by a partial execution with no crashes. It supports systematic testing of a multilayered I/O stack while properly identifying the layer responsible for the bugs.

Byte-addressable, non-volatile memory (NVM) is emerging as a promising technology. To facilitate its wide adoption, employing NVM in managed runtimes like JVM has proven to be an effective approach (i.e., managed NVM). However, such an approach is runtime specific, it lacks a generic abstraction across different managed languages. Similar to the well-known filesystem primitives that allow diverse programs to access the same file via the block I/O interface, managed NVM deserves the same system-wide property for persistent objects across managed runtimes with low overhead.

Pedestrian trajectory prediction is vital for transportation systems. Generally we can divide pedestrian behavior modeling into two categories, i.e., knowledge-driven and data-driven. The former might bring expert bias, and it sometimes generates unrealistic pedestrian movement due to unnecessary repulsive forces. The latter approach is popular nowadays but most existing neural networks, including fully connected long short-term memory (LSTM) networks, use a 1D vector to model their input and state. The shortcoming is that these works cannot learn spatial information about pedestrians, especially in a dense crowd. To tackle this, we propose to use tensors to represent essential environment features of pedestrians. Accordingly, a convolutional LSTM is designed and deepened to predict spatiotemporal trajectory sequences. As the tensor and convolution can learn better spatiotemporal interactions among pedestrians and environments, experimental results show that the proposed network can estimate more realistic trajectories for a dense crowd in evacuation and counterflow.

Pedestrian evacuation simulation is vital for urban civil engineers. Although there exist many works addressing the issue of emergency evacuation, only a few study the phenomenon of people actively squeezing to pass through an exit. To model this behavior, a three-circle model is adopted to represent the shape of pedestrians. Active rotation torque (ART) is proposed to model the active rotation behavior of pedestrians turning their torsos in the desired direction. This torque occurs either when a pedestrian is not facing his velocity direction, or when he wants to pass through a bottleneck. The equation of ART is designed and regressed with real pedestrian experiments, in which a gyroscope was used to measure the angle of torso rotation. The proposed torque model is then applied to manifold scenarios with various door widths and different safety separation belt settings. Then, both microscopic and macroscopic indexes, including evacuation time, rotation angle, and crowd density, are obtained to show that the proposed model can simulate both non-competitive and competitive pedestrian behaviors near exit bottlenecks more accurately than the circular social force model. Thus, the evacuation time of the exit can be predicted more precisely, which helps to design optimal multi-exit assignment strategies.