Semi-supervised semantic segmentation (SSSS) is vital in computational pathology, where dense annotations are costly and limited. Existing methods often rely on pixel-level consistency, which propagates noisy pseudo-labels and produces fragmented or topologically invalid masks. We propose Topology Graph Consistency (TGC), a framework that integrates graph-theoretic constraints by aligning Laplacian spectra, component counts, and adjacency statistics between prediction graphs and references. This enforces global topology and improves segmentation accuracy. Experiments on GlaS and CRAG demonstrate that TGC achieves state-of-the-art performance under 5–10% supervision and significantly narrows the gap to full supervision.

To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction.

Prompt-based techniques, such as prompt-tuning and prefix-tuning, have gained prominence for their efficiency in fine-tuning large pre-trained models. Despite their widespread adoption, the theoretical foundations of these methods remain limited. For instance, in prefix-tuning, we observe that a key factor in achieving performance parity with full fine-tuning lies in the reparameterization strategy. However, the theoretical principles underpinning the effectiveness of this approach have yet to be thoroughly examined. Our study demonstrates that reparameterization is not merely an engineering trick but is grounded in deep theoretical foundations. Specifically, we show that the reparameterization strategy implicitly encodes a shared structure between prefix key and value vectors. Building on recent insights into the connection between prefix-tuning and mixture of experts models, we further illustrate that this shared structure significantly improves sample efficiency in parameter estimation compared to non-shared alternatives. The effectiveness of prefix-tuning across diverse tasks is empirically confirmed to be enhanced by the shared structure, through extensive experiments in both visual and language domains. Additionally, we uncover similar structural benefits in prompt-tuning, offering new perspectives on its success. Our findings provide theoretical and empirical contributions, advancing the understanding of prompt-based methods and their underlying mechanisms.

LLaMA-Adapter has recently emerged as an efficient fine-tuning technique for LLaMA models, leveraging zero-initialized attention to stabilize training and enhance performance. However, despite its empirical success, the theoretical foundations of zero-initialized attention remain largely unexplored. In this paper, we provide a rigorous theoretical analysis, establishing a connection between zero-initialized attention and mixture-of-expert models. We prove that both linear and non-linear prompts, along with gating functions, can be optimally estimated, with non-linear prompts offering greater flexibility for future applications. Empirically, we validate our findings on the open LLM benchmarks, demonstrating that non-linear prompts outperform linear ones. Notably, even with limited training data, both prompt types consistently surpass vanilla attention, highlighting the robustness and adaptability of zero-initialized attention.

Low-rank Adaptation (LoRA) has emerged as a powerful and efficient method for fine-tuning large-scale foundation models. Despite its popularity, the theoretical understanding of LoRA has remained underexplored. In this paper, we present a theoretical analysis of LoRA by examining its connection to the Mixture of Experts models. Under this framework, we show that a simple technique, reparameterizing LoRA matrices, can notably accelerate the low-rank matrix estimation process. In particular, we prove that reparameterization can reduce the data needed to achieve a desired estimation error from an exponential to a polynomial scale. Motivated by this insight, we propose Reparameterized Low-Rank Adaptation (RepLoRA), incorporating a lightweight MLP to reparameterize the LoRA matrices. Extensive experiments across multiple domains demonstrate that RepLoRA consistently outperforms vanilla LoRA. With limited data, RepLoRA surpasses LoRA by a substantial margin of up to 40.0% and achieves LoRA’s performance using only 30.0% of the training data, highlighting the theoretical and empirical robustness of our PEFT method.

Exploiting the power of pre-trained models, prompt-based approaches stand out compared to other continual learning solutions in effectively preventing catastrophic forgetting, even with very few learnable parameters and without the need for a memory buffer. While existing prompt-based continual learning methods excel in leveraging prompts for state-of-the-art performance, they often lack a theoretical explanation for the effectiveness of prompting. This paper conducts a theoretical analysis to unravel how prompts bestow such advantages in continual learning, thus offering a new perspective on prompt design. We first show that the attention block of pre-trained models like Vision Transformers inherently encodes a special mixture of experts architecture, characterized by linear experts and quadratic gating score functions. This realization drives us to provide a novel view on prefix tuning, reframing it as the addition of new task-specific experts, thereby inspiring the design of a novel gating mechanism termed Non-linear Residual Gates (NoRGa). Through the incorporation of non-linear activation and residual connection, NoRGa enhances continual learning performance while preserving parameter efficiency. The effectiveness of NoRGa is substantiated both theoretically and empirically across diverse benchmarks and pretraining paradigms. Our code is publicly available at https://github.com/Minhchuyentoancbn/MoE_PromptCL.

The ability to detect edges is a fundamental attribute necessary to truly capture visual concepts. In this paper, we show that edges cannot be represented properly in the first layer of a conventional convolutional neural network, and further show that they are poorly captured in two popular neural network architectures VGG-16 and ResNet. The neural networks are found to rely on color information, which might vary in unexpected ways outside of the datasets used for their evaluation. To improve their robustness, we propose edge-detection units and show that they reduce performance loss and generate qualitatively different representations. By comparing various models, we show that the robustness of edge detection is an important factor contributing to the robustness of models against color noise.

LSTM-based language models have been shown effective in Word Sense Disambiguation (WSD). In particular, the technique proposed by Yuan et al. (2016) returned state-of-the-art performance in several benchmarks, but neither the training data nor the source code was released. This paper presents the results of a reproduction study and analysis of this technique using only openly available datasets (GigaWord, SemCor, OMSTI) and software (TensorFlow). Our study showed that similar results can be obtained with much less data than hinted at by Yuan et al. (2016). Detailed analyses shed light on the strengths and weaknesses of this method. First, adding more unannotated training data is useful, but is subject to diminishing returns. Second, the model can correctly identify both popular and unpopular meanings. Finally, the limited sense coverage in the annotated datasets is a major limitation. All code and trained models are made freely available.

Computation offloading has received much attention to improve the performance or energy efficiency of mobile systems that have usually limited and constrained resource capacities. Yet, applying the computation offloading technique in opportunistic networks that are highly dynamic and often become volatile still remains a challenge due to the following reasons: (1) technical difficulties in constructing efficient and reliable execution environment using commodity devices using WiFi, (2) a lack of runtime support for multiple clients that request diverse computational tasks and execute them concurrently with minimum performance impacts. In this paper, we introduce a new middleware system that provides an offloading framework operated in opportunistic networks. In particular, our middleware employs a publish-subscribe communication mechanism to provide multiple different communication models (e.g., one-to-one, one-to-many, many-to-one, and many-to-many) for different use cases. Furthermore, when distributing computational tasks to nearby nodes, our middleware takes their resource capabilities into consideration for efficient execution. Finally, since partial failure is an unavoidable artifact in highly dynamic and volatile opportunistic networks, we provide a simple, but effective failure handling mechanism. Our benchmarks and experimental results indicate that our approach enables programmers to easily apply computation offloading techniques in opportunistic networks when compared with the local execution.

Error propagation is a common problem in NLP.Reinforcement learning explores erroneous states during training and can therefore be more robust when mistakes are made early in a process.In this paper, we apply reinforcement learning to greedy dependency parsing which is known to suffer from error propagation.Reinforcement learning improves accuracy of both labeled and unlabeled dependencies of the Stanford Neural Dependency Parser, a high performance greedy parser, while maintaining its efficiency.We investigate the portion of errors which are the result of error propagation and confirm that reinforcement learning reduces the occurrence of error propagation.

Code offloading on mobile platforms has received much attention as a way of relieving heavy workload by utilizing power of the other devices or cloud servers. The offloading mechanisms rely on multiple platforms to be enabled on variety of mobile devices. To support platform heterogeneity, the execution code should be implemented in JavaScript and executed on the designate devices by correspondingly compatible JavaScript engine. In this paper, we present a novel distribution mechanism with a built-in JavaScript execution package, annotation processor and an engine to enable code offloading among the devices and servers in peer-to-peer (P2P) networks. This approach includes a set of constraints for code implementation so developers can easily integrate to their project. Our evaluation, based on a testbed with Android and Windows Phone devices, demonstrates the efficiency of offloading JavaScript-based packages on multiple devices, as well as compares the performance between JavaScript and native versions.

Indium phosphide heterojunction bipolar transistors (HBTs) find applications in very wide-band digital and mixed-signal integrated circuits (ICs). Devices fabricated in high-yield process flows at 500 nm feature size obtain 450 GHz cutoff frequencies and 5 V breakdown and enable high yield fabrication of integrated circuits having more than 3000 transistors. Laboratory devices at 250 nm feature size obtain 755 GHz . We describe device and circuit bandwidth limits associated with HBTs, develop scaling roadmaps for HBTs having lithographic minimum feature sizes between 512 and 64 nm, and identify key technological challenges in realizing 480-GHz digital ICs and 1000-GHz amplifiers. Key features of manufacturable self-aligned dielectric sidewall processes are described in detail.

In the above titled paper (ibid., vol. 96, no. 2, pp. 271-286, Feb 08), there are errors. Corrections are presented here.

Surface transportation systems play a crucial role in responding to natural disasters and other catastrophic incidents that have devastating impacts on the lives of people all over the world. Intelligent transportation systems (ITS) can play an important role not only in improving the operational efficiency of a transportation system but also in enhancing its safety and security. In this paper, we propose new techniques for augmenting ITS to improve and support homeland security. In particular, we propose two evacuation algorithms, i.e., all-links and fastest-links, and perform simulation studies to compare their performance. These algorithms are part of a smart traffic evacuation management system (STEMS) developed to provide rapid and efficient response to human-caused threats and disasters by automatically generating dynamic evacuation plans based on incident location and scope and subsequently automatically controlling traffic lights to direct evacuation traffic in a safe manner.

We present a design for a system that provides video-on-demand (VOD) services to mobile ad hoc clients. Such a system allows the clients to access video information anytime anywhere. MobiVoD, the proposed solution, overcomes many difficulties currently challenging video streaming in a mobile ad hoc network. The new environment includes a three-tier architecture, in which the mobile VOD system employs a periodic broadcast protocol to achieve maximum scalability; and the clients leverage an ad hoc network caching technique to minimize the service delay. This system can sustain client failure and mobility, and provide true VOD services to most clients.