Recovering text from sealed envelopes poses a severe security threat to physical privacy. While recent imaging advances can see through paper layers to recover general textures, they fail to reconstruct legible text due to the high-frequency detail and semantic structure required for OCR. To address this, we present the first unified pipeline for Physical Envelope Text Recovery (PETR), covering both attack and defense mechanisms. On the attack side, we design two entirely different recovery architectures: ETNN and ETFormer. In particular, we propose a Text-Aware Loss that injects text-specific guidance into the restoration process, where constraints on both text content and text structure jointly drive gradient back-propagation to enhance structural clarity. We also introduce an LLM-based evaluation protocol to measure the semantic readability of recovered content. On the defense side, we analyze these attack vectors to design secure envelopes using localized disruption patterns that impede recovery without affecting human readability after opening. Extensive experiments show our models outperform state-of-the-art baselines in both fidelity (PSNR/SSIM) and OCR accuracy. The source code and dataset are available on the project page https://github.com/YongfTao/PETR.

Text labels are widely used to convey auxiliary information in visualization and graphic design. The substantial variability in the categories and structures of labeled objects leads to diverse label layouts. Recent single-model learning-based solutions in label placement struggle to capture fine-grained differences between these layouts, which in turn limits their performance. In addition, although human designers often consult previous works to gain design insights, existing label layouts typically serve merely as training data, limiting the extent to which embedded design knowledge can be exploited. To address these challenges, we propose a mixture of cluster-guided experts (MoCE) solution for label placement. In this design, multiple experts jointly refine layout features, with each expert responsible for a specific cluster of layouts. A cluster-based gating function assigns input samples to experts based on representation clustering. We implement this idea through the Label Placement Cluster-guided Experts (LPCE) model, in which a MoCE layer integrates multiple feed-forward networks (FFNs), with each expert composed of a pair of FFNs. Furthermore, we introduce a retrieval augmentation strategy into LPCE, which retrieves and encodes reference layouts for each input sample to enrich its representations. Extensive experiments demonstrate that LPCE achieves superior performance in label placement, both quantitatively and qualitatively, surpassing a range of state-of-the-art baselines. Our algorithm is available at https://github.com/PingshunZhang/LPCE.

Medical image segmentation aims to identify anatomical structures at the voxel-level. Segmentation accuracy relies on distinguishing voxel differences. Compared to advancements achieved in studies of the inter-class variance, the intra-class variance receives less attention. Moreover, traditional linear classifiers, limited by a single learnable weight per class, struggle to capture this finer distinction. To address the above challenges, we propose a Multi-Prototype-based Embedding Refinement method for semi-supervised medical image segmentation. Specifically, we design a multi-prototype-based classification strategy, rethinking the segmentation from the perspective of structural relationships between voxel embeddings. The intra-class variations are explored by clustering voxels along the distribution of multiple prototypes in each class. Next, we introduce a consistency constraint to alleviate the limitation of linear classifiers. This constraint integrates different classification granularities from a linear classifier and the proposed prototype-based classifier. In the thorough evaluation on two popular benchmarks, our method achieves superior performance compared with state-of-the-art methods. Code is available at https://github.com/Briley-byl123/MPER.

Image inpainting aims to repair digital images with defects such as holes and scratches at both semantic and textural levels. Diffusion models have shown great success in image inpainting, delivering high-quality results. However, existing diffusion-based methods often overlook the shape of defective regions/masks, applying a uniform sampling strategy across varying shapes. This oversight may lead to low-quality or semantically inappropriate restored images. In this paper, we propose MAD-paint (Mask-Aware Diffusion sampling for inpainting), and show that applying different noise types tailored to specific defect regions/mask shapes during the reverse diffusion process can significantly improve the inpainting quality. We begin by introducing a metric for mask uncertainty to assess the impact of different masks on inpainting quality. Using this metric, we propose a mask-aware sampling approach that automatically adjusts its sampling strategy according to different mask shapes, as indicated by the mask uncertainty. In addition, leveraging the known image texture consistency, we propose a known region-guided iterative refinement mechanism to condition texture restoration. The experimental results demonstrate the advantages of our method over other diffusion-based inpainting methods.

Placing text labels is a common way to explain key elements in a given scene. Given a graphic input and original label information, how to place labels to meet both geometric and aesthetic requirements is an open challenging problem. Geometry-wise, traditional rule-driven solutions struggle to capture the complex interactions between labels, let alone consider graphical/appearance content. In terms of aesthetics, training/evaluation data ideally require nontrivial effort and expertise in design, thus resulting in a lack of decent datasets for learning-based methods. To address the above challenges, we formulate the task with a graph representation, where nodes correspond to labels and edges to interactions between labels, and treat label placement as a node position prediction problem. With this novel representation, we design a Label Placement Graph Transformer (LPGT) to predict label positions. Specifically, edge-level attention, conditioned on node representations, is introduced to reveal potential relationships between labels. To integrate graphic/image information, we design a feature aligning strategy that extracts deep features for nodes and edges efficiently. Next, to address the dataset issue, we collect commercial illustrations with professionally designed label layouts from household appliance manuals, and annotate them with useful information to create a novel dataset named the Appliance Manual Illustration Labels (AMIL) dataset. In the thorough evaluation on AMIL, our LPGT solution achieves promising label placement performance compared with popular baselines. Our algorithm and dataset are available at https://github.com/JingweiQu/LPGT.

Semantic correspondence aims to establish dense correspondences between semantically similar images. Multilevel image features have been commonly used in recent studies due to their rich information. However, this approach poses a challenging problem of how to distinguish the importance of multiple similarity scores for each candidate match. Moreover, the introduction of the level dimension increases ambiguous matches. To address these challenges, we develop a Multi-granularity Inter-level Attention-based Matching (MIAM) network. Specifically, multi-scale inter-level self-attention, conditioned on correlation patches of various sizes, is proposed to adjust the effect of similarities from different levels on building correspondences. Next, we introduce a dual dimensional re-weighting strategy to further alleviate the ambiguity issue. Based on the convolutional aggregation of the multi-level scores along the spatial and level dimensions, this strategy strengthens positive matches while suppressing negative ones. In the thorough evaluation on three semantic correspondence benchmarks, MIAM achieves competitive performance compared to popular methods. Our algorithm is available at https://github.com/2000LZZ/MIAM.

The binding affinity between drugs and proteins is a substantial part of the drug discovery process. Graph neural networks (GNNs) have shown great promise in graph-related structure by learning the representations of graphs, which are suitable for tasks such as binding affinity prediction. However, most of the existing GNN architectures only pay attention to the information flow on a single graph, while the interaction between two graphs is unconcerned. In this paper, we propose an attention-enhanced graph cross-convolution network (GCAT) to explore binding affinity on pure 3D atomistic geometry. It consists of two components: cross-convolution and self-attention pooling. Specifically, cross-convolution performs an aggregate-update mechanism to simulate the interaction between the protein and the drug, then self-attention pooling is adopted to capture global interactions and get graph-level representations. Extensive experiments conducted on the PDB-Bind dataset demonstrate the effectiveness of our GCAT.

Drug repurposing aims at identifying new indications for approved drugs that are outside the scope of the original indications. Understanding the acting mechanism among drugs, protein targets, and diseases, especially the interdependent and indecomposable relationships, is a critical step. However, most existing methods rely on pairwise relationships. To model the biological interactions between the three types of entities, which are likely ignored by the pairwise paradigm, we propose an end-to-end Event-Graph Neural Network (EGNN) to predict multivariate relationships of drugs, targets, and diseases for drug repurposing. Specifically, we introduce the event to describe the interdependence of drug-target-disease as a complete semantic unit and design the Event-Graph to model the multivariate relationships. To predict the potential relationships, we perform the representation learning on the Event-Graph by a bidirectional aggregating operation and an event-level attention mechanism. Experimental results on real-world datasets demonstrate the effectiveness and promising performance of EGNN compared with several competitive methods.

Graph matching aims at establishing correspondence between node sets of given graphs while keeping the consistency between their edge sets. However, outliers in practical scenarios and equivalent learning of edge representations in deep learning methods are still challenging. To address these issues, we present an Edge Attention-adaptive Graph Matching (EAGM) network and a novel description of edge features. EAGM transforms the matching relation between two graphs into a node and edge classification problem over their assignment graph. To explore the potential of edges, EAGM learns edge attention on the assignment graph to 1) reveal the impact of each edge on graph matching, as well as 2) adjust the learning of edge representations adaptively. To alleviate issues caused by the outliers, we describe an edge by aggregating the semantic information over the space spanned by the edge. Such rich information provides clear distinctions between different edges (e.g., inlier-inlier edges vs. inlier-outlier edges), which further distinguishes outliers in the view of their associated edges. Extensive experiments demonstrate that EAGM achieves promising matching quality compared with state-of-the-arts, on cases both with and without outliers. Our source code along with the experiments is available at https://github.com/bestwei/EAGM.

Potential drug-disease association prediction is important to facilitate drug discovery. However, most of existing drug-disease association prediction approaches rely on assembling multiple drug (disease)-related biological information, which is usually not comprehensively available, and they always fail to explore the latent information in drug-disease network. To tackle these challenges, we propose a graph neural network-based method for drug-disease association prediction, dubbed GNDD, with capturing the complex information between drugs and diseases dispense with any side information. Specifically, GNDD introduces the idea of collaborative filtering in recommendation system to avoid the dependency on multi-data. Furthermore, an embedding propagation strategy is exploited to model the high-order relationships in drug-disease network. We conduct experiments on the Comparative Toxicogenomics Database, demonstrating the effectiveness of our method in drug-disease association prediction.

Chemical Structural Formula(CSF) recognition plays an important role in the molecular design and component retrieval. However, sketch-based CSF recognition remains an obstacle in current retrieval systems. This paper introduces a system for sketch CSF recognition on smart mobile devices. A dual-mode-based method is proposed to distinguish the gestures for character inputs and non-character inputs instead of the ordinary segmentation approaches. An attribute graph model is established to describe effectively all necessary information of a sketched CSF. Chemical knowledge is adopted to refine the candidates of structure relationship among elements. The experiments results demonstrate that the proposed method outperforms the existing methods for free-sketch CSFs on effectiveness and flexibility.

Query-by-figure is an effective retrieval approach for educational documents. However, complex geometric diagrams in the field of mathematics education remain as obstacles in current retrieval systems. This study aims to explore a query method for plane geometric figures (PGFs) via sketched figures on smart mobile devices. We adopt an undirected graph model to describe PGFs and a divide-and-conquer strategy to analyze the relationships among strokes. Our main contribution is the detailed analysis of the stroke intersection that frequently occurs in PGFs. Numerous accurate elements obtained through overlapping analysis are then selected to construct strong descriptors for PGFs. Only the compressed query features, instead of a query figure, are transmitted to an image-based retrieval system located on a remote server, where the sketched PGF is finally recognized with low delay response. The experiments show that the proposed method achieves high efficiency and provides users with good interactive experience.

Multimedia education is playing a significant and increasing role for education purposes, thus leading to a large number of electronic documents. Plane geometry figures (PGFs), as important components of these documents, are regarded as very helpful information to most retrieval systems in the field of mathematics education. However, the burdensome work of annotation has become one of the chief obstacles to improve the efficiency of retrieval systems. In this paper, we introduce an active learning-based frame to select candidate instances for training the classifiers in retrieval systems, which are an emerging non-text-based information systems. In addition, an enhanced uncertainty measure and the selection of specific features of PGFs are proposed for our active learning algorithm. Comparative experiment results indicate that the proposed method effectively improves the performance of the PGF retrieval system and reduces the burdensome annotation workload.

Digital education is serving an increasingly important function in most educational institutions, thus resulting in the production of a large number of digital documents online for education purposes. However, convenient ways to retrieve mathematic geometry questions are lacking because current retrieval systems largely rely on keywords instead of geometry figure images. This study focuses on plane geometry figure (PGF) image retrieval with the aim of retrieving relevant geometry images that contain more structural information than a question text stem. To fully use geometrical properties, a Bag-of-shapes (BoS) method is proposed to build the feature descriptor of an image. The BoS method contains either basic geometric primitives or dual-primitive structures along with several specific geometrical features for shape description. Based on the BoS feature descriptor, we apply cosine similarity with group feature weight as vector similarity measure for ranking to achieve high efficiency. For a PGF image query, the retrieval results are provided in an appropriate ranking order, which has high visual similarity with respect to human perception. Retrieval experiments and evaluation results show the effectiveness and efficiency of the proposed BoS shape descriptor.

With the development of computer-aided education and digital library, there have emerged large numbers of digital documents online for education purposes. However, it is far from convenient to retrieve mathematic geometry questions because current retrieval systems largely rely on keywords instead of geometry figure images. We focus on plane geometry figure (PGF) image retrieval aiming at retrieving relevant geometry images that hold more similar geometric attributes and structure properties than a question text stem. Motivated by Attribute Graph (AG), and aiming to catch more delicate local geometric attributes and overall structure layout, we propose a Bilayer Geometric Attributed Graph (Bilayer-GAG) matching method to retrieve the relevant PGF images. The root node of Bilayer-GAG catches the spatial relationships among its children - the graph elements of the second layer, the second layer contains curvilinear geometric primitives and linear nested AGs that consist of nodes and edges with geometric signatures. Then we calculate the overall matching cost in three perspectives and finally retrieve top-k relevant Bilayer-GAGs. For a PGF image query, the retrieval results are shown in an appropriate ranking order, which has high visual similarity with respect to human perception. Retrieval experiments results show the effectiveness and efficiency of the proposed Bilayer-GAG.