Facial Action Unit (AU) recognition involves identifying subtle muscle movements corresponding to different AUs. Recent approaches have focused on localizing AUs using predefined Regions of Interest (RoIs) or learnable modules. However, these methods either overly depend on the precision of predefined RoIs or inaccurately localize background regions instead of the actual AU positions. To address this challenge, we propose a novel method, which automatically localizes each AU without relying on predefined RoIs or introducing learnable modules during the inference phase. Specifically, our approach decomposes the task into two subtasks: AU localization and AU state verification. We first align the direction between spatial features and the corresponding AU class weights to guide the model in localizing AUs. Next, we incorporate spatial and temporal aspects for precise AU state detection. From the perspective of spatial information learning, we propose a confidence-based AU relationship mining module that directs the model to focus on uncertain AUs. From the aspect of temporal information learning, we introduce a temporal sampling strategy that implicitly captures time-dependent features. Experimental results on the BP4D and DISFA datasets demonstrate the effectiveness of our method, showing that it outperforms existing approaches and achieves state-of-the-art performance in AU recognition.

Weakly supervised semantic segmentation (WSSS) leverages weak annotations to train semantic segmentation networks, thereby reducing the cost of pixel-level labeling. However, prevalent WSSS methods rely on Class Activation Map (CAM), which only focus on the most discriminative parts of a class, resulting in incomplete and inaccurate pseudo-labels. To resolve this, we propose a Beta Modulation (BM) method, which introduces multiple fixed forms of the Beta distribution to achieve dynamic attention to different activation level of features. BM provides supplementary information for overlooked regions and enhances the model’s ability to recognize objects as a whole. Additionally, we propose a Cross-Modality Alignment (CA) strategy to better differentiate features of semantically similar classes. It contrasts language descriptions of confusing false positive classes with global image features, yielding more distinctive and robust class representations. Extensive experiments conduct on the VOC12 natural dataset and BCSS medical dataset demonstrate state-of-the-art performance, with 83.90% mIoU on the VOC12 test set and 73.18% mIoU on the BCSS test set.

Histopathological whole-slide images (WSIs) segmentation is essential for precise tissue characterization in medical diagnostics. However, traditional approaches require labor-intensive pixel-level annotations. To this end, we study weakly supervised semantic segmentation (WSSS) which uses patch-level classification labels, reducing annotation efforts significantly. However, the complexity of WSIs and the challenge of sparse classification labels hinder effective dense pixel predictions. Moreover, due to the multi-label nature of WSI, existing approaches of single-label contrastive learning designed for the representation of single-category, neglecting the presence of other relevant categories and thus fail to adapt to WSI tasks. This paper presents a novel multi-label contrastive learning method for WSSS by incorporating class-specific embedding extraction with LLM features guidance. Specifically, we propose to obtain class-specific embeddings by utilizing classifier weights, followed by a dot-product-based attention fusion method that leverages LLM features to enrich their semantics, facilitating contrastive learning between different classes from single image. Besides, we propose a Robust Learning approach that leverages multi-layer features to evaluate the uncertainty of pseudo-labels, thereby mitigating the impact of noisy pseudo-labels on the learning process of segmentation. Extensive experiments have been conducted on two histopathological image segmentation datasets, i.e. LUAD dataset and BCSS dataset, demonstrating the effectiveness of our methods with leading performance.