With the accumulation of resources in the era of Big Data and the rise of pre-trained models in deep learning, optimizing neural networks for various tasks often involves different strategies for fine-tuning pre-trained models versus training from scratch. However, existing optimizers primarily focus on reducing the loss function by updating model parameters, without fully addressing the unique demands of these two major paradigms. In this paper, we propose DualOpt, a novel approach that decouples optimization techniques specifically tailored for these distinct training scenarios. For training from scratch, we introduce real-time layer-wise weight decay, designed to enhance both convergence and generalization by aligning with the characteristics of weight updates and network architecture. For more importantly fine-tuning, we integrate weight rollback with the optimizer, incorporating a rollback term into each weight update step. This ensures consistency in the weight distribution between upstream and downstream models, effectively mitigating knowledge forgetting and improving fine-tuning performance. Additionally, we extend the layer-wise weight decay to dynamically adjust the rollback levels across layers, adapting to the varying demands of different downstream tasks. Extensive experiments across diverse tasks, including image classification, object detection, semantic segmentation, and instance segmentation, demonstrate the broad applicability and state-of-the-art performance of DualOpt.

Computer-aided diagnosis greatly enhances personalized treatment planning and diagnostic efficiency by providing accurate dental anatomy through teeth segmentation. However, it still constrained by the scarcity of high-quality annotated dental datasets. To address this issue, this paper presents a dataset combining both 2D panoramic X-rays with over 6,500 images and 3D CBCT with over 580 volumes (88,500+ slices) to support the Semi-supervised Teeth Segmentation (STS) Challenge, which includes partially meticulous annotations and covers all age groups. Moreover, multi-phase semi-supervised teeth segmentation algorithms and high-confidence pseudo-labels refinement strategies were proposed by competitors during this challenge. Algorithms were verified on this proposed dataset and good segmentation performance were achieved, over 93+ and 80+ Dice score were obtained for top three 2D and 3D participants, demonstrating the high quality of this proposed dataset. This paper also summarizes the diverse methods employed by the top-ranking teams in the MICCAI 2023 STS Challenge. Our dataset is publicly accessible through Zenodo ( https://zenodo.org/records/10597292 ), and the participants’ code is hosted on GitHub ( https://github.com/ricoleehduu/STS-Challenge ).

Traditional methods for predicting treatment response often rely on readily available clinical factors. However, these methods often lack the granularity to capture the complex interplay between tumor heterogeneity and treatment efficacy. A Multi-graph Fusion (MGF) model that uses habitat subregion-derived radiomic features may help predicting the response to radiotherapy in glioma patients. Firstly, three structural and three physiological habitat regions were delineated using multi-parametric magnetic resonance imaging sequences. Then radiomic features derived from these habitat subregions were used to construct MGF model, which were trained on different combinations of habitat subregions. Each view corresponded to a graph constructed from a specific tumor habitat subregion. Lastly, proposed multi-view fusion module was employed to interpret critical views and interactions for predicting treatment response, while GNNExplainer was used to elucidate the contributions of each view. The MGF model incorporating all habitats achieved the highest area under the curve values of 0.848 (95% CI: 0.832-0.863) for the training cohort and 0.792 (95% CI: 0.767-0.818) for the validation cohort in predicting treatment response. The attention values indicated that physiological habitat 3 held the highest significance. The GNNExplainer revealed key nodes and radiomic features in each view. The MGF model utilizing all habitats-derived radiomics demonstrated the best performance in predicting treatment response. The combination of multi-view fusion module and GNNExplainer enables the framework to capture complex contextual information across six habitat subregions and provides interpretability regarding the factors influencing treatment response predictions.

Recent advances in video action recognition have achieved remarkable performance in coarse-grained macro-action classification by leveraging large-scale visual backbones and transformer architectures. However, extending these successes to fine-grained micro-action recognition remains a fundamental challenge due to the subtlety, brevity, and low motion intensity of micro-actions. In this paper, we propose a high-capacity framework for micro-action recognition, enhancing both representation learning and decision robustness. We scale to large-scale backbones using the VideoMAEv2 Giant model, enabling the extraction of finer spatial-temporal features. A Temporal-Spatial Connector (TSC) is introduced to dynamically highlight discriminative temporal frames and spatial regions, strengthening the model's focus on subtle motion cues critical for micro-action identification. To stabilize optimization and fully exploit the capacity of large models, we design a four-phase progressive training strategy, encompassing linear probing, full fine-tuning, connector-specific optimization, and classifier head refinement. Furthermore, we propose a novel ensemble decision mechanism that integrates Top-K predictions from diverse models via a Large Language Model (LLM), enhancing prediction consistency and robustness through multimodel consensus. Our method achieves an F1mean of 76.54% on the MA-52 dataset, ranking 3rd in the 2025 Micro-Action Analysis Grand Challenge and advancing the state of the art in fine-grained video understanding.

In the field of fine-grained visual recognition (FGVR), the ability to resolve minute and often subtle differences between highly similar object categories is paramount. The advent of vision transformers (ViTs) has marked a significant advancement in this domain, primarily due to their capacity to model the intricate interdependencies among object parts represented as image patches. However, their inherent single-scale processing limitation hampers their effectiveness in FGVR tasks. Furthermore, the challenge of uncertainty inherent in FGVR tasks remains unresolved, necessitating the development of methods that bolster the robustness of these models, particularly across varying scales of visual features. We introduce a new plug-in module that can be seamlessly integrated into ViT, called dual guidance enabled fuzzy inference (DGEFI), which combines fuzzy inference with dual guidance mechanisms. Dual guidance includes scale-aware guidance and probability guidance. The former strengthens the model's focus on salient scales, and the latter refines the distinction between similar categories by optimizing intraclass compactness and interclass separability. Fuzzy inference enables the model to adaptively tweak the influence of distinct scales in the final decision-making phase, thereby enhancing the overall accuracy of recognition tasks. We demonstrate the versatility and efficacy of our DGEFI module by integrating it into several leading ViT backbones, including ViT, Swin, Mvitv2, and EVA-02. Empirical results exhibit exceptional performance gains, with the integration of DGEFI into EVA-02 remarkable accuracy improvements, reaching 93.6% on the CUB-200-2011 dataset and 94.5% on the NA-Birds dataset, respectively, improving over the state-of-the-art method 0.5% and 1.5%.

Video action recognition has been a hot research direction in computer vision, with most existing technologies focusing on coarse-grained macro-action recognition. However, fine-grained action recognition remains challenging. Micro-actions, characterized by high fine-grained, low-intensity, and brief, are crucial for emotion recognition and psychological assessment applications. In this paper, we build on popular video action recognition frameworks as foundation models, introducing multi-auxiliary heads and hybrid loss optimization to advance micro-action recognition. Specifically, the Frame-Level pred and Coarse-Grained Body-Action auxiliary heads work collaboratively to enhance the model and Fine-Grained Micro-Action primary head for perceiving fine-grained and capturing keyframes. Incorporating F1 loss, ArcFace loss, and weighted multi-task loss improves training stability, convergence speed, and performance. Additionally, integrating the optical flow modality enriches the model's diversity, and ensemble learning across all foundational models. Finally, our method achieves a 75.37% F1-mean on the MA-52 dataset, ranking 1st in the Micro-Action Analysis Grand Challenge in conjunction with ACM MM'24. The code is available at https://github.com/qklee-lz/ACMMM2024-MAC.

Human brain extracts the spatial and temporal semantic information by processing the multi-modalities, which has contextually meaningful for perceiving and understanding the emotional state of an individual. However, there are two main challenges in modeling multimodal sequences: 1) the different sampling rates from multimodal data make the cross-modal interactions very difficult; 2) how to efficiently fuse unimodal representations and effectively capture relationships among multimodal data. In this paper, we design the weighted cross-modal attention mechanism, which not only captures the temporal correlation information and the spatial dependence information of each modality, but also dynamically adjusts the weight of each modality across different time steps. And the unimodal subtasks are led for assisting the representation learning of specific modality to jointly train the multimodal tasks and unimodal subtasks to explore the complementary relationships of each modality. Our model gets a new state-of-the-art record on the CMU-MOSI dataset and brings noticeable performance improvements on all the metrics. For the CMU-MOSEI dataset, the F1 score of the binary classification, the 7-class task, and the regression task of our model are still the highest among all models and the proposed model is only lower than the multimodal split attention fusion (MSAF) model with aligned data on the accuracy of the binary classification, showing the great performance of the suggested method.