Robust and reliable medical diagnosis using artificial intelligence is crucial, yet real-world clinical environments present significant challenges due to dynamic covariate shifts affecting multi-modal data (images, text, tabular). Existing methods, including the single-modal robust classifier LaDiNE, often fail under these complex, multi-modal shifts, lacking mechanisms for cross-modal invariance, dynamic modality fusion, and fine-grained uncertainty attribution. To address this gap, we propose DyMoLaDiNE (Dynamic Multi-Modal Latent Diffusion Nested-Ensembles), a framework designed for reliable medical diagnosis under dynamic multi-modal covariate shifts. DyMoLaDiNE introduces four key innovations: (1) a Cross-Modal Invariant Feature Extractor leveraging multi-modal Vision Transformers and contrastive learning to derive robust latent representations, (2) a Dynamic Modality Weighting Mechanism that adaptively adjusts modality contributions based on instance-specific reliability scores, (3) a Robust Multi-Modal Diffusion Ensemble utilizing conditional diffusion models conditioned on multi-modal inputs and reliability scores for flexible, calibrated density estimation, and (4) Modality-Attributed Uncertainty Quantification to decompose predictive uncertainty by input source. Extensive evaluations on diverse datasets (MedMD&RadMD, MultiCaRe, PadChest, TCIA RE-MIND, BRaTS, Camelyon16, PANDA) demonstrate that DyMoLaDiNE significantly outperforms (p 0.005) state-of-the-art methods (LDM, CMCL, CGMCL, CIIM, DTTL, FFL, ALDM, LaDiNE) in terms of classification accuracy, robustness under dynamic perturbations, confidence calibration (ECE), and precise uncertainty quantification (CPIW, CNPV), while providing superior modality attribution fidelity. Ablation studies confirm the necessity of each component. DyMoLaDiNE represents a significant advancement in trustworthy, robust multi-modal medical AI. Code supporting this study DyMoLaDiNE .

Light scattering and absorption degrade the quality of underwater images, and various image enhancement methods have been explored. However, the existing underwater image datasets lack corresponding high-quality references, and the degree of scattering and absorption is not strictly controlled. In this study, we constructed an image dataset with different degrees of light scattering and controlled water turbidity via a water tank. The Swin Transformer based dehazing network DehazeFormer has been improved, termed ITW-DehazeFormer, to enhance images acquired through turbid water. First, a histogram equalization pre-enhancement block is added. Second, the SKfusion block is replaced by a content-guided attention based fusion block to combine channel and spatial attention so that information interactions between different channels are guaranteed. Finally, a hybrid loss function combining space and frequency domain information is introduced. Experimental results show that ITW-DehazeFormer outperforms seven existing image enhancement methods in terms of several image quality metrics, including Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM) and multi-scale SSIM.

Vision and tactile are the most widely used perception modes in robotic interactions. We propose a Multimodal Visual-Tactile Sensor (MVTS) using a single camera to synchronously capture visual and tactile information. Unlike the traditional design with a fixed opaque gel layer, the MVTS consists of a color camera, a transparent elastomer layer embedded with color markers, and multiple LEDs. It can acquire images for vision like a normal camera, even during object contact. It can also use the markers to acquire tactile information while interacting with objects. In order to simultaneously acquire and separate tactile and visual information in single-shot, we designed a multimodal information separation framework based on a multi-task learning deep neural network. It adopts a shared feature encoder and two separate and parallel decoders to restore visual images and extract tactile maps. The MVTS was evaluated on multiple tasks such as force estimation, contact surface reconstruction, texture reconstruction, and volume estimation of grasped simple-shaped objects, and the results show that it has good performance in multimodal information acquisition, which can help robots better interact with the environment.

Electrocardiogram (ECG) arrhythmia identification is critical for early cardiovascular disease diagnosis and monitoring in Internet of Things (IoT) industry. Still, it is difficult due to complicated waveforms, individual variability, and the requirement for real-time analysis on resource-limited equipment. Traditional approaches sometimes fail to detect complicated spatial-temporal correlations in ECG data, limiting their efficiency in identifying arrhythmias. Furthermore, deploying these models in tinyML contexts, such as edge and IoT devices limited by large computational and memory needs, emphasizes the importance of lightweight, accurate models for real-time applications. Our suggested solution consists of three main components: SigNet, DualGCNN, and FusionGCNN. SigNet uses Separable Convolution layers to effectively extract local spatial features, making it ideal for IoT-based healthcare deployment. DualGCNN combines dual Graph Convolutional layers with spatial attention, allowing the model to capture local and global dependencies for better classification of arrhythmia. FusionGCNN combines the capabilities of GCN and SigNet with an effective feature fusion technique to improve feature representation while remaining computationally economical. Ablation tests show that FusionGCNN improves performance considerably, with greater accuracy (0.9641), lower training error (0.0004), and a higher F1 Score (0.9645) across a variety of ECG patterns. FusionGCNN, with its low training error, high stability, and computational economy, is well-suited to tinyML requirements, allowing implementation on edge and IoT devices for scalable, real-time ECG monitoring in healthcare.

This article presents an indirect adaptive interval type-3 (IT3) tracking fuzzy control for a class of unknown nonaffine nonlinear discrete-time networked control systems (NCSs) with denial-of-service (DoS) attacks. To mitigate the adverse effects of attacks, a novel two-mode attack compensator, incorporating an IT3 fuzzy model (IT3FM), is proposed to handle the nonlinear dynamics and uncertainties of NCSs by estimating the unavailable system output during active attacks. Then, an updating algorithm for parameter adjustment, guaranteed to converge via Lyapunov theory, is presented. Subsequently, an indirect IT3 fuzzy controller (IT3FC) is developed to ensure robust tracking performance. Theoretical analysis of the proposed control method shows the boundedness of the tracking error. To mitigate the computational complexity of the proposed control method, we use the same IT3 fuzzy sets for the IT3FM and IT3FC. We also employ a direct defuzzification method within the type-reduction process, bypassing the iterative Karnik-Mendel approach. Eventually, three nonlinear NCSs are given to demonstrate the robustness of the proposed control method.

Existing cross-domain image retrieval (CDIR) methods exhibit a strong dependency on prior knowledge of training categories, which leads to problems of class confusion and domain shift when encountering unseen categories in open-set environments. In this paper, we explore the CDIR task towards open-set environments and introduce the Hypergraph-Based Remaining Prototype Alignment (RePro) framework for this task. Specifically, to address the problem of unseen class confusion caused by the category differences, we utilize the Remaining Prototype Embedding (RPE) module to generate the remaining embeddings of images and treat these embeddings as domain noise, rather than directly mapping them to the explicit domain-unified prototypes. To overcome the problem of domain shift, our method leverages the high-order correlations among both domains and categories through the Heterogeneous Structure Alignment (HSA) module, by constructing a heterogeneous hypergraph based on intra-domain and inter-category correlations. Besides, we build two multi-domain datasets for open-set cross-domain image retrieval,i.e., OCD-PACS and OCD-VLCS. Each dataset is divided into seen and unseen categories for training and testing, and each class has four different domains of images. Extensive experiments and ablation studies on these two datasets demonstrate the superiority of our method over current state-of-the-art methods.

Monocular Depth Estimation (MDE) is a fundamental problem in computer vision with broad applications in various downstream tasks. While recent studies focus on designing increasingly complex and powerful deep learning methods to regress depth maps directly, we propose a novel approach by introducing the Virtual Point Cloud (VPC) as an intermediate representation to provide the approximate geometric prior for the MDE task. In this article, we design a multi-scale multi-space representation fusion-enhanced MDE framework to address the challenges of MDE. Specifically, to resolve the issue of scale ambiguity, we design a VPC feature extraction module to learn multi-scale 3D geometric information for the depth prior. Then, we explicitly introduce geometric constraints for global depth prediction by incorporating a multi-space representation fusion from both the texture features in 2D space and the geometric features in 3D space. To mitigate errors at object boundaries, we introduce a confidence map generated based on the quality of the VPC to refine the predicted depth map. Specifically, we construct convolution receptive fields based on 3D spatial distances in spherical coordinates, ensuring that the confidence map provides reliable geometric guidance at object boundaries. Furthermore, we propose an independent confidence geometric consistency loss to supervise the refinement process. Experimental results demonstrate that our method significantly outperforms state-of-the-art approaches across all evaluation metrics on the KITTI and NYU-Depth-v2 datasets, achieving RMSE improvements of 9.2% and 2.8%, respectively. Moreover, zero-shot evaluations on the nuScenes and SUN-RGBD datasets further validate the generalizability of our approach.

Automatic 3D shape analysis is heavily influenced by the pose of input 3D models, as the continuous nature of pose space introduces complexities that usually exceed the encoding capacities of standard deep learning frameworks. To tackle this challenge, we present Upright-Net+, an enhancement of our previous model, Upright-Net, specifically developed for estimating upright orientation in 3D point clouds. Our approach is grounded in the design principle that "form ever follows function," treating the natural base of an object as a functional structure that stabilizes it in its typical pose, influenced by physical laws and geometric properties. We reformulate the continuous orientation problem into a discrete classification task, focusing on learning the points that constitute the natural base of a 3D model. The upright orientation is determined by aligning the normal orientation of this base towards the mass center. To mitigate over-smoothing in the global feature embeddings from stacked graph convolutional layers, we introduce a Global Positional Encoding Module using Relative Distance Histogram Statistics Embedding (GPE-RDHS), which reduces structural ambiguity and enhances orientation estimation. We also enhanced a weighted residual loss term to penalize false positive predictions, enhancing overall model performance. Our method demonstrates exceptional performance in upright orientation estimation and reveals that the learned orientation-aware features significantly benefit downstream tasks, particularly in classification.

Most previous embedding-based methods for anomaly detection directly utilize the visual features extracted from pretrained CNN network. However, there usually exists a gap of domain between pretrained data and target data in anomaly detection. To alleviate this discrepancy, we introduce ReconFA in this paper, a self-supervised domain adaptation method for anomaly detection: firstly, we design a self-supervised multi-scale features fusion method with multi-scale aggregation module (MSAM) to enhance interaction of multi-level outputs of pretrained CNN. Secondly, we train an encoder to adapt the features extracted from pretrained CNN to target domain by a feature-reconstruction task. Meanwhile, we use encoder to reduce the dimension of features to save space complexity. Extensive experiments show that our ReconFA method outperforms previous methods on MVTec AD datasets and on more challenging and sophisticated datasets MPDD.

The graph-information-based fuzzy clustering has shown promising results in various datasets. However, its performance is hindered when dealing with high-dimensional data due to challenges related to redundant information and sensitivity to the similarity matrix design. To address these limitations, this article proposes an implicit fuzzy k-means (FKMs) model that enhances graph-based fuzzy clustering for high-dimensional data. Instead of explicitly designing a similarity matrix, our approach leverages the fuzzy partition result obtained from the implicit FKMs model to generate an effective similarity matrix. We employ a projection-based technique to handle redundant information, eliminating the need for specific feature extraction methods. By formulating the fuzzy clustering model solely based on the similarity matrix derived from the membership matrix, we mitigate issues, such as dependence on initial values and random fluctuations in clustering results. This innovative approach significantly improves the competitiveness of graph-enhanced fuzzy clustering for high-dimensional data. We present an efficient iterative optimization algorithm for our model and demonstrate its effectiveness through theoretical analysis and experimental comparisons with other state-of-the-art methods, showcasing its superior performance.

There are two key issues that limit further improvements in the performance of existing rotational detectors: 1) Periodic sudden change of the parameters in the rotating bounding box (RBBox) definition causes a numerical discontinuity in the loss (such as smoothL1 loss). 2) There is a gap of optimization asynchrony between the loss in the RBBox regression and evaluation metrics. In this paper, we define a new distance formulation between two convex polygons describing the overlapping degree and non-overlapping degree. Based on this smooth distance, we propose a loss called Polygon-to-Polygon distance loss (P2P Loss). The distance is derived from the area sum of triangles specified by the vertexes of one polygon and the edges of the other. Therefore, the P2P Loss is continuous, differentiable, and inherently free from any RBBox definition. Our P2P Loss is not only consistent with the detection metrics but also able to measure how far, as well as how similar, a RBBox is from another one even when they are completely non-overlapping. These features allow the RetinaNet using the P2P Loss to achieve 79.15% mAP on the DOTA dataset, which is quite competitive compared with many state-of-the-art rotated object detectors.

A mass of experiments shows that the pose of the input 3D models exerts a tremendous influence on automatic 3D shape analysis. In this paper, we propose Upright-Net, a deep-learning-based approach for estimating the upright orientation of 3D point clouds. Based on a well-known postulate of design states that “form ever follows function”, we treat the natural base of an object as a common functional structure, which supports the object in a most commonly seen pose following a set of specific rules, e.g. physical laws, functionality-related geometric properties, semantic cues, and so on. Thus we apply a data-driven deep learning method to automatically encode those rules and formulate the upright orientation estimation problem as a classification model, i.e. extract the points on a 3D model that forms the natural base. And then the upright orientation is computed as the normal of the natural base. Our proposed new approach has three advantages. First, it formulates the continuous orientation estimation task as a discrete classification task while preserving the continuity of the solution space. Second, it automatically learns the comprehensive criteria defining a natural base of general 3D models even with asymmetric geometry. Third, the learned orientation-aware features can serve well in downstream tasks. Results show that our network outperforms previous approaches on orientation estimation and also achieves remarkable generalization capability and transfer capability.

Multiple-target tracking in video (MTTV) presents a technical challenge in video surveillance applications. In this paper, we formulate the MTTV problem using dynamic Markov network (DMN) techniques. Our model consists of three coupled Markov random fields: 1) a field for the joint state of the multitarget; 2) a binary random process for the existence of each individual target; and 3) a binary random process for the occlusion of each dual adjacent target. To make the inference tractable, we introduce two robust functions that eliminate the two binary processes. We then propose a novel belief propagation (BP) algorithm called particle-based BP and embed it into a Markov chain Monte Carlo approach to obtain the maximum a posteriori estimation in the DMN. With a stratified sampler, we incorporate the information obtained from a learned bottom-up detector (e.g., support-vector-machine-based classifier) and the motion model of the target into the message propagation. Other low-level visual cues such as motion and shape can be easily incorporated into our framework to obtain better tracking results. We have performed extensive experimental verification, and the results suggest that our method is comparable to the state-of-art multitarget tracking methods in all the cases we tested.

To achieve robust system, more and more vision researchers take into account fusing multiple visual cues. In this paper, we propose a novel strategy to integrate multiple naive cues for head tracking. Firstly, a cue dependency model is constructed via graphical model. Secondly, a new inference procedure based on non-parametric belief propagation is built for cue integration. The work presented is thus a general framework easy to extend for other computer vision research problems. Experimental results imply that the strategy we propose is effective, and it is robust without estimation of cue reliability. 1