Unsupervised cross-modal hashing methods have become a core technology for retrieving vast amounts of heterogeneous multimedia information due to their advantages in retrieval speed and storage efficiency. Although these methods have made significant progress in the field of multimedia retrieval, they still face challenges related to inaccurate similarity measurements and incomplete embedding of relational information. To address these issues, we propose Unsupervised Hierarchical Dynamic Similarity Hashing(UHDSH) for multimedia retrieval. First, the Semantic Similarity Measurement Layer extracts common semantic information within multimedia data to construct a dynamic fluctuation similarity hypergraph, which guides the training of the hash function. Second, the Relational Constraint Hashing Layer, based on the dynamic fluctuation similarity hypergraph embedding technique and multimodal feature reconstruction, ensures the precise embedding of both semantic and relational information. Finally, we conducted comprehensive experiments on two widely used datasets, MIR Flickr and NUS-WIDE. Our proposed UHDSH method achieves a maximum improvement of 5.06% over the best baseline methods. The code is publicly available at https://github.com/YunfeiChenMY/UHDSH.

The global distribution of large-scale, multi-source, heterogeneous data has posed an urgent challenge for the efficient organization and retrieval of massive heterogeneous datasets. Hashing learning, known for its advantages in storage efficiency and retrieval speed, has emerged as a key technology to address the challenges of large-scale heterogeneous data. However, challenges related to the decentralized distribution of data and inter-institutional privacy security during transmission remain unresolved. To tackle these issues, we propose a Decentralized Multi-agent Distributed Hashing (DMDH) framework for multimedia retrieval. First, we innovatively introduce a multi-agent collaborative strategy, where each client is treated as an independent agent capable of selecting corresponding agents for model interaction based on its own needs, enabling efficient organization and management of distributed data. Second, we propose an adaptive client fusion mechanism, which leverages model exchange, model aggregation, and model updates to enable distributed model training for any cross-modal hashing method. Comprehensive experiments demonstrate the superiority and efficiency of the proposed DMDH framework, validating its effectiveness in addressing these critical challenges.

Cross-modal hashing enables efficient cross-modal retrieval by compressing multi-modal data into compact binary codes, but traditional methods primarily rely on centralized training, which is limited when handling large-scale distributed datasets. Federated learning presents a scalable alternative, yet existing federated frameworks for cross-modal hashing face challenges like data heterogeneity and imbalance, such as non-IID data distribution across clients. To address these challenges, we propose Personalized Federated learning with Lookahead for Adaptive cross-modal Hashing (PFedLAH) method, which combines Feature Adaptive Personalized Learning (FAPL) and Weight-aware Lookahead Adaptive Selection (WLAS) mechanism together. Initially, the FAPL module is designed for the client, enabling personalized learning to mitigate the effect of divergence between server and client resulting from non-IID data distribution, while the local optimization constraint mechanism is also integrated to avoid local optimization shift and ensure better alignment with global convergence. On the server side, WLAS module combines weight-aware adaptive client selection and gradient momentum lookahead to form a dynamic and intelligent client selection scheme, while enhancing the overall convergence and consistency through lookahead gradient prediction. Comprehensive experiments on widely used datasets, including MIRFlickr-25K, MS COCO, and NUS-WIDE, comparing state-of-the-art federated hashing methods, demonstrate the superior retrieval performance, robustness, and scalability of the PFedLAH method.

Cross-modal hashing is a crucial field focused on accurately pairing and retrieving data from different types, such as images and text, by aligning them in a simple, shared space. While there’s progress, many current techniques still need help fully capturing and keeping the rich meanings across these different types. A new method called Semantic Embedding-based Unsupervised Cross-Modal Hashing (CMSEH) has been introduced to tackle this. This method uses advanced networks to learn joint representations of images and text, making embedding and retrieving data across these different forms easier. The CMSEH method has several new and notable features. Firstly, it uses advanced cross-modal hashing to improve retrieval performance and creates binary codes that keep the original data's neighborhood structure. This is done by creating a joint semantic relevance matrix that combines information from different types, capturing their inherent similarities. Secondly, the CMSEH method introduces a new way of creating hash codes that focuses on relevance and keeping consistent meanings through deep semantic embedding, which makes retrieval more effective. Finally, the method introduces a way to learn hash functions that consistently embed semantic information using advanced networks, tailoring them for each data type. Tests of the CMSEH method show that it performs better than existing methods in terms of accuracy and efficiency in retrieval, marking a significant step forward in cross-modal retrieval methods.

Surgical phase segmentation plays an important role in computer-assisted surgery systems, aiming to recognize what step or what action is operating in the video frame. Existing methods focus on improving the accuracy and precision of video segmentation, but ignore semantic consistency and temporal continuity of video frames in the intra-phase, which is necessary to apply in realistic computer-assisted equipment. Meanwhile, recent works almost extract long-term dependencies by Temporal Convolutional Network, but we heed high layers in TCN lose fine-grained information for detecting surgical steps and further affect phase segmentation task. To address these problems, we propose a Surgical Phase Segmentation Framework (SPHASE) which contains a multimodal feature fusion process and follows a multi-branch predictor. Moreover, we design a multimodal feature fusion mechanism when aggregate optical flow feature and I3D feature. The extensive experiments on AutoLaparo, Cholec80, and M2CAI2016 datasets demonstrate our method outperforms the state-of-the-art method by a large margin, especially in the JACC metric, which means SPHASE is more applicable in the surgical operating room.