As the proliferation of video content continues, and many video archives lack suitable metadata, therefore, video retrieval, particularly through example-based search, has become increasingly crucial. Existing metadata often fails to meet the needs of specific types of searches, especially when videos contain elements from different modalities, such as visual and audio. Consequently, developing video retrieval methods that can handle multi-modal content is essential. In designing our novel video retrieval framework named Multi-modal Video Search by Examples (MVSE)1, we focused on accuracy (precision and recall), efficiency (retrieval time in seconds), interactivity, and extensibility, with key components including advanced data processing and a user-friendly interface aimed at enhancing search effectiveness and user experience. With the advent of Large Language Models (LLMs), the interaction between multimodal data, including image and audio has been transformed with a significant leap forward towards a bigger goal of artificial general intelligence. This workshop aims to bring together experts from diverse domains to explore the possibilities of developing novel ways of multimodal data search, understanding and interaction.

Abstract As the proliferation of video content continues, and many video archives lack suitable metadata, therefore, video retrieval, particularly through example‐based search, has become increasingly crucial. Existing metadata often fails to meet the needs of specific types of searches, especially when videos contain elements from different modalities, such as visual and audio. Consequently, developing video retrieval methods that can handle multi‐modal content is essential. An innovative Multi‐modal Video Search by Examples (MVSE) framework is introduced, employing state‐of‐the‐art techniques in its various components. In designing MVSE, the authors focused on accuracy, efficiency, interactivity, and extensibility, with key components including advanced data processing and a user‐friendly interface aimed at enhancing search effectiveness and user experience. Furthermore, the framework was comprehensively evaluated, assessing individual components, data quality issues, and overall retrieval performance using high‐quality and low‐quality BBC archive videos. The evaluation reveals that: (1) multi‐modal search yields better results than single‐modal search; (2) the quality of video, both visual and audio, has an impact on the query precision. Compared with image query results, audio quality has a greater impact on the query precision (3) a two‐stage search process (i.e. searching by Hamming distance based on hashing, followed by searching by Cosine similarity based on embedding); is effective but increases time overhead; (4) large‐scale video retrieval is not only feasible but also expected to emerge shortly.

Abstract Facial identity is subject to two primary natural variations: time-dependent (TD) factors such as age, and time-independent (TID) factors including sex and race. This study aims to address a broader problem known as variation-invariant face recognition (VIFR) by exploring the question: “How can identity preservation be maximized in the presence of TD and TID variations?" While existing state-of-the-art (SOTA) methods focus on either age-invariant or race and sex-invariant FR, our approach introduces the first novel deep learning architecture utilizing multi-task learning to tackle VIFR, termed “multi-task learning-based variation-invariant face recognition (MTLVIFR)." We redefine FR by incorporating both TD and TID, decomposing faces into age (TD) and residual features (TID: sex, race, and identity). MTLVIFR outperforms existing methods by 2% in LFW and CALFW benchmarks, 1% in CALFW, and 5% in AgeDB (20 years of protocol) in terms of face verification score. Moreover, it achieves higher face identification scores compared to all SOTA methods. Open source code .

Abstract Stochastic local search (SLS) algorithms are well known for their ability to efficiently find models of random instances of the SAT problem, especially for uniform random k‐SAT instances. Two processes affect most SLS solvers—the initial assignment of the variables and the heuristics that select which variable to flip. In the last few years, the work on generating the appropriate initial assignment has not been paid much attention or seen much progress, while most SLS solvers focused on the heuristic algorithm. The present work aims to improve SLS algorithms on uniform random k‐SAT instances by developing effective methods for generating the initial assignment of variables in a controlled way. First, the allocation strategy introduced recently for 3‐SAT instances is extended to initialize the initial assignment on random k‐SAT instances. Then a concept of an initial probability distribution of the clause‐to‐variable ratio of the instance is introduced to determine the parameters of the allocation strategy. This combined method is added to the beginning of six state‐of‐the‐art SLS algorithms in order to generate initial assignments of variables in a controlled way instead of generating them randomly, resulting in six extended SLS algorithms named WalkSATlm_E, DCCASat_E, Score2SAT_E, CSCCSat_E, Probsat_E, and Sparrow_E, respectively. They are then evaluated in terms of their capabilities and efficiency on uniform random k‐SAT instance from the random track of SAT Competitions in 2016, 2017, and 2018. Experimental results show that these improved SLS solvers outperform their original performance, especially WalkSAT_E, Score2SAT_E, and CSCCSat_E outperform the winner of the random track of SAT competition in 2017. In addition, based on the initial probability distribution method, the present work proposes a parameter tuning and analysis of random 3‐SAT instances and provides an additional comparative analysis with the state‐of‐the‐art random SLS solvers based on large‐scale experiments.