Accurately predicting molecular properties can identify more promising drug candidates and facilitate the process of drug discovery. There are many advances in methods for molecular property prediction using self-supervised learning. However, most methods tend to focus only on a certain representation, resulting in reduced information diversity. Moreover, less attention is paid to the functional groups in the molecule, which are highly related to the molecular properties. To solve the problem, a multimodal contrastive learning framework called MMCL for molecular property prediction is proposed, which learns common features among similar molecules by utilizing different views of molecules for predicting molecular properties. In addition, by explicitly representing functional groups as nodes in the molecular graph, the model is prompted to learn features related to molecular properties. Prediction results on benchmark datasets and a series of case studies show that MMCL has optimal performance in most property prediction tasks and can capture information related to chemical semantics. MMCL can be used as an efficient deep learning tool for molecular-related tasks to facilitate the drug discovery process.

Abstract Large multimodal language models (MLLMs) such as GPT-4V and GPT-4o have achieved remarkable advancements in understanding and generating multimodal content, showcasing superior quality and capabilities across diverse tasks. However, their deployment faces significant challenges, including slow inference, high computational cost, and impracticality for on-device applications. In contrast, the emergence of small MLLMs, exemplified by the LLava-series models and Phi-3-Vision, offers promising alternatives with faster inference, reduced deployment costs, and the ability to handle domain-specific scenarios. Despite their growing presence, the capability boundaries between large and small MLLMs remain underexplored. In this work, we conduct a systematic and comprehensive evaluation to benchmark both small and large MLLMs, spanning general capabilities such as object recognition, temporal reasoning, and multimodal comprehension, as well as real-world applications in domains like industry and automotive. Our evaluation reveals that small MLLMs can achieve comparable performance to large models in specific scenarios but lag significantly in complex tasks requiring deeper reasoning or nuanced understanding. Furthermore, we identify common failure cases in both small and large MLLMs, highlighting domains where even state-of-the-art models struggle. We hope our findings will guide the research community in pushing the quality boundaries of MLLMs, advancing their usability and effectiveness across diverse applications.

Organoids, 3D in vitro cultures replicating their tissue or organ of origin, hold great potential for disease research, drug testing, and regenerative medicine. However, current analysis methods primarily rely on staining techniques, which limit high-throughput analysis and require manual adjustments. Automated and non-invasive methods remain underexplored, and most rely on small-scale datasets that fail to capture the complexity of organoids, which vary significantly in shape and size and have more intricate backgrounds than cells. To address these limitations, we developed MSSOrg, a multiscale framework for organoid localization and classification, integrating Local and Global State Space Models (SSMs) to balance pixel dependencies and texture features. MSSOrg mitigates background interference and image noise in bright-field images, achieving superior performance in both organoid detection and morphological classification. This innovation enables real-time, large-scale organoid analysis, providing a powerful tool for drug screening and disease research, ultimately accelerating progress in these fields.

Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview. Most current pretraining methods, however, are limited to one or two modalities. We present i-Code, a self-supervised pretraining framework where users may flexibly combine the modalities of vision, speech, and language into unified and general-purpose vector representations. In this framework, data from each modality are first given to pretrained single-modality encoders. The encoder outputs are then integrated with a multimodal fusion network, which uses novel merge- and co-attention mechanisms to effectively combine information from the different modalities. The entire system is pretrained end-to-end with new objectives including masked modality unit modeling and cross-modality contrastive learning. Unlike previous research using only video for pretraining, the i-Code framework can dynamically process single, dual, and triple-modality data during training and inference, flexibly projecting different combinations of modalities into a single representation space. Experimental results demonstrate how i-Code can outperform state-of-the-art techniques on five multimodal understanding tasks and single-modality benchmarks, improving by as much as 11% and demonstrating the power of integrative multimodal pretraining.

Abstract Multi‐antenna signal detection is one of the most critical and challenging issues for ambient backscatter communication (AmBC) systems. This paper proposes an efficient multi‐antenna AmBC signal detection method, called Bayesian‐MLE (maximum likelihood estimation). It shows good performance on high transmission rate, detection accuracy and low energy consumption. Particularly, a practical multi‐antenna AmBC system model is developed to offer transmit‐receive diversity, and then an efficient multi‐antenna AmBC signal detection method is presented using Bayesian optimization and MLE theory. Furthermore, to maximize the detection performance, an optimal detection threshold selection scheme is developed. Particularly, a non‐central chi‐square distribution conditional probability density function (PDF) is considered instead of the conventional Gaussian PDF. Extensive qualitative and quantitative experiments are performed, showing that the proposed Bayesian‐MLE detector achieves the state‐of‐the‐art signal detection performance.

Most existing nonblind denoising approaches assumed the noise to be homogeneous white Gaussian distribution with known intensity. However, it is difficult to know beforehand or model accurately real-world noises with complex hybrid distribution and noise intensity. In this paper, active joint prior learning (JPL) is proposed for real-world ISAR image blind denoising. (1) To explore strong model hierarchy and components relationship automatically, a novel graphical Dirichlet mixture process (GDMP) model is developed, where the latent representations and component hyperparameters are jointly learned from each other. (2) A multiscale joint learning strategy (MJLS) is proposed to take advantage of both the optimization- and discriminative learning-based capabilities. The external noiseless, internal noisy image information and their relationships are jointly explored simultaneously. (3) Low-rank weighted sparse learning (LWSL) is proposed to learn sparse discriminative correlation components for robust prior learning, and latent low-rank embedding for GDMP patterns self-adaptive inference. Extensive experimental results on ISAR image datasets demonstrate the effectiveness of the proposed model for both synthesis and real-world noisy ISAR images, and the proposed method outperforms the state-of-the-art denoising methods.