Pre-trained on extensive text and image corpora, current Multi-Modal Large Language Models (MLLM) have shown strong capabilities in general visual reasoning tasks. However, their performance is still lacking in physical domains that require understanding diagrams with complex physical structures and quantitative analysis based on multi-modal information. To address this, we develop a new framework, named **M**ulti-Modal Scientific Re**A**soning with **P**hysics Perception and **S**imulation (**MAPS**) based on an MLLM. MAPS decomposes expert-level multi-modal reasoning task into physical diagram understanding via a Physical Perception Model (PPM) and reasoning with physical knowledge via a simulator. The PPM module is obtained by fine-tuning a visual language model using carefully designed synthetic data with paired physical diagrams and corresponding simulation language descriptions. At the inference stage, MAPS integrates the simulation language description of the input diagram provided by PPM and results obtained through a Chain-of-Simulation process with MLLM to derive the underlying rationale and the final answer. Validated using our collected college-level circuit analysis problems, MAPS significantly improves reasoning accuracy of MLLM and outperforms all existing models. The results confirm MAPS offers a promising direction for enhancing multi-modal scientific reasoning ability of MLLMs. We will release our code, model and dataset used for our experiments upon publishing of this paper.

Abstract Intelligent transport systems (ITS) is a popular field of research in smart city. Image‐based vehicle recognition is one of the most promising new techniques. The traditional appearance‐based method has some limitations in feature view. The machine learning algorithm can satisfy multiple feature views, but it mainly adopts an improved supervised learning method, which needs manual annotation and has low efficiency. The semi‐supervised collaborative training method is widely used in the image recognition process to improve machine learning algorithms' accuracy and generalization performance. This paper proposes DFT (deep feature‐based training) method for vehicle recognition in smart cities. DFT is also a semi‐supervised collaborative training method on basis of two base learners. DFT adjusts data pre‐processing and training process, optimizes the constructing a disagreement encoding network, and expands the recognition disagreement of pseudo‐labelled samples‐based training set. Compared with the typical collaborative training methods, DFT greatly accelerates the model's training process by reducing the convergence time, and improves the efficiency of vehicle recognition, while remaining the recognition accuracy unchanged.