Large Language Models (LLMs) have made significant strides in recent years, performing a wide range of complex tasks across domains. However, in specialized fields like healthcare, LLMs often fall short due to their reliance on general-purpose data and their inability to effectively process domain-specific, multi-modal inputs. Medical data, which typically includes both textual and visual information (e.g., X-rays, MRIs), requires models capable of processing these diverse formats in a cohesive and efficient manner. The demand for lightweight, multi-modal systems that can operate in resource-constrained clinical environments is growing, as many healthcare settings lack the computational power required for large-scale models.In this work, we present a novel framework designed to optimize multi-modal medical tasks by leveraging pre-trained Mixture of Experts (MoE) models. We avoid redundant training by copying the weights from the Med MoE model, including the vision encoder, word embedding, and self-attention modules. The frozen weights allow the model to efficiently inherit general medical knowledge without the need for further training. The next phase focuses on training task-specific expert models through knowledge distillation, transferring the capabilities of larger models to smaller, specialized experts. This architecture enables the system to handle both discriminative and generative tasks while significantly reducing computational overhead.Our approach integrates both expert and Expert of Expert(EoE) models, inspired by the multi-disciplinary team (MDT) strategy used in clinical settings, to enhance the system’s ability to address complex medical decision-making scenarios. By selectively activating relevant experts based on task demands, the framework provides scalable, high-performance learning across diverse medical contexts. Comprehensive experiments demonstrate that our framework achieves great performance with a significantly reduced parameter count, offering a practical and efficient solution for real-world medical applications.

The rapid growth of industries and urbanization has significantly contributed to air pollution, leading to detrimental effects on both human health and the environment. Airborne pollutants, such as fine particulate matter (PM2.5, PM10), and gaseous pollutants, including NOx, SO2, and CO, are closely associated with serious health conditions like lung cancer, cardiovascular diseases, and respiratory disorders. While pollution levels are influenced by these pollutants, meteorological factors such as temperature, humidity, and wind speed also play a critical role in shaping air quality. Accurate prediction of the Air Quality Index (AQI), which reflects the combined impact of pollutants and meteorological conditions, remains a challenging task.This paper presents a comprehensive study on predicting air quality levels in New York City using 24 years of meteorological data and pollutant concentrations, including PM2.5, CO, SO2, and Ozone. Leveraging advanced time series forecasting models, we aim to capture the temporal dynamics of air quality in response to climate variables such as temperature, precipitation, wind speed, and humidity. Our models accurately forecast pollutant levels and the Air Quality Index (AQI), providing insights into the influence of meteorological factors on urban air quality. The results demonstrate the effectiveness of these models in short-term air quality prediction, which can contribute to the development of early warning systems and inform public health interventions. This research highlights the critical role that climate patterns play in pollution levels, offering a data-driven foundation for urban environmental management.