The growing complexity of deep learning models introduces challenges in scalability and adaptability. This paper explores how modular design, inspired by software engineering, can enhance deep learning systems. This paper also showed that modern deep learning techniques such as Mixture of Experts (MoE) and LoRA are advancing toward higher modularity. By promoting modular architectures, we emphasize the need to move beyond monolithic models toward more reusable, maintainable, and scalable AI systems, providing a potential research direction for future work.

This paper introduces Global Contextualized Representations (GCoRe) – an extension for existing transformer-based language models. GCoRe addresses limitations in capturing global context and long-range dependencies by utilizing Graph Neural Networks for graph inference on a context graph constructed from the input text. Global contextualized features, derived from the context graph, are added to the token representations from the base language model. Experiment results show that GCoRe improves the performance of the baseline model (DeBERTa v3) by 0.57% on the HotpotQA dataset and by 0.15% on the SQuAD v2 dataset. In addition, GCoRe is able to answer questions that require logical reasoning and multi-hop inference, while the baseline model fails to provide correct answers.

This paper presents a reinforcement learning approach for foreign exchange trading. Inspired by technical analysis methods, this approach makes use of technical indicators by encoding them into Gramian Angular Fields and searches for patterns that indicate price movements using convolutional neural networks (CNN). In addition to the policy that determines the action to take, an extra regression head is utilized to determine the size of market orders. This paper also experimentally shows that maximizing the return of individual trade or cumulative reward in a finite time window results to better performance.