Long Range Wide Area Network (LoRaWAN) provides a promising solution for long-range, low-power wireless communication in Internet of Things (IoT) applications. However, energy consumption remains a critical challenge that significantly impacts network lifetime and scalability. Traditional LoRaWAN implementations rely on automatic and continuous transmission parameter allocation schemes managed by centralized gateways, which fail to adapt to dynamic network conditions such as varying link quality, node mobility, and fluctuating energy levels, leading to excessive energy consumption and reduced network performance. In this work, we propose Adaptive Energy Efficient Routing Protocol (AEERP), a reinforcement learning-based framework that dynamically optimizes transmission parameters in LoRaWAN network. Our approach leverages Expected State-Action-Reward-State-Action (SARSA) to enable intelligent adaptation of spreading factor, transmission power, and gateway selection based on real-time conditions. Key innovations include: (1) a hybrid Q-value initialization strategy combining domain knowledge with exploration, (2) adaptive discount factor and learning rate mechanisms, and (3) joint optimization of transmission parameters based on residual energy, distance, and Packet Delivery Ratio (PDR). Extensive simulations demonstrate that AEERP significantly outperforms state-of-the-art methods, achieving substantial energy conservation, superior PDR, enhanced signal-to-noise ratios, and reduced bit error rates. Results show that AEERP effectively balances energy efficiency, and communication reliability for sustainable LoRaWAN deployments.