Edge computing significantly enhanced the growth of edge-assistant video streaming applications. However, challenges such as unpredictable wireless conditions, resource constraints, and task redundancy have intertwined impacts on the overall performance of edge video streaming systems (EVS). Therefore, it is essential to have an integrated framework that addresses resource management, computational offloading, and video task preprocessing. Existing optimization strategies often neglect the simultaneous management of computational offloading, resource allocation, and video task preprocessing, leading to a suboptimal system utility. Moreover, they struggle to handle high-dimensional decision variables. On the other hand, learning-based adaptive schemes fall short in integrating distributed decisions and ensuring the scalability of wireless devices. Additionally, current approaches lack adaptive incentives. To bridge these gaps, we propose a novel framework called AIRA, which is based on improved multi-agent reinforcement learning (MARL) and smart contracts. AIRA manages resources, video compression, and adaptive incentives in a distributed manner. It consists of a MARL-driven cooperative learning algorithm (CLA) and a smart contract-guided adaptive incentive mechanism. Leveraging an actor-critic structure, the CLA enables wireless devices to master strategies for resource allocation, video task compression, and offloading, utilizing historical data. Notably, the CLA incorporates an attention mechanism to select pivotal tuples from the observation-action pairings among different agents, ensuring improved scalability and computational prowess. Evaluations based on real-world trajectories demonstrate that AIRA enables adaptive incentives. Compared to state-of-the-art approaches, CLA effectively enhances the long-term system utility and scalability of EVS.