Deploying Vision Transformers (ViTs) in edge computing environments presents significant challenges due to their high computing demands and the resource constraints of edge devices. While collaborative edge computing and dynamic token dropping offer potential solutions, existing approaches suffer from rigid strategies that fail to adapt to diverse conditions of network and computing resources at the edge. This paper introduces EasyViT, an adaptive framework that optimizes ViT deployment through the joint coordination of collaborative edge computing and dynamic token dropping. Key innovations include: (1) A token dropping model that integrates dynamic token dropping and collaborative edge environments, formulating an integer linear programming (ILP) optimization problem. (2) An Approximate Stochastic Gradient Descent (ASGD) method with atomic gradient calculation, which transforms the NP-hard ILP problem into a continuous space for rapid near-optimal solution generation. Extensive evaluations on a real-world edge testbed with multiple Raspberry Pi nodes demonstrate that EasyViT achieves 1.06–5.06× speedup over baseline methods under 20 configurations of edge environments, while maintaining model accuracy within 2.8% degradation. The proposed framework exhibits the adaptability across diverse ViT architectures, network bandwidths, and computing resources.