The popularity of cloud services has provided a new medium for video streams transmission. Cloud desktops, as a representative multimedia application, facilitate interaction between users and cloud via video streams, garnering widespread adoption in various fields. The network condition directly affects the transmission. Therefore, accurate throughput prediction helps guide the allocation of network resources, avoiding a decline in user experience due to insufficient resources and waste caused by excessive resources. Recent works focus more on the temporal characteristics of throughput. However, we believe that throughput of video streaming is significantly influenced by usage scenario. In this paper, we propose a transfer-based autoencoder framework DeskTransfer for throughput prediction in frequent switching cloud desktop scenarios. Specifically, we construct the Scenario Autoencoder and Throughput Autoencoder to respectively learn the scenario and throughput features from historical usage records. By adopting an adversarial mechanism, we design transfer algorithm using latent vectors, enabling the model suitable for multiple scenarios. We collect real-world data from a project cooperated with Lenovo Research for experiment and compare our solution with leading methods on public datasets to validate its effectiveness.