To understand different aspects of online human behaviors, e.g., the public stances toward various social and political issues, contextual target-specific stance detection has become one of the most important studies on social media. Considering the lack of appropriate data for the studies of contextual target-specific stance detection on Twitter, which is one of the most popular online social platforms worldwide, we introduce CTSDT, a new dataset that consists of a large number of annotated target-specific conversations collected from Twitter. Furthermore, we propose a new contextual target-specific stance detection model called ConMulAttn, which is the first method that can learn both the contents of the posts and the concrete relationships between the posts in a conversation. We conduct extensive evaluation using CTSDT as well as another two popular datasets, CreateDebate and ConvinceMe, for contextual target-specific stance detection. The evaluation results validate the necessity of introducing our dataset CTSDT. Besides, according to the evaluation results, our proposed model ConMulAttn can outperform the state-of-the-art contextual target-specific stance detection method by up to 25% in F1score, indicating the effectiveness and superiority of our solution. Our study has the potential to assist policymakers in utilizing conversation data from online social platforms to efficiently gain real-time insights into public stances on target topics, such as vaccination.