Finding similar users is one of the probable applications in social media. The similarity between users can be measured in two different approaches: the semantic similarity and the similarity in terms of social relations. These two approaches can be combined with different weight factors. However, the conventional combination scheme has a critical drawback that the weight factors are fixed for every user and thus it is not optimized at those users that are using rare terms or do not have sufficient relations with other users. To address this problem, in this paper, we propose an adaptive combination scheme of tag-based similarity and link-based similarity in which the weight factors are dynamically determined for each user by evaluating each user's characteristics such as tag commonness and link strength. The experimental results with a Flickr data set show that the proposed scheme consistently outperforms the previous work by about 20%.