Recently, how to provide better POI recommendation performance by exploiting representation learning frame work is still one challenging task in LBSNs. Most existing methods either focus on modeling few limited information without considering other important information such as social influence and sentiment factor, or only rely on traditional shallow models to combine different factors, lacking effective integrating way to learn latent representations by fully utilizing multi-aspect interaction relations. To address these issues, we propose a novel sentiment-aware POI recommendation framework, dubbed as Senti2LSTM, which is capable of learning more comprehensive representations of users and POIs with fusion of multi-relations. Specifically, we first employ dual LSTMs to capture different sentimental embeddings for users and POIs respectively from emotional comments in LBSNs, and then we integrate them into the aggregation of propagation embeddings for users and POIs when learning their latent representations from user-POI bipartite graph and social link graph in LBSNs. Additionally, we also consider discriminating the importance of different social neighbors by leveraging social based attention mechanism, which makes social friends with common sentiments have more similar preferences.Finally,extensive experimental results conducted on two real-world datasets, e.g., Foursquare-NYC and Yelp2018, have demonstrated the effectiveness of our proposed Senti2LSTM in sentiment learning, and significantly outperforming the state-of-the-art POI recommendation methods