Photographs are one of the most fundamental ways for human beings to capture their social experiences and smiling is one of the most common actions associated with photo-taking. Photos, thus provide a unique opportunity to study the phenomena of mixing of different people and also the smiles expressed by individuals in these social settings. In this work, we study whether a social media-based computational framework can be employed to obtain smile and diversity scores at very fine, individual relationship resolution, and study their associations. We analyze two data sets from different social networks, Twitter and Instagram, over different time periods. Primarily looking at photographs, using computer vision APIs, we capture the diversity of social interactions in terms of age, gender, and race of those present, and smile levels. Analysis of both data sets suggest similar and significant findings: (a) people, in general, tend to smile more in the presence of others; and (b) people tend to smile more in a more diverse company. The results can help scale, test, and validate multiple theories related to affect and diversity in sociology, psychology, biology, and urban planning, and inform future mechanisms for encouraging people to smile more often in everyday settings