The interplay between mood and eating episodes has been extensively researched within the fields of nutrition, psychology, and behavioral science, revealing a connection between the two. Previous studies have relied on questionnaires and mobile phone self-reports to investigate the relationship between mood and eating. In more recent work, phone sensor data has been utilized to characterize both eating behavior and mood independently, particularly in the context of mobile food diaries and mobile health applications. However, current literature exhibits several limitations: a lack of investigation into the generalization of mood inference models trained with data from various everyday life situations to specific contexts like eating; an absence of studies using sensor data to explore the intersection of mood and eating; and inadequate examination of model personalization techniques within limited label settings, a common challenge in mood inference (i.e., far fewer negative mood reports compared to positive or neutral reports). In this study, we examined the everyday eating and mood using two separate datasets from two different studies: (i) Mexico (N \({}_{MEX}\) = 84, 1,843 mood-while-eating reports with a label distribution of positive: 51.7%, neutral: 38.6%, and negative: 9.8%) in 2019, and (ii) eight countries (N \({}_{MUL}\) = 678, 329K mood reports, including 24K mood-while-eating reports with a label distribution of positive: 83%, neutral: 14.9%, and negative: 2.2%) in 2020, which contain both passive smartphone sensing and self-report data. Our results indicate that generic mood inference models experience a decline in performance in specific contexts, such as during eating, highlighting the issue of sub-context shifts in mobile sensing. Moreover, we discovered that population-level (non-personalized) and hybrid (partially personalized) modeling techniques fall short in the commonly used three-class mood inference task (positive, neutral, negative). Additionally, we found that user-level modeling posed challenges for the majority of participants due to insufficient labels and data in the negative class. To overcome these limitations, we implemented a novel community-based personalization approach, building models with data from a set of users similar to the target user. Our findings demonstrate that mood-while-eating can be inferred with accuracies 63.8% (with F1 score of 62.5) for the MEX dataset and 88.3% (with F1 score of 85.7) with the MUL dataset using community-based models, surpassing those achieved with traditional methods.

Diversity-aware platform design is a paradigm that responds to the ethical challenges of existing social media platforms. Available platforms have been criticized for minimizing users' autonomy, marginalizing minorities, and exploiting users' data for profit maximization. This paper presents a design solution that centers the well-being of users. It presents the theory and practice of designing a diversity-aware platform for social relations. In this approach, the diversity of users is leveraged in a way that allows like-minded individuals to pursue similar interests or diverse individuals to complement each other in a complex activity. The end users of the envisioned platform are students, who participate in the design process. Diversity-aware platform design involves numerous steps, of which two are highlighted in this paper: 1) defining a framework and operationalizing the "diversity" of students, 2) collecting "diversity" data to build diversity-aware algorithms. The paper further reflects on the ethical challenges encountered during the design of a diversity-aware platform.

Understanding food consumption patterns and contexts using mobile sensing is fundamental to build mobile health applications that require minimal user interaction to generate mobile food diaries. Many available mobile food diaries, both commercial and in research, heavily rely on self-reports, and this dependency limits the long term adoption of these apps by people. The social context of eating (alone, with friends, with family, with a partner, etc.) is an important self-reported feature that influences aspects such as food type, psychological state while eating, and the amount of food, according to prior research in nutrition and behavioral sciences. In this work, we use two datasets regarding the everyday eating behavior of college students in two countries, namely Switzerland (Nch=122) and Mexico (Nmx=84), to examine the relation between the social context of eating and passive sensing data from wearables and smartphones. Moreover, we design a classification task, namely inferring eating-alone vs. eating-with-others episodes using passive sensing data and time of eating, obtaining accuracies between 77% and 81%. We believe that this is a first step towards understanding more complex social contexts related to food consumption using mobile sensing.

Smartphone applications that use passive sensing to support human health and well-being primarily rely on: (a) generating low-dimensional representations from high-dimensional data streams; (b) making inferences regarding user behavior; and (c) using those inferences to benefit application users. Meanwhile, sometimes these datasets are shared with third parties as well. Human-centered ubiquitous systems need to ensure that sensitive attributes of users are protected when applications provide utility to people based on such behavioral inferences. In this paper, we demonstrate that inferences of sensitive attributes of users (gender, body mass index category) are possible using low-dimensional and sparse data coming from mobile food diaries (a combination of sensor data and self-reports). After exposing this potential risk, we demonstrate how deep learning techniques can be used for feature transformation to preserve sensitive user information while achieving high accuracies for application-related inferences (e.g. inferring the type of consumed food). Our work is based on two datasets of daily eating behavior of 160 young adults from Switzerland (NCH=122) and Mexico (NMX=38). Results show that using the proposed approach, accuracies in the order of 75%-90% can be achieved for application related inferences, while reducing the sensitive inference to almost random performance.

Airbnb is changing the landscape of the hospitality industry, and to this day, little is known about the inferences that guests make about Airbnb listings. Our work constitutes a first attempt at understanding how potential Airbnb guests form first impressions from images, one of the main modalities featured on the platform. We contribute to the multimedia community by proposing the novel task of automatically predicting human impressions of ambiance from pictures of listings on Airbnb. We collected Airbnb images, focusing on the countries Switzerland and Mexico as case studies, and used crowdsourcing mechanisms to gather annotations on physical and ambiance attributes, finding that agreement among raters was high for most of the attributes. Our cluster analysis showed that both physical and psychological attributes could be grouped into three clusters. We then extracted state-of-the-art features from the images to automatically infer the annotated variables in a regression task. Results show the feasibility of predicting ambiance impressions of homes on Airbnb, with up to 42% of the variance explained by our model, and best results were obtained using activation layers of deep convolutional neural networks trained on the Places dataset, a collection of scene-centric images.

There is a growing interest in social and urban computing to employ crowdsourcing as means to gather impressions of urban perception for indoor and outdoor environments. Previous studies have established that reliable estimates of urban perception can be obtained using online crowdsourcing systems, but implicitly assumed that the judgments provided by the crowd are not dependent on the background knowledge of the observer. In this paper, we investigate how the impressions of outdoor urban spaces judged by online crowd annotators, compare with the impressions elicited by the local inhabitants, along six physical and psychological labels. We focus our study in a developing city where understanding and characterization of these socio-urban perceptions is of societal importance. We found statistically significant differences between the two population groups. Locals perceived places to be more dangerous and dirty, when compared with online crowd workers; while online annotators judged places to be more interesting in comparison to locals. Our results highlight the importance of the degree of familiarity with urban spaces and background knowledge while rating urban perceptions, which is lacking in some of the existing work in urban computing.

The mobile and ubiquitous nature of conversational social video has placed video blogs among the most popular forms of online video. For this reason, there has been an increasing interest in conducting studies of human behavior from video blogs in affective and social computing. In this context, we consider the problem of mood and personality trait impression inference using verbal and nonverbal audio-visual features. Under a multi-label classification framework, we show that for both mood and personality trait binary label sets, not only the simultaneous inference of multiple labels is feasible, but also that classification accuracy increases moderately for several labels, compared to a single-label approach. The multi-label method we consider naturally exploits label correlations, which motivate our approach, and our results are consistent with models proposed in psychology to define human emotional states and personality. Our approach points to the automatic specification of co-occurring emotional states and personality, by inferring several labels at once, compared to single-label approaches. We also propose a new set of facial features, based on emotion valence from facial expressions, and analyze their suitability in the multi-label framework.

The aim of this work is to provide a fast approach for monocular SLAM initialization by constructing an initial 3-D map with interest points that are susceptible to be automatically tracked. Interest points' depth is inferred by means of a linear regression model, which estimates depth on the basis of local image appearance. Our contributions are: (1) a new scheme for learning and predicting associations between depth and local image appearance using RGB-D data; and (2) the use of this scheme for the initialization of state-of-the-art visual SLAM systems from a single image frame. To the best of our knowledge, this is the first attempt to automatically initialize a SLAM system by associating depth to sensor features through machine learning techniques. We performed a series of tests by making use of the celebrated PTAM system and obtained very promising results. We show successful one-shot initialization examples accomplished by applying our proposed approach to unstructured scene environments.

Recognizing classes of objects from their shape is an unsolved problem in machine vision that entails the ability of a computer system to represent and generalize complex geometrical information on the basis of a finite amount of prior data. A practical approach to this problem is particularly difficult to implement, not only because the shape variability of relevant object classes is generally large, but also because standard sensing devices used to capture the real world only provide a partial view of a scene, so there is partial information pertaining to the objects of interest. In this work, we develop an algorithmic framework for recognizing classes of deformable shapes from range data. The basic idea of our component-based approach is to generalize existing surface representations that have proven effective in recognizing specific 3D objects to the problem of object classes using our newly introduced symbolic-signature representation that is robust to deformations, as opposed to a numeric representation that is often tied to a specific shape. Based on this approach, we present a system that is capable of recognizing and classifying a variety of object shape classes from range data. We demonstrate our system in a series of large-scale experiments that were motivated by specific applications in scene analysis and medical diagnosis.

Craniosynostosis is a serious and common disease of children, caused by premature fusion of the sutures of the skull. The resulting abnormal skull growth can lead to severe deformity, increased intra-cranial pressure, vision, hearing and breathing problems. In this work we develop an algorithmic framework to accurately classify deformations caused by sagittal craniosynostosis. The basic idea is to combine our novel cranial image shape descriptors and off-the-shelf classification technologies to encode morphological variations that characterize the synostotic skull. We demonstrate the efficacy of our approach in a series of large-scale classification experiments that compare the performance of our proposed image descriptors to those of traditional clinical indices and Fourier-based measurements.

Most of the work on 3D object recognition from range data has used an alignment-verification approach in which a specific 3D object is matched to an exact instance of the same object in a scene. This approach has been successfully used in industrial machine vision, but it is not capable of dealing with the complexities of recognizing classes of similar objects. This paper undertakes this task by proposing and testing a component-based methodology encompassing three main ingredients: 1) a new way of learning and extracting shape-class components from surface shape information; 2) a new shape representation called a symbolic surface signature that summarizes the geometric relationships among components; and 3) an abstract representation of shape classes formed by a hierarchy of classifiers that learn object-class parts and their spatial relationships from examples.

We present and empirically test a novel approach for categorizing 3-D free form ob- ject shapes represented by range data . In contrast to traditional surface-signature based systems that use alignment to match specific objects, we adapted the newly introduced symbolic-signature representation to classify deformable shapes [10]. Our approach con- structs an abstract description of shape classes using an ensemble of classifiers that learn object class parts and their corresponding geometrical relationships from a set of numeric and symbolic descriptors. We used our classification engine in a series of large scale dis- crimination experiments on two well-defined classes that share many common distinctive features. The experimental results suggest that our method outperforms traditional numeric signature-based methodologies. 1

The paper considers the problem of shape-based recognition and pose estimation of 3D free-form objects in scenes that contain occlusion and clutter. Our approach is based on a novel set of discriminating descriptors called spherical spin images, which encode the shape information conveyed by classes of distributions of surface points constructed with respect to reference points on the surface of an object. The key to this approach is the relationship that exists between the l/sub 2/ metric, which compares n-dimensional signatures in Euclidean space, and the metric of the compact space on which the class representatives (spherical spin images) are defined. The connection allows us to efficiently utilize the linear correlation coefficient to discriminate scene points which have spherical spin images that are similar to the spherical spin images of points on the object being sought. The paper also addresses the problem of compressed spherical-spin-image representation by means of a random projection of the original descriptors that reduces the dimensionality without a significant loss of recognition/localization performance. Finally, the efficacy of the proposed representation is validated in a comparative study of the two algorithms presented that use uncompressed and compressed spherical spin images versus two previous spin image algorithms reported previously (A.E. Johnson and M. Hebert, 1999). The results of 2012 experiments suggest that the performance of our proposed algorithms is significantly better with respect to accuracy and speed than the performance of the other algorithms tested.

The relation between morphological gray-level connected operators and segmentation algorithms based on region merging/classification strategies has been pointed out several times in the literature. However, to the best of our knowledge, the formal relation between them has not been established. This paper presents the link between the two domains based on the observation that both connected operators and segmentation algorithms share a key mechanism: they simultaneously operate on images and on partitions, and therefore they can be described as operations on a joint image-partition model. As a result, we analyze both segmentation algorithms and connected operators by defining operators on complete product lattices, that explicitly model gray-level and partition attributes. In the first place, starting with a complete lattice of partitions, we initially define the concept of the segmentation model as a mapping in a product lattice, whose elements are three-tuples consisting of a partition, an image that models the partition attributes, and an image that represents the gray-level model associated to the segmentation. Then, assuming a conditional ordering relation, we show that any region merging/classification segmentation algorithm can be defined as an extensive operator in such a complete product lattice, in the second place, we proposed a very similar lattice-based extended representation of gray-level functions in the context of connected operators, that highlights the mathematical analogy with segmentation algorithms, but in which the ordering relation is different. We use this framework to show that every region merging/classification segmentation algorithm indeed corresponds to a connected operator. While this result provides an explanation to previous work in the area, it also opens possibilities for further analysis in the two domains. From this perspective, we additionally study some theoretical properties of a general region merging segmentation algorithm.