We investigate the possibility of summarizing meal image collections in an image-based dietary assessment system. A segmentation algorithm detects food regions in each meal image. Pairwise matching of images based on color and texture information of these regions forms a similarity matrix of the image collection. We cluster the nodes of the graph constructed using this matrix, to identify natural groupings of meal images according to their content. Representative images from these clusters form summaries of the large image collection. We conduct a user study to evaluate the effective ness of the proposed algorithms in summarizing meal image collections, and report the results.

In this paper, we present an image-analysis based approach to calorie content estimation for dietary assessment. We make use of daily food images captured and stored by multiple users in a public Web service called FoodLog. The images are taken without any control or markers. We build a dictionary dataset of 6512 images contained in FoodLog the calorie content of which have been estimated by experts in nutrition. An image is compared to the ground truth data from the point of views of multiple image features such as color histograms, color correlograms and SURF fetures, and the ground truth images are ranked by similarities. Finally, calorie content of the input food image is computed by linear estimation using the top n ranked calories in multiple features. The distribution of the estimation shows that 79% of the estimations are correct within ±40% error and 35% correct within ±20% error.

Food images have been receiving increased attention in recent dietary control methods. We present the current status of our web-based system that can be used as a dietary management support system by ordinary Internet users. The system analyzes image archives of the user to identify images of meals. Further image analysis determines the nutritional composition of these meals and stores the data to form a Foodlog. The user can view the data in different formats, and also edit the data to correct any mistakes that occurred during image analysis. This paper presents detailed analysis of the performance of the current system and proposes an improvement of analysis by pre-classification and personalization. As a result, the accuracy of food balance estimation is significantly improved.

We propose a system for retrieving multimedia related to a person's travel, using location data captured with a GPS receiver, mobile phone or a camera. The user makes simple sketches on a map displayed on a computer screen, to submit spatial, temporal or spatio-temporal queries regarding his travel. The system segments the location data and images, analyzes sketches made by a user, identifies the query, and retrieves relevant results. These results, combined with online maps and virtual tours rendered using street view panoramas, form multimedia travel stories. We present the system's current status and conclude with future directions.

We propose a system for retrieving human locomotion patterns from tracking data captured within a large geographical area, over a long period of time. A GPS receiver continuously captures data regarding the location of the person carrying it. A constrained agglomerative hierarchical clustering algorithm segments these data according to the person's navigational behavior. Sketches made on a map displayed on a computer screen are used for specifying queries regarding locomotion patterns. Two basic sketch primitives, selected based on a user study, are combined to form five different types of queries. We implement algorithms to analyze a sketch made by a user, identify the query, and retrieve results from the collection of data. A graphical user interface combines the user interaction strategy and algorithms, and allows hierarchical querying and visualization of intermediate results. We evaluate the system using a collection of data captured during nine months. The constrained hierarchical clustering algorithm is able to segment GPS data at an overall accuracy of 94% despite the presence of location-dependent noise. A user study was conducted to evaluate the proposed user interaction strategy and the usability of the overall system. The results of this study demonstrate that the proposed user interaction strategy facilitates fast querying, and efficient and accurate retrieval, in an intuitive manner.

We present the current status of a system based on a network of sensors for event retrieval from a home like environment. A large number of cameras, microphones and pressure based floor sensors are used for continuous data capture. The data are analyzed independently and the results recorded in a central database, where they are combined for efficient retrieval and summarization of the video and audio data. We describe the detection of basic actions, events, and faces using image analysis. The users can query the system interactively to retrieve video, audio, and key frames corresponding to events. We report results of performance evaluations of the algorithms and discuss issues related to sensing, analysis and retrieval of data.

A system for retrieving video sequences created by tracking humans in a smart environment, by using spatial queries, is presented. Sketches made on a graphical user interface using a pointing device are used as the means of entering multiple types of queries. After preprocessing and coordinate system conversion, the sketches are analyzed to identify the type of the query. Directional search algorithms based on the minimum distance between points is applied for finding the best matches to the sketch. The results are ranked according to the similarity and presented to the user. The results of an initial evaluation are reported. The paper concludes with an outline of possible future directions.

A system for retrieving video captured in a ubiquitous environment is presented. Data from pressure-based floor sensors are obtained as a supplementary input together with video from multiple stationary cameras. Unsupervised data mining techniques are used to reduce noise present in floor sensor data. An algorithm based on agglomerative hierarchical clustering is used to segment footpaths of individual persons. Video handover is proposed and two methods are implemented to retrieve video and key frame sequences showing a person moving in the house. Users can query the system based on time and retrieve video or key frames using either of the handover techniques. We compare the results of retrieval using different techniques subjectively. We conclude with suggestions for improvements, and future directions.

A system for video summarization in a ubiquitous environment is presented. Data from pressure-based floor sensors are clustered to segment footsteps of different persons. Video handover has been implemented to retrieve a continuous video showing a person moving in the environment. Several methods for extracting key frames from the resulting video sequences have been implemented, and evaluated by experiments. It was found that most of the key frames the human subjects desire to see could be retrieved using an adaptive algorithm based on camera changes and the number of footsteps within the view of the same camera. The system consists of a graphical user interface that can be used to retrieve video summaries interactively using simple queries.