Referring to documents is common when making things, but there is a difficulty caused by the gap between a written description and the actual context of making. For example, when cooking following a recipe, people may lose their current position in the recipe, misunderstand the required amount of ingredients because of complicated measuring units, or skip steps by mistake. We address these problems by selecting cooking as our domain. Our proposed cooking support system, MimiCook, embodies a recipe in a real kitchen counter and directly navigates a user. The system consists of a computer, a depth camera, a projector, and a scaling device. It displays step-by-step instructions directly onto the utensils and ingredients, and controls the guidance display in accordance with the user's situations. The integrated scaling device also helps users to avoid mistakes with measuring units. Results of our user study shows participants found it easier to cook with the system and even subjects who had never cooked the assigned recipe did not make any mistakes.

Although many cooking recipes have been shared in the world, there are still many homemade meals that are never written down as recipes. The difficulty to create recipes of such homemade meals lies on (1) experienced cookers might not have accurate information of ingredients (e.g., weight and timing) since they often decide the amount of ingredients just by intuition, (2) they often have trouble to record such accurate information while cooking. To solve these problems, we propose a smart cutting board, CookTab, which allows a user to easily record his/her cooking activities (e.g., the names and quantities of ingredients) while provides real-time feedback to motivate him/her.