Abstract This paper proposes an approach for training visuo-haptic object recognition models for robots using synthetic datasets generated by 3D virtual simulations. In robotics, where visual object recognition has witnessed considerable progress due to an abundance of image datasets, the scarcity of diverse haptic samples has resulted in a noticeable gap in research on machine learning incorporating the haptic sense. Our proposed methodology addresses this challenge by utilizing 3D virtual simulations to create realistic synthetic datasets, offering a scalable and cost-effective solution to integrate haptic and visual cues for object recognition seamlessly. Acknowledging the importance of multimodal perception, particularly in robotic applications, our research not only closes the existing gap but envisions a future where intelligent agents possess a holistic understanding of their environment derived from both visual and haptic senses. Our experiments show that synthetic datasets can be used for training object recognition in haptic and visual modes by incorporating noise, performing some preprocessing, data augmentation, or domain adaptation. This work contributes to the advancement of multimodal machine learning toward a more nuanced and comprehensive robotic perception.

The use of image processing and classification for agricultural applications has been widely studied and has led to work such as the automatic grading of fruit and vegetables, yield approximation and defect detection. Image segmentation is one of the first steps to identify the region of interest within an image. This paper presents an approach to automatic segmentation and classification of healthy and defective Carabao mangoes. K-means, range filtering and color-channel segmentation were utilized so that the varying texture and color of mangoes due to the surface defects can be considered. Results show that the proposed technique performs better than the classical K-means segmentation. The performance of segmentation step has a considerable influence on the precision of the classification model. Segmented and not segmented images were trained using KNN, SVM, MLP and CNN. The experiments showed that the models performed better when trained with segmented images.

The increasing incidence of melanoma has led to development of computer-aided diagnosis systems that classify dermoscopic images. A fundamental problem however during the pre-processing stage is the removal of artifacts such as hair. Hair strands introduce additional edges, which can be problematic when performing automatic skin lesion segmentation. This paper proposes a straightforward approach to automatic hair and consequently noise removal. The process starts with a median filter on each color space of RGB, a bottom hat filter, a binary conversion, a dilation and morphological opening, and then the removal of small connected pixels. The detected hair regions are then filled up using harmonic inpainting. Experiments were carried out on the PH2 datasets and compared to DullRazor. We also generated synthetic hair on skin images and measured the reconstruction quality using peak signal-to-noise ratio.

Abstract Paper pop‐ups are interesting three‐dimensional books that fascinate people of all ages. The design and construction of these pop‐up books however are done manually and require a lot of time and effort. This has led to computer‐assisted or automated tools for designing paper pop‐ups. This paper proposes an approach for automatically converting a 3D model into a multi‐style paper pop‐up. Previous automated approaches have only focused on single‐style pop‐ups, where each is made of a single type of pop‐up mechanisms. In our work, we combine multiple styles in a pop‐up, which is more representative of actual artist's creations. Our method abstracts a 3D model using suitable primitive shapes that both facilitate the formation of the considered pop‐up mechanisms and closely approximate the input model. Each shape is then abstracted using a set of 2D patches that combine to form a valid pop‐up. We define geometric conditions that ensure the validity of the combined pop‐up structures. In addition, our method also employs an image‐based approach for producing the patches to preserve the textures, finer details and important contours of the input model. Finally, our system produces a printable design layout and decides an assembly order for the construction instructions. The feasibility of our results is verified by constructing the actual paper pop‐ups from the designs generated by our system.

Origamic architecture (OA) is a form of papercraft that involves cutting and folding a single sheet of paper to produce a 3D pop-up, and is commonly used to depict architectural structures. Because of the strict geometric and physical constraints, OA design requires considerable skill and effort. In this paper, we present a method to automatically generate an OA design that closely depicts an input 3D model. Our algorithm is guided by a novel set of geometric conditions to guarantee the foldability and stability of the generated pop-ups. The generality of the conditions allows our algorithm to generate valid pop-up structures that are previously not accounted for by other algorithms. Our method takes a novel image-domain approach to convert the input model to an OA design. It performs surface segmentation of the input model in the image domain, and carefully represents each surface with a set of parallel patches. Patches are then modified to make the entire structure foldable and stable. Visual and quantitative comparisons of results have shown our algorithm to be significantly better than the existing methods in the preservation of contours, surfaces, and volume. The designs have also been shown to more closely resemble those created by real artists.

A paper sliceform or lattice-style pop-up is a form of papercraft that uses two sets of parallel paper patches slotted together to make a foldable structure. The structure can be folded flat, as well as fully opened (popped-up) to make the two sets of patches orthogonal to each other. Automatic design of paper sliceforms is still not supported by existing computational models and remains a challenge. We propose novel geometric formulations of valid paper sliceform designs that consider the stability, flat-foldability and physical realizability of the designs. Based on a set of sufficient construction conditions, we also present an automatic algorithm for generating valid sliceform designs that closely depict the given 3D solid models. By approximating the input models using a set of generalized cylinders, our method significantly reduces the search space for stable and flat-foldable sliceforms. To ensure the physical realizability of the designs, the algorithm automatically generates slots or slits on the patches such that no two cycles embedded in two different patches are interlocking each other. This guarantees local pairwise assembility between patches, which is empirically shown to lead to global assembility. Our method has been demonstrated on a number of example models, and the output designs have been successfully made into real paper sliceforms.

We consider a repository of animation models and motions that can be reused to generate new animation sequences. For instance, a user can retrieve an animation of a dog kicking its leg (in air) and manipulate the result to generate a new animation where the dog is kicking a ball. In this particular example, inverse kinematics technique can be used to retarget the kicking motion of a dog to a ball. This approach of reusing models and motions to generate new animation sequences can be facilitated by operations such as querying of animation databases for required models and motions, and manipulation of the query results to meet new constraints. However, manipulation operations such as motion retargeting are quite complex in nature. Hence, there is a need for visualizing the queries on animation databases as well as the manipulation operations on the query results. In this paper, we propose a visually interactive method for reusing motions and models, by adjusting the query results from animation databases for new situations while at the same time, keeping the desired properties of the original models and motions. Here, a user first queries for animation objects, i.e., geometric models and motions. Then, the user interactively makes new animations by visually manipulating the query results. Depending on the orders in which the GUIs (Graphical User Interfaces) are invoked and the parameters are changed, the system automatically generates a sequence of operations, a list of SQL-like syntax commands, and applies it to the query results of motions and models. With the help of visualization tools, the user can view the changes before accepting them.