Pre-trained multi-modal models, such as CLIP, provide transferable embeddings and show promising results in diverse applications. However, the analysis of learned multi-modal embeddings is relatively unexplored, and the embedding transferability can be improved. In this work, we observe that CLIP holds separated embedding subspaces for two different modalities, and then we investigate it through the lens of \textit{uniformity-alignment} to measure the quality of learned representation. Both theoretically and empirically, we show that CLIP retains poor uniformity and alignment even after fine-tuning. Such a lack of alignment and uniformity might restrict the transferability and robustness of embeddings. To this end, we devise a new fine-tuning method for robust representation equipping better alignment and uniformity. First, we propose a \textit{Geodesic Multi-Modal Mixup} that mixes the embeddings of image and text to generate hard negative samples on the hypersphere. Then, we fine-tune the model on hard negatives as well as original negatives and positives with contrastive loss. Based on the theoretical analysis about hardness guarantee and limiting behavior, we justify the use of our method. Extensive experiments on retrieval, calibration, few- or zero-shot classification (under distribution shift), embedding arithmetic, and image captioning further show that our method provides transferable representations, enabling robust model adaptation on diverse tasks.

Previous deep learning-based line segment detection (LSD) suffers from the immense model size and high computational cost for line prediction. This constrains them from real-time inference on computationally restricted environments. In this paper, we propose a real-time and light-weight line segment detector for resource-constrained environments named Mobile LSD (M-LSD). We design an extremely efficient LSD architecture by minimizing the backbone network and removing the typical multi-module process for line prediction found in previous methods. To maintain competitive performance with a light-weight network, we present novel training schemes: Segments of Line segment (SoL) augmentation, matching and geometric loss. SoL augmentation splits a line segment into multiple subparts, which are used to provide auxiliary line data during the training process. Moreover, the matching and geometric loss allow a model to capture additional geometric cues. Compared with TP-LSD-Lite, previously the best real-time LSD method, our model (M-LSD-tiny) achieves competitive performance with 2.5% of model size and an increase of 130.5% in inference speed on GPU. Furthermore, our model runs at 56.8 FPS and 48.6 FPS on the latest Android and iPhone mobile devices, respectively. To the best of our knowledge, this is the first real-time deep LSD available on mobile devices.

Human-Object Interaction (HOI) detection is the task of identifying a set of (human, object, interaction) triplets from an image. Recent work proposed transformer encoder-decoder architectures that successfully eliminated the need for many hand-designed components in HOI detection through end-to-end training. However, they are limited to single-scale feature resolution, providing suboptimal performance in scenes containing humans, objects, and their interactions with vastly different scales and distances. To tackle this problem, we propose a Multi-Scale TRansformer (MSTR) for HOI detection powered by two novel HOI-aware deformable attention modules called Dual-Entity attention and Entity-conditioned Context attention. While existing deformable attention comes at a huge cost in HOI detection performance, our proposed attention modules of MSTR learn to effectively attend to sampling points that are essential to identify interactions. In experiments, we achieve the new state-of-the-art performance on two HOI detection benchmarks.

This study presents an age and gender estimation system that considers ethnic difference in face images using a Convolutional Neural Network(CNN) and Support Vector Machine(SVM). Most age and gender estimation systems using face images are trained on ethnicity-biased databases. Therefore, these systems show limited performance on face images of ethnic groups occupying a small proportion of the training data. To resolve this problem, we propose an age and gender estimation system that considers the ethnic difference in face images. At the first stage of the system, the ethnicity of the facial image is determined by a CNN trained with manually collected face images of Asian and non-Asian celebrities. Then, one of the SVM classifiers is selected according to the ethnicity for the final age and gender estimation. We compared the proposed system with an estimation system that does not consider ethnic difference. The result shows improved performance for age estimation but no improvement for gender recognition.

We present a baseline convolutional neural network (CNN) structure and image preprocessing methodology to improve facial expression recognition algorithm using CNN. To analyze the most efficient network structure, we investigated four network structures that are known to show good performance in facial expression recognition. Moreover, we also investigated the effect of input image preprocessing methods. Five types of data input (raw, histogram equalization, isotropic smoothing, diffusion-based normalization, difference of Gaussian) were tested, and the accuracy was compared. We trained 20 different CNN models (4 networks × 5 data input types) and verified the performance of each network with test images from five different databases. The experiment result showed that a three-layer structure consisting of a simple convolutional and a max pooling layer with histogram equalization image input was the most efficient. We describe the detailed training procedure and analyze the result of the test accuracy based on considerable observation.

The solar powered energy harvesting sensor node is a key technology for Internet of Things (IoT), but currently it offers only a small amount of energy storage and is capable of harvesting only a trivial amount of energy. Therefore, new technology for managing the energy associated with this sensor node is required. In particular, it is important to manage the transmission interval because the level of energy consumption during data transmission is the highest in the sensor node. If the proper transmission interval is calculated, the sensor node can be used semi‐permanently. In this study, the authors propose an energy prediction algorithm that uses the light intensity of fluorescent lamps in an indoor environment. The proposed algorithm can be used to accurately estimate the amount of energy that will be harvested by a solar panel using a weighted average for light intensity. Then, the optimal transmission interval is calculated using the amount of predicted harvested energy and residual energy. The results from the authors’ experimental testbeds show that their algorithm's performance is better than the existing approaches. The energy prediction error of their algorithm is approximately 0.5%.

Needs and studies on service robot are growing with rapid market extension. Generating human-like motions on robot is one research area of human-robot interaction. There are several studies about motion generation and humanoid design. However, kinematic analysis of humanoid design for service tasks should also performed. This paper focuses on motor alignment of humanoid shoulder. After we measure human greeting gestures with 3D capture camera and predefine workspace, simulation is performed to find motor alignment on humanoid shoulder to minimize total required motor power. Result shows that there are specific alignment for designing shoulder to perform greeting tasks on small motor powers.

In this paper, we propose a mobility-based prediction algorithm with dynamic LGD (Link Going Down) triggering for vertical handover by applying the IS (information server) of IEEE 802.21 MIH (media independent handover). The proposed algorithm predicts a possible moving area (PMA) of the mobile terminal based on mobility information (the velocity, coordinate values, position, movement detection, etc) in IS. Since the PMA indicates a next target cell for handover, it can advance the LGD trigger point dynamically to prepare for handover beforehand. The analytical results show that our prediction algorithm can reduce handover latency for MIPv6 (mobile IPv6), FMIPv6 (fast handover for mobile IPv6) by advancing the LGD trigger point.