Zooming and panning are fundamental input actions for exploring complex 2D and 3D scenes and data such as images, maps, and designs. Multi-touch zoom/pan interactions have been proven effective on mobile devices, and have been directly ported to HMDs, where they are typically accomplished by analogous but relatively large-scale movements of both hands. We argue that such motions are inefficient and induce fatigue and explore how the eye-tracking features of HMDs can be leveraged to achieve improvements. We evaluated three interaction techniques that combine gaze with two-handed, one-handed, and head-based input in a study (N = 24) that contrasts them against a baseline two-handed technique. The results indicate that gaze-assisted two- and one-handed techniques outperform the baseline (17%-36% faster), while our head-based technique achieves similar performance to the Baseline but leaves the hands free for other tasks. We further developed a VR application demonstrating these techniques and validating their practical applicability.

This study investigates how avatar customization in virtual reality (VR) impacts trust formation between unacquainted individuals and how such trust transfers to subsequent face-to-face (FtF) meetings. A user study with 48 participants was conducted, where participants were assigned to either a “Similar-Self” condition, with avatars resembling their real-world appearance, or an “Alt-Self” condition, with creative avatars. The results showed that “Similar-Self” avatars led to higher initial integrity-based trust perceptions, though both avatar conditions exhibited similar trust growth during VR encounters. Trust carried over from VR to FtF with a brief recalibration period and ultimately increased beyond VR levels in FtF encounters. This research provides insights into how VR can support the development of trust in early-stage interactions and offers implications for Social VR platforms to better support trustworthy interactions across virtual-physical boundaries.

The more-than-human field has contributed numerous opportunities for interacting with nature, animals, plants, and microorganisms. However, few studies have examined water ecosystems. Current water-related work primarily treats water as a medium for human-centered activities, rarely positioning water as an interactive subject. Building upon prior research, we explore how to better integrate water more playfully into digital-physical interactions as an interactive subject. We designed and developed Water’s Echo, an AI-powered public installation that enables human-water communication through a playful stone-throwing dialogue. We conducted a field study at a local pond, recruiting 15 residents to participate in Water’s Echo—a playful conversational interaction. Our findings indicate that this playful dialogue approach raises participants’ awareness and understanding of surrounding aquatic environments. This research provides insights for design researchers to establish engaging water ecology interactions across cultural communities, promoting a More-than-human perspective in re-examining human-nature relationships.

Thanks to the development of virtual reality (VR) technology, there is growing research on VR avatar body deformation effects. However, previous research mainly focused on passive body deformation expression, leaving users with limited methods to actively control their virtual bodies. To address this gap, we explored user-controlled forearm deformation by investigating how hand and forearm gestures can be mapped to various degrees of avatar forearm deformation. We conducted a gesture design workshop with six designers to generate gesture sets for different forearm deformations and deformation degrees, resulting in 15 gesture sets. Then, we selected the three highest-rated gesture sets and conducted a comparative study to evaluate the sense of embodiment and user performance across the three gesture sets. Our findings provide design suggestions for gesture-controlled forearm deformation in VR.

The purpose of multi-label image classification is to assign multiple labels for multiple objects presented in one image. Recent research efforts exploit graph convolution network (GCN) to learn the label co-occurrence dependencies for enhancing the semantic representation. Although these methods have achieved promising results, they can not capture the intrinsic correlation between objects in images and do not consider the inter-channel relationship. In addition, the previous methods treat each single image independently and fail to explore the relationship between different images. To address the above challenges, we propose a novelDualRelationGraphNetwork (DRGN) model, which adopts a double branch structure to excavate rich semantic information from intra-image and cross-image simultaneously. Specifically, we first develop an intra-image channel-relation mining (ICM) module to mine the inter-channel relationship in features while learning the importance of different channels. Secondly, we design a new GCN-based intra-image spatial-relation exploring (ISE) module to capture the correlation between objects in individual image. Notably, ISE module and ICM module can complement and promote each other from the spatial and channel dimensions of images to improve the correlation between objects in individual image. Thirdly, we propose a novel GCN-based cross-image semantic learning (CSL) module to learn the semantic relationship between different images in the mini-batch. Through graph reasoning, our CSL module can iteratively refine input image features by acquiring common semantic information from other images in the mini-batch. Extensive experiments on the MS-COCO 2014, PASCAL VOC 2007, and VG-500 datasets demonstrate that the proposed DRGN model outperforms current state-of-the-art methods.

Image captioning has made significant achievement through developing feature extractor and model architecture. Recently, the image region features extracted by object detector prevail in most existing models. However, region features are criticized for the lacking of background and full contextual information. This problem can be remedied by providing some complementary visual information from patch features. In this paper, we propose a Double-Stream Position Learning Transformer Network (DSPLTN) which exploits the advantages of region features and patch features. Specifically, the region-stream encoder utilizes a Transformer encoder with Relative Position Learning (RPL) module to enhance the representations of region features through modeling the relationships between regions and positions respectively. As for the patch-stream encoder, we introduce convolutional neural network into the vanilla Transformer encoder and propose a novel Convolutional Position Learning (CPL) module to encode the position relationships between patches. CPL improves the ability of relationship modeling by combining the position and visual content of patches. Incorporating CPL into the Transformer encoder can synthesize the benefits of convolution in local relation modeling and self-attention in global feature fusion, thereby compensating for the information loss caused by the flattening operation of 2D feature maps to 1D patches. Furthermore, an Adaptive Fusion Attention (AFA) mechanism is proposed to balance the contribution of enhanced region and patch features. Extensive experiments on MSCOCO demonstrate the effectiveness of the double-stream encoder and CPL, and show the superior performance of DSPLTN.

Currently, attention mechanism receives enormous interest and has been extensively employed in the fields of Person Re-Identification (RE-ID), as it gains superior performance in learning discriminative feature representations. However, most off-the-shelf attention methods are still vulnerable to cross-view inconsistency problem. Besides, they merely exploit imprecise channel attention information and coarse-grained spatial attention of homogeneous scales, being insufficient to capture subtle differences among highly-similar individuals. To this end, we propose a novel Attention-Aligned Network (AANet) to address the aforementioned problems, in which a novel Omnibearing Foreground-aware Attention (OFA) module, Attention Alignment Mechanism (AAM) and an improved triplet loss with hard mining are proposed to learn foreground attentive features for RE-ID. Specifically, AANet firstly leverages OFA module to exploit heterogeneous-scale spatial attention and foreground-aware channel attention information. Then AANet further reduces the impact of background clutter and learns camera-invariant and background-invariant representations by virtue of AAM. Last but not least, an improved triplet loss with hard mining is also introduced to enhance the feature learning capability, which can jointly minimize the intra-class distance and maximize the inter-class distance in each triplet unit. Extensive experiments are carried out to demonstrate that the proposed method outperforms most current methods on three main RE-ID benchmarks.

Image Captioning, which automatically describes an image with natural language, is regarded as a fundamental challenge in computer vision. In recent years, significant advance has been made in image captioning through improving attention mechanism. However, most existing methods construct attention mechanisms based on singular visual features, such as patch features or object features, which limits the accuracy of generated captions. In this article, we propose a Bidirectional Co-Attention Network (BCAN) that combines multiple visual features to provide information from different aspects. Different features are associated with predicting different words, and there are a priori relations between these multiple visual features. Based on this, we further propose a bottom-up and top-down bi-directional co-attention mechanism to extract discriminative attention information. Furthermore, most existing methods do not exploit an effective multimodal integration strategy, generally using addition or concatenation to combine features. To solve this problem, we adopt the Multivariate Residual Module (MRM) to integrate multimodal attention features. Meanwhile, we further propose a Vertical MRM to integrate features of the same category, and a Horizontal MRM to combine features of the different categories, which can balance the contribution of the bottom-up co-attention and the top-down co-attention. In contrast to the existing methods, the BCAN is able to obtain complementary information from multiple visual features via the bi-directional co-attention strategy, and integrate multimodal information via the improved multivariate residual strategy. We conduct a series of experiments on two benchmark datasets (MSCOCO and Flickr30k), and the results indicate that the proposed BCAN achieves the superior performance.

In the task of Visual Question Answering (VQA), most state-of-the-art models tend to learn spurious correlations in the training set and achieve poor performance in out-ofdistribution test data.Some methods of generating counterfactual samples have been proposed to alleviate this problem.However, the counterfactual samples generated by most previous methods are simply added to the training data for augmentation and are not fully utilized.Therefore, we introduce a novel selfsupervised contrastive learning mechanism to learn the relationship between original samples, factual samples and counterfactual samples.With the better cross-modal joint embeddings learned from the auxiliary training objective, the reasoning capability and robustness of the VQA model are boosted significantly.We evaluate the effectiveness of our method by surpassing current state-of-the-art models on the VQA-CP dataset, a diagnostic benchmark for assessing the VQA model's robustness.

While the intersection of blockchains and the Internet of Things (IoT) have received considerable research interest lately, Nakamoto-style blockchains possess a number of qualities that make them poorly suited for many IoT scenarios. Specifically, they require high network connectivity and are power-intensive. This is a drawback in IoT environments where battery-constrained nodes form an unreliable ad hoc network such as in digital agriculture. In this paper we present Vegvisir, a partition-tolerant blockchain for use in power-constrained IoT environments with limited network connectivity. It is a permissioned, directed acyclic graph (DAG)-structured blockchain that can be used to create a shared, tamperproof data repository that keeps track of data provenance. We discuss the use cases, architecture, and challenges of such a blockchain.