Zoos play a crucial role in public education and wildlife engagement, yet traditional visits often lack interactive elements that foster meaningful connections between visitors and animals. In this paper, we introduce ZooWear, a head-mounted wearable featuring cartoon-style animal ears that provides haptic feedback patterns simulating an animal’s reactions to other species in the food chain. With ZooWear, we focused on examining its effectiveness in affording perspective-taking during human-animal encounters, promoting embodied knowledge retention, and enriching the zoo experiences. We first conducted a between-subject experiment in lab-based virtual zoo visits to evaluate its effectiveness in creating effective learning experiences and enhancing connections to wildlife. This was followed by a real-world zoo experiment, which showed that ZooWear promoted nature connectedness and enabled more emotionally and socially engaging experiences. Our findings highlight the potential of integrating perspective-taking into zoo experiences through animal-inspired wearables, embodied sensory feedback, and narrative-driven experiences.

Traffic prediction is pivotal in intelligent transportation systems. Existing works focus mainly on improving overall accuracy, overlooking a crucial problem of whether prediction results will lead to biased decisions by transportation authorities. In practice, the uneven deployment of traffic sensors in different urban areas produces imbalanced data, making the traffic prediction model fail in some urban areas and leading to unfair regional decision-making that eventually severely affects equity and quality of residents’ life. Existing fairness machine learning models struggle to maintain fair traffic prediction over prolonged periods. Although these models might achieve fairness at certain time slots, this static fairness will break down as traffic conditions change. To fill this research gap, we investigate prolonged fair traffic prediction, introducing two novel fairness metrics, i.e., region-based static fairness and sensor-based dynamic fairness, tailored to fairness fluctuations over time and across areas. An innovative prolonged fairness traffic prediction framework, namely FairTP, is then proposed. FairTP achieves prolonged fairness by alternating between “sacrifice” and “benefit” the prediction accuracy of each traffic sensor or area, ensuring that the number of these two actions are balanced over time. Specifically, FairTP incorporates a state identification module to discriminate whether the traffic sensors or areas are in a “sacrifice” or “benefit” state, thereby enabling prolonged fairness-aware traffic predictions. Additionally, we devise a state-guided balanced sampling strategy to select training examples to further enhance prediction fairness by mitigating the performance disparities among areas with uneven sensor distribution over time. Extensive experiments in two real-world datasets show that FairTP significantly improves prediction fairness without causing significant accuracy degradation.

Human pose estimation in computer vision predominantly focuses on the visible modality, with limited research on the infrared modality. No existing methods demonstrate robust performance across both modalities, missing their complementary strengths. This gap arises from the lack of a multimodal benchmark and the difficulty of developing robust multimodal capabilities. To address this, we introduce MMPD, a novel visible-infrared multimodal pose benchmark with high-quality annotations for both modalities. Leveraging MMPD, we expose the limitations of state-of-the-art methods due to modality variance. To overcome this challenge, we propose a novel method-agnostic scheme called AMMPE. By employing the Modality Adversarial Enhancement Stage and Modality Interaction Stage, AMMPE easily incorporates multimodal information without additional pose annotations and enhances effective modality interaction. Extensive experiments demonstrate that AMMPE improves performance in both visible and infrared modalities, achieving excellent modality robustness. The code and benchmark is avaible at: https://github.com/ICANDOALLTHINGSSS/Adversarial-Multi-Modality-Pose-Estimation

This paper presents a novel Two-Stage Diffusion Model (TS-Diff) for enhancing extremely low-light RAW images. In the pre-training stage, TS-Diff synthesizes noisy images by constructing multiple virtual cameras based on a noise space. Camera Feature Integration (CFI) modules are then designed to enable the model to learn generalizable features across diverse virtual cameras. During the aligning stage, CFIs are averaged to create a target-specific CFIT, which is fine-tuned using a small amount of real RAW data to adapt to the noise characteristics of specific cameras. A structural reparameterization technique further simplifies CFITfor efficient deployment. To address color shifts during the diffusion process, a color corrector is introduced to ensure color consistency by dynamically adjusting global color distributions. Additionally, a novel dataset, QID, is constructed, featuring quantifiable illumination levels and a wide dynamic range, providing a comprehensive benchmark for training and evaluation under extreme low-light conditions. Experimental results demonstrate that TS-Diff achieves state-of-the-art performance on multiple datasets, including QID, SID, and ELD, excelling in denoising, generalization, and color consistency across various cameras and illumination levels. These findings highlight the robustness and versatility of TS-Diff, making it a practical solution for low-light imaging applications. Source codes and models are available at https://github.com/CircccleK/TS-Diff

Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guoqiang Wei, Yang Wei, Yuchen Zhang, Tao Kong, Ruihua Song. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

Fast and continuous data acquisition (DAQ) with well resolved spectral information is essential for high-speed and high-fidelity measurement of thermophysical parameters of industrial processes using laser absorption spectroscopy tomography (LAST). However, the state-of-the-art DAQ systems suffer: inability to collect raw spectral data in real-time due to the very high data throughput; degradation of spectral integrity when excessive on-chip down-sampling is implemented to reduce data throughput. In this article, we designed a star-networked and reconfigurable DAQ system for real-time LAST imaging at kilo-Hz frame rate. The DAQ system is embedded with a new field programmable gate array (FPGA)-accelerated digital lock-in technique, whereby a cascaded integrator-comb (CIC) filter is implemented for down-sampling of the raw signal with well-maintained spectral information. Furthermore, a customized data-encapsulation protocol is developed to enable continuity of real-time data communication between the front-end DAQ hubs and back-end processor. Performance of the developed DAQ system is experimentally validated by flame temperature imaging at 1 kHz, providing the necessary temporal resolution to penetrate turbulent flow and related industrial processes such as reaction propagation.

POI recommendation is practically important to facilitate various Location-Based Social Network (LBSN) services, and has attracted rising research attention recently. Existing works generally assume the available POI check-ins reported by users are the ground-truth depiction of user behaviors. However, in real application scenarios, the check-in data can be rather unreliable (e.g. sparse, incomplete and inaccurate) due to both subjective and objective causes including positioning error and user privacy concerns. The data uncertainty issue may lead to significant negative impacts on the performance of the POI recommendation, but is not fully explored by existing works. To this end, this paper investigates a novel problem of robust POI recommendation by considering the uncertainty factors of the user check-ins, and proposes a Bayes-enhanced Multi-view Attention Network (BayMAN for short) to effectively address it. Specifically, we construct three POI graphs to comprehensively model the dependencies among the POIs from different views, including the personal POI transition graph, the semantic-based POI graph and distance-based POI graph. As the personal POI transition graph is usually sparse and sensitive to noise, we design a Bayes-enhanced spatial dependency learning module for data augmentation from the local view. A Bayesian posterior guided graph augmentation approach is adopted to generate a new graph with collaborative signals to increase the data diversity. Then both the original and the augmented graphs are used for POI representation learning to counteract the data uncertainty issue. Next, the POI representations of the three view graphs are input into the proposed multi-view attention-based user preference learning module. By incorporating the semantic and distance correlations of POIs, the user preference can be effectively refined and finally robust recommendation results are achieved. We conduct extensive experiments over three real-world LSBN datasets. The results show that BayMAN significantly outperforms the state-of-the-art methods in POI recommendation when the available check-ins are incomplete and noisy.

Tunable diode laser absorption spectroscopy tomography (TDLAST) has been widely applied for imaging two-dimensional distributions of industrial flow-field parameters, e.g., temperature and species concentration. Two main interested imaging objectives in TDLAST are the local combustion and its radiation in the entire sensing region. State-of-the-art algorithms were developed to retrieve either of the two objectives. In this article, we address both by developing a novel multioutput imaging neural network, named as spatially progressive neural network (SpaProNet). This network consists of locally and globally prioritized reconstruction stages. The former enables hierarchical imaging of the finely resolved and highly accurate local combustion, but coarsely resolved background. The latter retrieves a fine-resolved image for the entire sensing region, at the cost of slightly trading off the reconstruction accuracy in the combustion zone. Furthermore, the proposed network is driven by the hydrodynamics of the real reactive flows, in which the training dataset is obtained from large eddy simulation. The proposed SpaProNet is validated by both simulation and lab-scale experiment. In all test cases, the visual and quantitative metric comparisons show that the proposed SpaProNet outperforms the existing methods from the following two perspectives: 1) the locally prioritized stage provides ever-better accuracy in the combustion zone; and 2) the globally prioritized stage shows turbulence-indicative accuracy in the entire sensing region for diagnosis of heat radiation from the flame and flame-air interactions.

Crowd flow prediction, which aims to predict the in/out flows of different areas of a city, plays an important role in various applications like intelligent transportation. The challenges of this problem lie in both dynamic mobility patterns of crowds and complex spatial-temporal correlations. Meanwhile, crowd flow is highly correlated to and affected by the Origin-Destination (OD) locations of the flow trajectories, which is largely ignored by existing works. In this paper, we study the novel problem of predicting the crowd flow and flow OD simultaneously, and propose a multi-task bayes-enhanced adversarial spatial temporal network entitled MBA-STNet. MBA-STNet adopts a shared-private framework that contains private spatial-temporal encoders, a shared spatial-temporal encoder, and decoders to learn the task-specific features and shared features. To effectively extract discriminative shared features, an adversarial loss on shared feature extraction is incorporated to reduce information redundancy. A Bayesian Heterogeneous Spatio-temporal Attention Network is designed to learn complex spatio-temporal correlations and alleviate data uncertainty. We also design an attentive temporal queue to capture the complex temporal dependency automatically without domain knowledge. Extensive evaluations are conducted over the bike and taxicab trip datasets in New York. The results demonstrate that the proposed MBA-STNet is superior to state-of-the-art methods.

A fundamental trade-off between effectiveness and efficiency needs to be balanced when designing an online question answering system. Effectiveness comes from sophisticated functions such as extractive machine reading comprehension (MRC), while efficiency is obtained from improvements in preliminary retrieval components such as candidate document selection and paragraph ranking. Given the complexity of the real-world multi-document MRC scenario, it is difficult to jointly optimize both in an end-to-end system. To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the documentlevel and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension. Specifically, irrelevant documents and paragraphs are first filtered out with simple functions for efficiency consideration. Then we jointly train three modules on the remaining texts for better tracking the answer: the document extraction, the paragraph extraction and the answer extraction. Experiment results show that the proposed method outperforms the previous state-of-the-art methods on two large-scale multidocument benchmark datasets, i.e., TriviaQA and DuReader. In addition, our online system can stably serve typical scenarios with millions of daily requests in less than 50ms.

This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning. Most of the traditional methods in MRC assume that the knowledge used to get the correct answer generally exists in the given documents. However, in real-world task, part of knowledge may not be mentioned and machines should be equipped with the ability to leverage external knowledge. In this paper, we integrate relational knowledge into MRC model for commonsense reasoning. Specifically, based on a pre-trained language model (LM), We design two auxiliary relation-aware tasks to predict if there exists any commonsense relation and what is the relation type be-tween two words, in order to better model the interactions between document and candidate answer option. We conduct experiments on two multi-choice benchmark datasets: the SemEval-2018 Task11 and the Cloze Story Test. The experimental results demonstrate the effectiveness of the proposed method, which achieves superior performance compared with the comparable baselines on both datasets.

Bin Bi, Chen Wu, Ming Yan, Wei Wang, Jiangnan Xia, Chenliang Li. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.