Data uncertainty refers to the degree of uncertainty in model predictions caused by data variability, challenging the model robustness in multimedia applications. The increased variability of uncontrolled face images, such as mask occlusion and image blur, increases the intra-class differences and inter-class similarities. As a result, the ambiguity in the learned features exacerbates the uncertainty of sample-to-class membership. Traditional face recognition models are deterministic point embedding models that fail to measure data uncertainty. Probabilistic embedding models, such as advanced Data Uncertainty Learning (DUL), represent each face image as a Gaussian distribution to measure data uncertainty. However, these models perform random sampling from the distribution once per training, and the sampled points may fall into different class regions, leading to training oscillation. Therefore, we propose a robust Region Uncertainty Learning (RUL) method, which adopts the entire Gaussian distribution of each sample during each training epoch, and estimates the region relations between the sample distribution region and the class region to measure the sample-to-class membership. In fact, DUL is a special case of the proposed RUL. Specifically, DUL estimates point-with-region relations and only represents absolute membership and non-membership. In contrast, RUL estimates region-with-region relations, enabling it to additionally represent incomplete membership. This more comprehensive membership measurement fully represents the uncertainty of membership, enhancing the model performance and robustness in uncontrolled scenes. Furthermore, for robust face recognition, we propose two RUL-based angular margin losses, AngleFace and RegionFace, to adaptively adjust the learning weights according to the uncertainty of membership. Finally, we comprehensively evaluate the effectiveness of RUL on various face datasets, and profoundly analyze the role of region relations. In future, we will explore the applicability of RUL in other tasks.

The expressiveness of historical reviews in capturing user preferences has garnered significant attention in recommender systems. However, this technology still has certain limitations. Firstly, irrelevant reviews can introduce noise that may adversely affect the performance of the model. Secondly, existing approaches often assume a flat structure for review features, thus failing to capture the intricate and hierarchical nature of user–item interactions. Thirdly, it is challenging for review-based recommendation models to effectively assess the usefulness of reviews due to sparse supervision signals. To address these challenges, we propose a novel Hierarchical Gating and Discriminative model for rating prediction. Specifically, we introduce a local gating module that utilizes personalized end-to-end differential thresholds to select reviews in a relatively “hard” manner, thereby minimizing the impact of noisy reviews while facilitating model training. Additionally, we incorporate a global gating module to assess the overall usefulness of review signals by estimating the uncertainties inherent in historical reviews. Moreover, we propose a hierarchical discriminative network to develop self-supervision signals at both global and local levels to guide the learning of the hierarchical gating network. Extensive experiments on public datasets have demonstrated the effectiveness of the proposed model, and further investigations provide deep insight into its superiority.

Since mask occlusion causes plentiful loss of facial feature, Masked Face Recognition (MFR) is a challenging image processing task, and the recognition results are susceptible to noise. However, existing MFR methods are mostly deterministic point embedding models, which are limited in representing noise images. Moreover, Data Uncertainty Learning (DUL) fails to achieve reasonable performance in MFR. Therefore, we propose a novel two-stream convolutional network, masked face data uncertainty learning (MaskDUL), that solves the problems by sampling uncertainty and intra-class distribution learning in MFR. Specifically, a Hard Kullback-Leibler Divergence (H-KLD) method is proposed to serve as an adaptive variance regularizer and a magnitude-based module is adopted to adaptively adjust the angular margin of different samples. Finally, insightful evaluation demonstrates the effectiveness and robustness of our MaskDUL.

In the post-epidemic era, Masked Face Recognition (MFR) is of great significance to our daily life, but it confronts a severe challenge of lacking real-world large-scale masked face datasets with identity labels.Moreover, mask enhances the diversity of face images and further improves the requirements for datasets.To address the above problem, we propose a novel CycleGAN-based masked face generation method MaskedFace-GAN (MFGAN), which is able to generate correct, authenticlooking and type-diverse masked face while ensuring the invariance of facial features.We design a three-stage training pipeline for MFGAN, which corresponds to three modules, respectively.Specifically, a facial feature detector is adopted to guide the model to generate the correct mask in the correct position.Then, by utilizing a mask binary segmentation module, the authenticity of generated images can be guaranteed.Lastly, with mask style encoder, the model can be optimized towards generating typediverse masked faces.Finally, comparing with advanced masked face synthesis and generation methods comprehensively, our MFGAN achieves the best results.Then we apply the generated masked face datasets to MFR model training, which further proves the feasibility of training MFR models on generated datasets and the effectiveness and advancement of MFGAN compared with other state-of-the-art methods.

Deep Convolutional Neural Networks (DCNNs) have enhanced the performance of semantic image segmentation but many challenges still remain. Specifically, some details may be lost due to the downsampling operations in DCNNs. Furthermore, objects may appear in an image at different scales, and extracting features using convolutional filters with large sizes is costly in computation. Moreover, in many cases, contextual information, such as global and background features, is potentially useful for semantic segmentation. In this paper, we address these challenges by proposing a Multi-Receptive Atrous Convolutional Network (MRACN) for semantic image segmentation. The proposed MRACN captures the multi-receptive features and the global features at different receptive scales of the input. MRACN can serve as a module easily being integrated into existing models. We adapt the ResNet-101 model as the backbone network and further propose a MRACN segmentation model (MRACN-Seg). The experimental results demonstrate the effectiveness of the proposed model on two datasets: a benchmark dataset (PASCAL VOC 2012) and our industry dataset.

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

In recommender systems, the user-item interaction data is usually sparse and not sufficient for learning comprehensive user/item representations for recommendation. To address this problem, we propose a novel dual-bridging recommendation model (DBRec). DBRec performs latent user/item group discovery simultaneously with collaborative filtering, and interacts group information with users/items for bridging similar users/items. Therefore, a user's preference over an unobserved item, in DBRec, can be bridged by the users within the same group who have rated the item, or the user-rated items that share the same group with the unobserved item. In addition, we propose to jointly learn user-user group (item-item group) hierarchies, so that we can effectively discover latent groups and learn compact user/item representations. We jointly integrate collaborative filtering, latent group discovering and hierarchical modelling into a unified framework, so that all the model parameters can be learned toward the optimization of the objective function. We validate the effectiveness of the proposed model with two real datasets, and demonstrate its advantage over the state-of-the-art recommendation models with extensive experiments.

The human genome can reveal sensitive information and is potentially re-identifiable, which raises privacy and security concerns about sharing such data on wide scales. In this work, we propose a preventive approach for privacy-preserving sharing of genomic data in decentralized networks for Genome-wide association studies (GWASs), which have been widely used in discovering the association between genotypes and phenotypes. The key components of this work are: a decentralized secure network, with a privacy- preserving sharing protocol, and a gene fragmentation framework that is trainable in an end-to-end manner. Our experiments on real datasets show the effectiveness of our privacy-preserving approaches as well as significant improvements in efficiency when compared with recent, related algorithms.

Unlike traditional video recommendations, micro-video inherits the characteristics of social platforms, such as social relation. A large amount of micro-videos showing explosive growth is badly affecting the user’s choice. In this paper, we propose a multi-source multi-net micro-video recommendation model that recommends micro-videos fitting users’ best interests. Different from existing works, as micro-video inherits the characteristics of social platforms, we simultaneously incorporate multi-source content data of items and multi-networks of users to learn user and item representations for recommendation. This information can be complementary to each other in a way that multi-modality data can bridge the semantic gap among items, while multi-type user networks, such as following and reposting, are able to propagate the preferences among users. Furthermore, to discover the hidden categories of micro-videos that properly match users’ interests, we interactively learn the user–item representations and perform the hidden item category clustering. The resulted categorical representations are interacted with user representations to model user preferences at different levels of hierarchies. Finally, multi-source content item data, multi-type user networks and hidden item categories are jointly modelled in a unified recommender, and the parameters of the model are collaboratively learned to boost the recommendation performance. Experiments on a real dataset demonstrate the effectiveness of the proposed model and its advantage over the state-of-the-art baselines.

In this paper, we propose solutions to advance answer selection in Community Question Answering (CQA). Unlike previous works, we propose a hybrid attention mechanism to model question-answer pairs. Specifically, for each word, we calculate the intra-sentence attention indicating its local importance and the inter-sentence attention implying its importance to the counterpart sentence. The inter-sentence attention is based on the interactions between question-answer pairs, and the combination of these two attention mechanisms enables us to align the most informative parts in question-answer pairs for sentence matching. Additionally, we exploit user information for answer selection due to the fact that users are more likely to provide correct answers in their areas of expertise. We model users from their written answers to alleviate data sparsity problem, and then learn user representations according to the informative parts in sentences that are useful for question-answer matching task. This mean of modelling users can bridge the semantic gap between different users, as similar users may have the same way of wording their answers. The representations of users, questions and answers are learnt in an end-to-end neural network in a mean that best explains the interrelation between question-answer pairs. We validate the proposed model on a public dataset, and demonstrate its advantages over the baselines with thorough experiments.

Numerous methods have been proposed to address different aspects of human activity recognition. However, most of the previous approaches are static in terms of the data sources used for the recognition task. As sensors can be added or can fail and be replaced by different types of sensors, creating an activity recognition model that is able to leverage dynamically available sensors becomes important. In this paper, we propose methods for activity learning and activity recognition adaptation in environments with dynamic sensor deployments. Specifically, we propose sensor and activity context models to address the problem of sensor heterogeneity, so that sensor readings can be pre-processed and populated into the recognition system properly. Based on those context models, we propose the learning-to-rank method for activity learning and its adaptation. To model the temporal characteristics of the human behaviours, we add temporal regularization into the learning and prediction phases. We use comprehensive datasets to demonstrate effectiveness of the proposed method, and show its advantage over the conventional machine learning algorithms in terms of recognition accuracy. Our method outperforms hybrid models that combine typical machine learning methods with graphical models (i.e. HMM, CRF) for temporal smoothing.

The rapid adoption of mobile phones and growing number of mobile services (like Whatsapp) have created high volume of mobile data traffic. Compared to the traditional voice and SMS services, mobile data is becoming more and more important. Bandwidth is in high demand and is becoming critical for user experiences with mobile contents. Due to a variety of factors (such as admission control policies, signal fading), bandwidth available to a device cannot be directly mapped to the signal strength. This creates a need for bandwidth estimation on mobile phones, similar to the signal strength indicator, to show the estimated bandwidth at the instant. Due to the complexity of bandwidth measurement and/or limitations of earlier generations of mobile phones, there is a small amount of research on bandwidth estimation for mobile phones. This paper revisits a variety of bandwidth estimation methods for wireless and mobile networks, analyses why most of the solutions fail, studies the accuracy limitation of bandwidth estimation in mobile networks, and finds a feasible solution for dynamically estimating bandwidth on mobile phones. We propose GPing-Pair, a low-cost bandwidth approximation for cellular connections that estimates bandwidth on the mobile device without server support. We implemented the proposed method as an application for Android phones.

The number of mobile users of on demand video is growing rapidly. However, bandwidth fluctuation in the 3G/LTE technologies is an obstacle in providing high quality smooth video playout for users on the go. In this paper, we present ColStream that can aggregate bandwidth from ubiquitous devices to ensure high quality video streaming with minimal stalling time. ColStream dynamically adjusts the set of collaborators and the size of video chunks that the collaborators need to pre-fetch ahead of the video chunks playout time to provide smooth video playout. ColStream uses a multi-objective optimisation method to maximise bandwidth and minimise cost. ColStream requires neither external servers nor proxies to provide its functionality. The paper describes the ColStream functionality, architecture, applied algorithms, ColStream prototype, and evaluation of its suitability for effective video streaming in mobile environments.