Leveraging deep learning to explore the associations between voices and faces has attracted extensive research interest. Usually, this research is formalized by cross-modal verification, matching, and retrieval tasks. Most works rely on a single local or global optimization objective for learning, ignoring that testing scenarios may prefer different optimization objectives. For example, local objectives are more helpful for verification and matching tasks, while global objectives contribute more to retrieval. In this study, we proposed a learning framework based on local and global objectives to improve the generalizability of the learned representations. Firstly, we explored two ways of applying supervised contrastive loss (SCL) to learn voice-face representations. Secondly, we designed a contrastive-form global optimization objective, which shows better performance and training efficiency. Experiments on the VoxCeleb dataset demonstrate the effectiveness of our framework.

Learning associations between voices and faces have recently attracted much research interest. Previous studies mainly focused on improving loss function design for learning better representations, neglecting that informative inputs are the prerequisite of association findings and effective representation alignments. Based on this motivation, we proposed an unsupervised transformer-based learning framework, which fuses the knowledge of readily available single-modal expert models to learn better representations. Benefiting from higher-quality inputs, a simple noise contrastive estimation (NCE) loss is used for training. In addition, we proposed a statistical batch construction (SBC) strategy, which obtains high-quality negative samples during the unsupervised learning process. Experiments on the VoxCeleb1 dataset demonstrate the effectiveness of our framework. It yields SOTA results in voice-face verification, matching, and retrieval tasks. In the challenging gender-constrained matching tasks, it achieves over 81% (ACC), which is 5% higher than the previous.

Voice-face association learning (VFAL) aims to tap into the potential connections between voices and faces. Most studies currently address this problem in a supervised manner, which cannot exploit the wealth of unlabeled video data. To solve this problem, we propose an unsupervised learning framework: Self-Lifting (SL), which can use unlabeled video data for learning. This framework includes two iterative steps of "clustering" and "metric learning". In the first step, unlabeled video data is mapped into the feature space by a coarse model. Then unsupervised clustering is leveraged to allocate pseudo-label to each video. In the second step, the pseudo-label is used as supervisory information to guide the metric learning process, which produces the refined model. These two steps are performed alternately to lift the model's performance. Experiments show that our framework can effectively use unlabeled video data for learning. On the VoxCeleb dataset, our approach achieves SOTA results among the unsupervised methods and has competitive performance compared with the supervised competitors. Our code is released on Github.

Word embedding has been well accepted as an important feature in the area of natural language processing (NLP). Specifically, the Word2Vec model learns high-quality word embeddings and is widely used in various NLP tasks. The training of Word2Vec is sequential on a CPU due to strong dependencies between word–context pairs. In this paper, we target to scale Word2Vec on a GPU cluster. To do this, one main challenge is reducing dependencies inside a large training batch. We heuristically design a variation of Word2Vec, which ensures that each word–context pair contains a non-dependent word and a uniformly sampled contextual word. During batch training, we “freeze” the context part and update only on the non-dependent part to reduce conflicts. This variation also directly controls the training iterations by fixing the number of samples and treats high-frequency and low-frequency words equally. We conduct extensive experiments over a range of NLP tasks. The results show that our proposed model achieves a 7.5 times acceleration on 16 GPUs without accuracy drop. Moreover, by using high-level Chainer deep learning framework, we can easily implement Word2Vec variations such as CNN-based subword-level models and achieves similar scaling results.

Smart contracts (SCs) run on the distributed ledger technology (DLT) platform and can implement agreements between participants without a trusted third party. This paper uses the DLT and SC techniques to build distributed applications composed of existing services. In practice, there are many functionally-equivalent services on the Internet. To beat their competitors, the service providers usually offer flexible QoS and use dynamic pricing strategies. Moreover, the service providers can change at runtime, e.g., they may encounter problems so that their QoS drops suddenly. This makes achieving the optimization goal at runtime (e.g., the maximization of the utility) more difficult. To address this problem, first, this paper proposes an SC-based negotiation framework. The SCs can ensure that the transactions are automatically and reliably performed as agreed upon between the service requesters and providers. The DLTs can provide the reliable data of the requests and responses of the service requesters and providers to the SCs. In addition, the SCs can identify the troubled service providers, and find other service providers to replace them at runtime. Second, this paper proposes a Bayesian Nash equilibrium (BNE) of the service providers. In the BNE, the cost-efficient service providers offer the high QoS the service requester asks for and report their costs truthfully. This BNE enables the selection of the cost-efficient service providers and the achievement of the (near) maximization of the service requesters' utility. This paper implements the proposed negotiation framework on a DLT platform called Hyperledger Fabric. The experiment results demonstrate that the proposed approach outperforms the existing approaches and can adapt to the changes of the service providers.

Bag-of-ngrams (BoN) models are commonly used for representing text. One of the main drawbacks of traditional BoN is the ignorance of n-gram's semantics. In this paper, we introduce the concept of Neural Bag-of-ngrams (Neural-BoN), which replaces sparse one-hot n-gram representation in traditional BoN with dense and rich-semantic n-gram representations. We first propose context guided n-gram representation by adding n-grams to word embeddings model. However, the context guided learning strategy of word embeddings is likely to miss some semantics for text-level tasks. Text guided n-gram representation and label guided n-gram representation are proposed to capture more semantics like topic or sentiment tendencies. Neural-BoN with the latter two n-gram representations achieve state-of-the-art results on 4 document-level classification datasets and 6 semantic relatedness categories. They are also on par with some sophisticated DNNs on 3 sentence-level classification datasets. Similar to traditional BoN, Neural-BoN is efficient, robust and easy to implement. We expect it to be a strong baseline and be used in more real-world applications.

The number of word embedding models is growing every year.Most of them are based on the co-occurrence information of words and their contexts.However, it is still an open question what is the best definition of context.We provide a systematical investigation of 4 different syntactic context types and context representations for learning word embeddings.Comprehensive experiments are conducted to evaluate their effectiveness on 6 extrinsic and intrinsic tasks.We hope that this paper, along with the published code, would be helpful for choosing the best context type and representation for a given task.

Convolutional Neural Networks (CNNs) are widely used in NLP tasks.This paper presents a novel weight initialization method to improve the CNNs for text classification.Instead of randomly initializing the convolutional filters, we encode semantic features into them, which helps the model focus on learning useful features at the beginning of the training.Experiments demonstrate the effectiveness of the initialization technique on seven text classification tasks, including sentiment analysis and topic classification.

The existing word representation methods mostly limit their information source to word co-occurrence statistics.In this paper, we introduce ngrams into four representation methods: SGNS, GloVe, PPMI matrix, and its SVD factorization.Comprehensive experiments are conducted on word analogy and similarity tasks.The results show that improved word representations are learned from ngram cooccurrence statistics.We also demonstrate that the trained ngram representations are useful in many aspects such as finding antonyms and collocations.Besides, a novel approach of building co-occurrence matrix is proposed to alleviate the hardware burdens brought by ngrams.

An important issue in QoS-aware Web service composition is how to select a set of Web services to perform the tasks within a requested service while meeting global QoS constraints. We consider the Web services are self-interested and will use dynamic pricing strategy. In general, the service cost is the minimum price acceptable to a Web service. We can obtain a composite Web service with the maximum utility by assigning the tasks to the Web services with the lowest costs. A Web service usually will not expose his cost, and thus, we face a decision making problem with incomplete information. Recent approaches use iterative combinatorial auction to address the problem. However, truthful bidding is not optimal strategy for Web services in these approaches. In this paper, we propose an incentive mechanism for choosing the optimal Web service for each task and show there exists a Bayesian Nash equilibrium of Web services, in which each Web service will bid truthfully. Finally, the experimental results show that our mechanism outperforms the existing combinatorial auction-based approaches.

The service-oriented paradigm offers support for engineering service-based systems based on service composition. QoS (Quality of Service)-aware service composition chooses a set of services to collectively construct a service-based system, while satisfying global QoS constraints and budget restriction. The service providers naturally are self-interested and strive to maximize their own utilities. Existing approaches use iterative combinatorial auction to address the problem. However, truthful bidding is not optimal strategy for service providers in these approaches. In this paper, we propose a multi-round Vickrey auction to choose an optimal service provider for each task while satisfying our global QoS constraints and budget restriction, and show there may exist a Bayesian Nash equilibrium, in which the service providers will not choose strategically to stay silent and will truthfully bid. Finally, the experimental results show that our approach outperforms the existing combinatorial auction-based approaches.

With the rapid growth of social media, sentiment analysis has received growing attention from both academic and industrial fields. One line of researches for sentiment analysis is to feed bag-of-words (BOW) text representation into classifiers. Usually, raw BOW requires weighting schemes to obtain better performance, where important words are given more weights while unimportant ones are given less weights. Another line of researches focuses on neural models, where distributed text representations are learned from raw texts automatically. In this paper, we take advantages of techniques in both lines of researches. We use words’ weights to guide neural models to focus on important words. Various supervised weighting schemes are explored in this work. We discover that better text features are learned for sentiment analysis when suitable weighting schemes are applied upon neural models.

In the originally published article, the acknowledgment section is missing. Please find it as follows.

NBSVM is one of the most popular methods for text classification and has been widely used as baselines for various text representation approaches. It uses Naive Bayes (NB) feature to weight sparse bag-of-n-grams representation. N-gram captures word order in short context and NB feature assigns more weights to those important words. However, NBSVM suffers from sparsity problem and is reported to be exceeded by newly proposed distributed (dense) text representations learned by neural networks. In this paper, we transfer the n-grams and NB weighting to neural models. We train n-gram embeddings and use NB weighting to guide the neural models to focus on important words. In fact, our methods can be viewed as distributed (dense) counterparts of sparse bag-of-n-grams in NBSVM. We discover that n-grams and NB weighting are also effective in distributed representations. As a result, our models achieve new strong baselines on 9 text classification datasets, e.g. on IMDB dataset, we reach performance of 93.5% accuracy, which exceeds previous state-of-the-art results obtained by deep neural models. All source codes are publicly available at https://github.com/zhezhaoa/neural_BOW_toolkit.

Most of the existing word embedding models only consider the relationships between words and their local contexts (e.g. ten words around the target word). However, information beyond local contexts (global contexts), which reflect the rich semantic meanings of words, are usually ignored. In this paper, we present a general framework for utilizing global information to learn word and text representations. Our models can be easily integrated into existing local word embedding models, and thus introduces global information of varying degrees according to different downstream tasks. Moreover, we view our models in the co-occurrence matrix perspective, based on which a novel weighted term-document matrix is factorized to generate text representations. We conduct a range of experiments to evaluate word and text representations learned by our models. Experimental results show that our models outperform or compete with state-of-the-art models. Source code of the paper is available at https://github.com/zhezhaoa/cluster-driven.

Relation descriptor is the text string which best describes the pre-defined relation between two entities. Relation descriptor can help people to know the specific semantics between two entities, which is very meaningful for knowledge base construction. Traditional relation descriptor extraction method use nominal features, whose expressibility is limited. Word embeddings via deep learning technology can reflect more syntactic and semantic information of words. In this paper we introduce the word embeddings to relation descriptor extraction problem. In order to obtain word semantic classes, we cluster words based on word embeddings and adopt the word cluster feature. Experimental results have shown that word embeddings feature and word cluster feature can improve the performance of relation descriptor extraction obviously. Furthermore the word cluster feature is more robust than word embeddings feature on the relation descriptor extraction. The best method can save 44% and 33% training data to achieve the same performance as the basic method on two datasets.

Partially supervised text classification has received great research attention since it only uses positive and unlabeled examples as training data. This problem can be solved by automatically labeling some negative (and more positive) examples from unlabeled examples before training a text classifier. But it is difficult to guarantee both high quality and quantity of the new labeled examples. In this paper, a multi-level example based learning method for partially supervised text classification is proposed, which can make full use of all unlabeled examples. A heuristic method is proposed to assign possible labels to unlabeled examples and partition them into multiple levels according to their labeling confidence. A text classifier is trained on these multi-level examples using weighted support vector machines. Experiments show that the multi-level example based learning method is effective for partially supervised text classification, and outperforms the existing popular methods such as Biased-SVM, ROC-SVM, S-EM and WL.

Automatic analysis of sentiments expressed in large scale online reviews is very important for intelligent business applications. Sentiment classification is the most popular task of sentiment analysis, which is more challenging than traditional topic-based text classification. Basic features, such as vocabulary words, are not enough to classify sentiments well. Deep Belief Network (DBN) is introduced to discover more abstract features of sentiments. To capture full information of the features, large-size network can be constructed, but at the same time, large-size network tends to over fit the training data and even noise, which will reduce the generalization ability of the network. In this paper, L2-norm Deep Belief Network (L2DBN) is proposed, which uses L2-norm regularization to optimize the network parameters of DBN. L2DBN is first initialized by an unsupervised layer-wise training algorithm, and then fine-tuned by a supervised procedure. Network parameters are optimized using both classification loss and network complexity. Experimental results show that the proposed L2DBN outperforms the state-of-the-art method and the basic DBN on golden, noisy and heterogeneous datasets.

Domain-specific term extraction contributes to all domain-oriented natural language processing tasks. Given a small set of domain-specific terms as seed terms, new terms from unlabeled corpora can be extracted by bootstrapping a term classifier to discover the association between seed terms and new terms. Traditional term representation method for domain-specific term extraction represents a term in a feature space of documents, which depicts association of terms which share common documents. This representation can't depict the inner-document information of terms and requires extracted terms to occur in multiple documents. A new term representation method in global contextual space is proposed for domain-specific term extraction in this paper. This representation mechanism depicts the association of terms which share common global contexts. The information of terms within certain document and among corpora is depicted by global contexts. Experiments on Chinese web corpus show that the proposed domain-specific term extraction method with global contextual representation outperforms traditional method with representation mechanism in documents space. The improvement for low frequency terms is much higher for the proposed method.

A Multi-document Rhetorical Structure (MRS) is proposed for multi-document automatic summarization task. This structure can represent interrelationship between text units at different levels of granularity and can describe simultaneously the happen and change of various events. MRS simplify traditional multi-document representation in cross structure theory and supplement change and distribution information of events topics which cannot be obtained in information fusion theory. Concretely, a series of algorithms including building MRS, multi-document information fusion based MRS and summarization generation are proposed. The capability of concurrently fuse multiple knowledge sources of MRS strategies is testified by sets of experiments and shows good result.

MOTIVATION: We are motivated by the fast-growing number of protein structures in the Protein Data Bank with necessary information for prediction of protein-protein interaction sites to develop methods for identification of residues participating in protein-protein interactions. We would like to compare conditional random fields (CRFs)-based method with conventional classification-based methods that omit the relation between two labels of neighboring residues to show the advantages of CRFs-based method in predicting protein-protein interaction sites. RESULTS: The prediction of protein-protein interaction sites is solved as a sequential labeling problem by applying CRFs with features including protein sequence profile and residue accessible surface area. The CRFs-based method can achieve a comparable performance with state-of-the-art methods, when 1276 nonredundant hetero-complex protein chains are used as training and test set. Experimental result shows that CRFs-based method is a powerful and robust protein-protein interaction site prediction method and can be used to guide biologists to make specific experiments on proteins. AVAILABILITY: http://www.insun.hit.edu.cn/~mhli/site_CRFs/index.html. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.