The task of text-video retrieval aims to understand the correspondence between language and vision and has gained increasing attention in recent years. Recent works have demonstrated the superiority of local spatio-temporal relation learning with graph-based models. However, most existing graph-based models are handcrafted and depend heavily on expert knowledge and empirical feedback, which may be unable to mine the high-level fine-grained visual relations effectively. These limitations result in their inability to distinguish videos with the same visual components but different relations. To solve this problem, we propose a novel cross-modal retrieval framework, Bi-Branch Complementary Network (BiC-Net), which modifies Transformer architecture to effectively bridge text-video modalities in a complementary manner via combining local spatio-temporal relation and global temporal information. Specifically, local video representations are encoded using multiple Transformer blocks and additional residual blocks to learn fine-grained spatio-temporal relations and long-term temporal dependency, calling the module a Fine-grained Spatio-temporal Transformer (FST). Global video representations are encoded using a multi-layer Transformer block to learn global temporal features. Finally, we align the spatio-temporal relation and global temporal features with the text feature on two embedding spaces for cross-modal text-video retrieval. Extensive experiments are conducted on MSR-VTT, MSVD, and YouCook2 datasets. The results demonstrate the effectiveness of our proposed model. Our code is public at https://github.com/lionel-hing/BiC-Net .

Open set domain adaptation aims to transfer knowledge in the presence of unknown samples in the target domain. Previous approaches use additional classifiers or threshold-based methods to identify unknown samples and try to investigate the information of class diversity within the unknown samples. Despite achieving excellent adaptation results, these methods ignore those samples that lie on the cluster boundaries, especially the clustering-based methods. In this paper, we propose a novel Neighbor Prototype Contrastive Clustering (NPC2) method, which uses the Local Semantic Structure (LSS) to help these low-confidence samples located on the boundary of clusters to return to their own clusters. Further, we propose Local Semantic Consistency (LSC) to evaluate the clustering result and apply it to the domain adaptation process as a metric to assess the reliability of the samples. Results on four benchmarks show that our NPC2significantly outperforms most state-of-the-art methods with higher LSC.

Unsupervised Domain Adaptation(UDA) aims to learn domain-invariant characteristics of source and target domains so that classifiers learned from the source domain can be applied to the unlabeled target domain. Now some methods trying to build an intermediate domain to alleviate the domains discrepancy. However, since the target samples are unlabeled, it will cause great confusion to the model if the proportion of source and target domain samples in the intermediate domain samples nearly equals. Moreover, the stylistic bias of the CNNs is often ignored. Therefore, we propose to gradually reduce the distribution discrepancy between domains by constructing continuous multiple intermediate domains. To address the style discrepancy between domains, we propose Self-Exchange and Cross-Exchange modules to reduce the style discrepancy between domains. In addition, we propose to focus on sample probability divergence to select reliable samples. Experiments show the proposed method has achieved significant performance improvement in several cross-domain benchmark tests.

Recently, nonshared-and-imbalanced unsupervised domain adaption has been proposed to fix domain shift from Big Data source domain with long-tail distribution to specific small target domain with imbalanced distribution, including two challenges: 1) nonshared classes sharing in big data with long-tail distribution; and 2) imbalanced domain adaptation. Prior approaches explore knowledge sharing between classes to improve performance of unsupervised domain adaption methods. However these methods have inductive bias for prior tree or graph. And previous contrastive domain adaptation methods take center-based prototypes as positive samples which only coarsely characterize the domain structure, and fail to depict the local data structure. To fix these problems, we propose a novel framework called contrastive memory feature transfer (CMFT). To solve nonshared data sharing without inductive bias, we build a centroid memory baseddirected memory transfermechanism to enhance imbalanced class features with similar nonshared class centroid. To address the imbalanced domain adaptation, we design a fault-tolerant and fine-grainedneighborhood prototypefor the contrastive learning which can narrow the domain shift. The proposed CMFT outperforms previous methods on most benchmarks.

Traffic flow parameters are an important data support for the research and development of several technologies in the intelligent transportation system. Therefore, accurate and real-time estimation of traffic flow is particularly important for urban traffic. In this study, a real-time traffic flow detection system framework was constructed based on video image collection and analysis. According to the vehicle detection and tracking results, a traffic flow parameter estimation model and an improved LSTM network are proposed for spatiotemporal counting feature recognition. The results conclude that the developed framework can estimate the traffic flow density and count vehicles, as well as estimate the traffic flow velocity and traffic volume to estimate and optimize traffic flow, respectively. Additionally, the simulation results show that the proposed method can not only counts the two-way traffic vehicles quickly and accurately, but also avoids the use of the complex multi-target tracking method to spatiotemporal correlation of a single target, increases the speed and accuracy of the spatiotemporal information processing procedure, and has stronger scene adaptability.

Despite the recent progress of cross-modal text-to-video retrieval techniques, their performance is still unsatisfactory. Most existing works follow a trend of learning a joint embedding space to measure the distance between global-level or local-level textual and video representation. The fine-grained interactions between video segments and phrases are usually neglected in cross-modal learning, which results in suboptimal retrieval performances. To tackle the problem, we propose a novel Fine-grained Cross-modal Alignment Network (FCA-Net), which considers the interactions between visual semantic units (i.e., sub-actions/sub-events) in videos and phrases in sentences for cross-modal alignment. Specifically, the interactions between visual semantic units and phrases are formulated as a link prediction problem optimized by a graph auto-encoder to obtain the explicit relations between them and enhance the aligned feature representation for fine-grained cross-modal alignment. Experimental results on MSR-VTT, YouCook2, and VATEX datasets demonstrate the superiority of our model as compared to the state-of-the-art method.

With the proliferation of social networks, group activities have become an essential ingredient of our daily life. A growing number of users share their group activities online and invite their friends to join in. This imposes the need of an in-depth study on the group recommendation task, i.e., recommending items to a group of users. Despite its value and significance, group recommendation remains an unsolved problem due to 1) the weights of group members are crucial to the recommendation performance but are rarely learnt from data; 2) social followee information is beneficial to understand users' preferences but is rarely considered; and 3) user-item interactions are helpful to reinforce the performance of group recommendation but are seldom investigated. Toward this end, we devise neural network-based solutions by utilizing the recent developments of attention network and neural collaborative filtering (NCF). First of all, we adopt an attention network to form the representation of a group by aggregating the group members' embeddings, which allows the attention weights of group members to be dynamically learnt from data. Second, the social followee information is incorporated via another attention network to enhance the representation of individual user, which is helpful to capture users' personal preferences. Third, considering that many online group systems also have abundant interactions of individual users on items, we further integrate the modeling of user-item interactions into our method. Through this way, the recommendation for groups and users can be mutually reinforced. Extensive experiments on the scope of both macro-level performance comparison and micro-level analyses justify the effectiveness and rationality of our proposed approaches.

In this article, we present a novel solution to automating the procurement of food materials by using electronic scales, which can automatically identify the food materials along weighing them. Although the CNN model is regarded as one of the most effective solutions to image recognition, the traditional techniques cannot handle the mismatch problem between the lab training data and the real world data. To solve the problem, we propose to embed a partial-and-imbalanced domain adaptation technique (tree adaptation network) in the deep learning model, which can borrow knowledge from sibling classes, to overcome the imbalance problem, and transfer knowledge from the source domain to the target domain, to fight the mismatch problem between the lab training data and the real world data. Experiments show that the proposed approach outperforms state-of-the-art algorithms. Furthermore, the proposed techniques have already been used in practice.

Recently, deep learning models have emerged as powerful tools for relation extraction.However, little work has been done on relation extraction for the Chinese language.One major challenge for relation extraction in Chinese texts is that Chinese sentences have no obvious word segmentation.This ambiguity increases the possibility of word segmentation errors.Another challenge is the lack of broad-scale Chinese text datasets.In this paper, we propose an attention-based multi-instance multi-label bidirectional long short-term memory network for distantly supervised Chinese relation extraction.Our model takes Chinese character embeddings and position embeddings as input without Chinese word segmentation errors.Then, the attention mechanism is used to extract richer Chinese character and sentence features.Finally, we handle the multi-label nature of relation extraction by using multi-label loss functions in the neural network classifier.Based on the idea of distant supervision, we constructed a new dataset for relation extraction in Chinese texts.Experiments on this dataset show that our method has achieved relatively high performance, and that the proposed network architecture is suitable for Chinese relation extraction.Furthermore, we also ran experiments on a popular English benchmark dataset, and the results show that our method is superior to some existing methods. Index Terms-Chinese relation extraction

Heterogeneous device services generated by various devices in different contexts prevent users from efficiently and correctly consuming device services. This seriously hinders the development of Internet of Things. This paper addresses the problems appearing in device discovery and device interaction. It devises a user interoperability framework (UIF) to enable device users to interoperate with heterogeneous devices of different contexts with consistent syntax and semantics. In this framework, a new separation strategy is provided; a device representation method for real, common, and virtual devices is devised; and a device transformability model is proposed to guarantee the proper transformation of device syntax and semantics. To demonstrate the correctness of UIF, a UIF prototype is implemented and several experiment methods are compared to determine which one should be adopted as semantic relatedness computing tools in device discovery for device users and in common device publishing for device providers.

For the multilingual semantic interoperations in cross-organizational enterprise systems and e-commerce systems, semantic consistency is a research issue that has not been well resolved. This paper contributes to improving multilingual semantic interoperation by proposing a concept-connected near synonym (NSG) framework for concept disambiguation. NSG framework provides a vocabulary preprocessing process of collaborative vocabulary editing, which further ensures semantically consistent vocabulary for building semantically consistent business processes and documents between context-different information systems. The vocabulary preprocessing offered by NSG automates the process of finding potential near synonym sets and identifying collaboratively editable near synonym sets. The realization of NSG framework includes a probability model that computes concept values between concepts based on a newly introduced semantic relatedness method-SRCT. In this paper, SRCT-based methods are implemented and compared with some existing semantic relatedness methods. Experiments have shown that SRCT-based methods outperform the existing methods. This paper has made an improvement on the existing methods of semantic relatedness and reduces the collaboration cost of collaborative vocabulary editing.