Recently, substantial advancements in pre-trained visionlanguage models have greatly enhanced the capabilities of multi-modal dialog systems. These models have demonstrated significant improvements by fine-tuning on downstream tasks. However, the existing pre-trained models primarily focus on effectively capturing the alignment between vision and language modalities, often ignoring the intricate nature of dialog context. In this paper, we propose a parameter-efficient prompt-tuning method named DialCLIP for multi-modal dialog retrieval. Specifically, our approach introduces a multi-modal context prompt generator to learn context features which are subsequently distilled into prompts within the pre-trained vision-language model CLIP. Besides, we introduce domain prompt to mitigate the disc repancy from the downstream dialog data. To facilitate various types of retrieval, we also design multiple experts to learn mappings from CLIP outputs to multi-modal representation space, with each expert being responsible to one specific retrieval type. Extensive experiments show that DialCLIP achieves stateof-the-art performance on two widely recognized benchmark datasets (i.e., PhotoChat and MMDialog) by tuning a mere 0.04% of the total parameters. These results highlight the efficacy and efficiency of our proposed approach, underscoring its potential to advance the field of multi-modal dialog retrieval.

User reviews on online consumption platforms are crucial for both consumers and merchants, serving as a reference for purchase decisions and product improvement. However, fake reviews can mislead consumers and harm merchant profits and reputation. Developing effective methods for detecting deceptive reviews is crucial to protecting the interests of both parties. In recent years, research on fake review detection has focused on improving machine learning and neural network methods to enhance the accuracy of fake review detection, neglecting the fundamental and necessary work of text feature representation for reviews. High-quality review text feature representation affects or even determines the quality and performance of fake review detection methods. The increasing prevalence of fake reviews results in a more complex distribution within the feature space of review texts, thus necessitating review embedding methods that exhibit comprehensive semantic comprehension and contextual awareness of review texts. To improve the quality of textual feature representation, we propose a review-embedding attention-based long short-term memory (A-LSTM) method that can encode the global semantics of reviews and detect the deception of the review content. A-LSTM uses attention gates to discover the importance of words, and by analyzing the importance of words, it can help distinguish the characteristics of real and fake reviews, and we propose an attention loss function to solve the problem of class imbalance. On the Yelp dataset, the accuracy of deceptive review detection has increased to 90.9%.

Perceiving multi-modal information and fulfilling dialogues with humans is a long-term goal of artificial intelligence.Pre-training is commonly regarded as an effective approach for multi-modal dialogue.However, due to the limited availability of multi-modal dialogue data, there is still scarce research on multi-modal dialogue pre-training.Yet another intriguing challenge emerges from the encompassing nature of multi-modal dialogue, which involves various modalities and tasks.Moreover, new forms of tasks may arise at unpredictable points in the future.Hence, it is essential for designed multi-modal dialogue models to possess sufficient flexibility to adapt to such scenarios.This paper proposes PaCE, a unified, structured, compositional multi-modal dialogue pretraining framework.It utilizes a combination of several fundamental experts to accommodate multiple dialogue-related tasks and can be pre-trained using limited dialogue and extensive non-dialogue multi-modal data.Furthermore, we propose a progressive training method where old experts from the past can assist new experts, facilitating the expansion of their capabilities.Experimental results demonstrate that PaCE achieves state-of-the-art results on eight multi-modal dialog benchmarks.

Many practical applications, such as social media and monitoring system, will constantly generate streaming data, which has problems of instability, lack of labels and multiclass imbalance. In order to solve these problems, a cluster-based active learning method is proposed to achieve data stream classification. Firstly, a label query strategy combining marginal threshold matrix is proposed, which selects difficult to classify or potential concept drift samples for marking, to solve the problem of high cost label and unbalanced data. Secondly, dynamic maintenance of a group of micro clusters, by adjusting the weight of micro clusters in the model, correctly reflects the current data distribution, and finally, uses the buffer to store new micro clusters to participate in the update of the model, to adapt to the new data environment. Experimental results on three real data sets and three synthetic data sets show that compared with the classical data stream classification algorithm, it is less affected by concept drift and has higher classification accuracy than the online semi-supervised learning algorithm ADSM. The average accuracy of the six datasets increased by 5.56%, 2.32%, 1.77%, 1.83%, 3.78%, and 2.04%, respectively. The model processes data streams online and improves classification performance with less memory consumption.

People’s increasingly frequent online activity has generated a large number of reviews, whereas fake reviews can mislead users and harm their personal interests. In addition, it is not feasible to label reviews on a large scale because of the high cost of manual labeling. Therefore, to improve the detection performance by utilizing the unlabeled reviews, this article proposes a fake reviews detection model based on vertical ensemble tri-training and active learning (VETT-AL). The model combines the features of review text with the user behavior features as feature extraction. In the VETT-AL algorithm, the iterative process is divided into two parts: vertical integration within the group and horizontal integration among the groups. The intra-group integration is to integrate three original classifiers by using the previous iterative models of the classifiers. The inter-group integration is to adopt the active learning based on entropy to select the data with the highest confidence and label it, and as the result of that, the second generation classifiers are trained by the traditional process to improve the accuracy of the label. Experimental results show that the proposed model has a good classification performance.

Explicit representations of the global match distributions of pixel-wise correspondences between pairs of images are desirable for uncertainty estimation and downstream applications. However, the computation of the match density for each pixel may be prohibitively expensive due to the large number of candidates. In this paper, we propose Hierarchical Discrete Distribution Decomposition (HD^3), a framework suitable for learning probabilistic pixel correspondences in both optical flow and stereo matching. We decompose the full match density into multiple scales hierarchically, and estimate the local matching distributions at each scale conditioned on the matching and warping at coarser scales. The local distributions can then be composed together to form the global match density. Despite its simplicity, our probabilistic method achieves state-of-the-art results for both optical flow and stereo matching on established benchmarks. We also find the estimated uncertainty is a good indication of the reliability of the predicted correspondences.

Environment perception is an important task with great practical value and bird view is an essential part for creating panoramas of surrounding environment. Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging. To tackle this problem, we propose the BridgeGAN, i.e., a novel generative model for bird view synthesis. First, an intermediate view, i.e., homography view, is introduced to bridge the large gap. Next, conditioned on the three views (frontal view, homography view and bird view) in our task, a multi-GAN based model is proposed to learn the challenging cross-view translation. Furthermore, to guarantee one-to-one cross-view correspondences and consistent cross-view feature representations, two consistency constraints are designed for our task. Extensive experiments conducted on a synthetic dataset have demonstrated that the images generated by our model are much better than those generated by existing methods, with more consistent global appearance and sharper details. Ablation studies and discussions show its reliability and robustness in some challenging cases.

We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and egomotion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in an end-to-end manner. Specifically, geometric relationships are extracted over the predictions of individual modules and then combined as an image reconstruction loss, reasoning about static and dynamic scene parts separately. Furthermore, we propose an adaptive geometric consistency loss to increase robustness towards outliers and non-Lambertian regions, which resolves occlusions and texture ambiguities effectively. Experimentation on the KITTI driving dataset reveals that our scheme achieves state-of-the-art results in all of the three tasks, performing better than previously unsupervised methods and comparably with supervised ones.

Software inspection is well-known as an effective means of defect detection. Nevertheless, recent research has suggested that the technique requires further development to optimize the inspection process. As the process is inherently group-based, one approach to improving performance is to attempt to minimize the commonality within the process and the group. This work proposes an approach to add diversity into the process by using a cognitively-based team selection mechanism. The paper argues that a team with diverse information processing strategies, as defined by the selection mechanism, maximize the number of different defects discovered.