Gastrointestinal (GI) endoscopy has been an active field of research motivated by the large number of highly lethal GI cancers. Early GI cancer precursors are often missed during the endoscopic surveillance. The high missed rate of such abnormalities during endoscopy is thus a critical bottleneck. Lack of attentiveness due to tiring procedures, and requirement of training are few contributing factors. An automatic GI disease classification system can help reduce such risks by flagging suspicious frames and lesions. GI endoscopy consists of several multi-organ surveillance, therefore, there is need to develop methods that can generalize to various endoscopic findings. In this realm, we present a comprehensive analysis of the Medico GI challenges: Medical Multimedia Task at MediaEval 2017, Medico Multimedia Task at MediaEval 2018, and BioMedia ACM MM Grand Challenge 2019. These challenges are initiative to set-up a benchmark for different computer vision methods applied to the multi-class endoscopic images and promote to build new approaches that could reliably be used in clinics. We report the performance of 21 participating teams over a period of three consecutive years and provide a detailed analysis of the methods used by the participants, highlighting the challenges and shortcomings of the current approaches and dissect their credibility for the use in clinical settings. Our analysis revealed that the participants achieved an improvement on maximum Mathew correlation coefficient (MCC) from 82.68% in 2017 to 93.98% in 2018 and 95.20% in 2019 challenges, and a significant increase in computational speed over consecutive years.

Graph clustering, which aims at discovering sets of related vertices in graph-structured data, plays a crucial role in various applications, such as social community detection and biological module discovery. With the huge increase in the volume of data in recent years, graph clustering is used in an increasing number of real-life scenarios. However, the classical and state-of-the-art methods, which consider only single-view features or a single vector concatenating features from different views and neglect the contextual correlation between pairwise features, are insufficient for the task, as features that characterize vertices in a graph are usually from multiple views and the contextual correlation between pairwise features may influence the cluster preference for vertices. To address this challenging problem, we introduce in this paper, a novel graph clustering model, dubbed contextual correlation preserving multiview featured graph clustering (CCPMVFGC) for discovering clusters in graphs with multiview vertex features. Unlike most of the aforementioned approaches, CCPMVFGC is capable of learning a shared latent space from multiview features as the cluster preference for each vertex and making use of this latent space to model the inter-relationship between pairwise vertices. CCPMVFGC uses an effective method to compute the degree of contextual correlation between pairwise vertex features and utilizes view-wise latent space representing the feature-cluster preference to model the computed correlation. Thus, the cluster preference learned by CCPMVFGC is jointly inferred by multiview features, view-wise correlations of pairwise features, and the graph topology. Accordingly, we propose a unified objective function for CCPMVFGC and develop an iterative strategy to solve the formulated optimization problem. We also provide the theoretical analysis of the proposed model, including convergence proof and computational complexity analysis. In our experiments, we extensively compare the proposed CCPMVFGC with both classical and state-of-the-art graph clustering methods on eight standard graph datasets (six multiview and two single-view datasets). The results show that CCPMVFGC achieves competitive performance on all eight datasets, which validates the effectiveness of the proposed model.

The digital age has empowered brands with new and more effective targeted marketing tools in the form of key opinion leaders (KOLs). Because of the KOLs' unique capability to draw specific types of audience and cultivate long-term relationship with them, correctly identifying the most suitable KOLs within a social network is of great importance, and sometimes could govern the success or failure of a brand's online marketing campaigns. However, given the high dimensionality of social media data, conducting effective KOL identification by means of data mining is especially challenging. Owing to the generally multiple modalities of the user profiles and user-generated content (UGC) over the social networks, we can approach the KOL identification process as a multimodal learning task, with KOLs as a rare yet far more important class over non-KOLs in our consideration. In this regard, learning the compact and informative representation from the high-dimensional multimodal space is crucial in KOL identification. To address this challenging problem, in this paper, we propose a novel subspace learning algorithm dubbed modality-consistent harmonized discriminant embedding (MCHDE) to uncover the low-dimensional discriminative representation from the social media data for identifying KOLs. Specifically, MCHDE aims to find a common subspace for multiple modalities, in which the local geometric structure, the harmonized discriminant information, and the modality consistency of the dataset could be preserved simultaneously. The above objective is then formulated as a generalized eigendecomposition problem and the closed-form solution is obtained. Experiments on both synthetic example and a real-world KOL dataset validate the effectiveness of the proposed method.

Integrating media elements of various medium, multimedia is capable of expressing complex information in a neat and compact way. Early studies have linked different sensory presentation in multimedia with the perception of human-like concepts. Yet, the richness of information in multimedia makes understanding and predicting user perceptions in multimedia content a challenging task both to the machine and the human mind. This paper presents a novel multi-task feature extraction method for accurate prediction of user perceptions in multimedia content. Differentiating from the conventional feature extraction algorithms which focus on perfecting a single task, the proposed model recognizes the commonality between different perceptions (e.g., interestingness and emotional impact), and attempts to jointly optimize the performance of all the tasks through uncovered commonality features. Using both a media interestingness dataset and a media emotion dataset for user perception prediction tasks, the proposed model attempts to simultaneously characterize the individualities of each task and capture the commonalities shared by both tasks, and achieves better accuracy in predictions than other competing algorithms on real-world datasets of two related tasks: MediaEval 2017 Predicting Media Interestingness Task and MediaEval 2017 Emotional Impact of Movies Task.

Following the rising prominence of online social networks, we observe an emerging trend for brands to adopt influencer marketing, embracing key opinion leaders (KOLs) to reach potential customers (PCs) online. Owing to the growing strategic importance of these brand key assets, this paper presents a novel feature extraction method named Multi-modal Asset-aware Projection (M2A2P) to learn a discriminative subspace from the high-dimensional multi-modal social media data for effective brand key asset discovery. By formulating a new asset-aware discriminative information preserving criterion, M2A2P differentiates with the existing multi-model feature extraction algorithms in two pivotal aspects: 1) We consider brand's highly imbalanced class interest steering towards the KOLs and PCs over the irrelevant users; 2) We consider a common observation that a user is not exclusive to a single class (e.g. a KOL can also be a PC). Experiments on a real-world apparel brand key asset dataset validate the effectiveness of the proposed method.

As social media grown to become an integral part of many people's daily life, brands are quick to launch targeted social media marketing campaign to acquire new potential customers online. To facilitate the potential customer discovery process, a costly and labor intensive manual selection process is done to build a brand portfolio consisting of multimedia data relevant to the brand. To automate this process in a cost-effective way, in this paper, we propose a novel Multi-Modal Distance Metric Learning (M2DML) method, which learns a data-dependent similarity metric from multi-modal media data, aiming at assisting the brands to retrieve appropriate media data from social networks for potential customer discovery. To comprehensively model the supervised information of multi-modal data, M2DML aims to learn both the intra-modality and inter-modality distance metrics simultaneously. To further explore the unsupervised information of the dataset, M2DML aims to preserve the manifold structure of the multi-modal data. The proposed method is then formulated as a standard eigen-decomposition problem and the closed form solution is efficiently computed. Experiments on a standard multi-modal media dataset and a self-collected dataset validate the effectiveness of the proposed method.

Accurate and effective discovery of a brand's key assets, namely, Key Opinion Leaders (KOLs) and potential customers, plays an essential role in marketing campaigns. In a massive online social network, brands are challenged with identifying a small portion of key assets over an enormous volume of irrelevant users, making the problem a highly imbalanced one. Moreover, having to deal with social media data that are usually high-dimensional, the task of brand key asset discovery can be immensely expensive yet inaccurate if the information are not processed efficiently to extract representative features from the original space prior to the learning process. To address the above issues, we propose a novel method dubbed Cluster-wise Biased Discriminant Projection (CBDP) to uncover the compact and informative features from users' data for brand key asset discovery. CBDP conducts a two-layer learning procedure. In the first layer, a Discriminant Clustering (DC) scheme is developed to partition the original dataset into clusters with maximum discriminant capacity. In the second layer, a Biased Discriminant Projection (BDP) algorithm is proposed and performed on each cluster to map the high-dimensional data to the low-dimensional subspace, where the discriminant information of classes with high importance/preference is preserved. A unified mapping function of CBDP is finally established by integrating these two layers. Experiments on both synthetic examples and a real-world brand key asset dataset validate the effectiveness of the proposed method.