Every year, the VISion Understanding and Machine intelligence (VISUM) summer school runs a competition where participants can learn and share knowledge about Computer Vision and Machine Learning in a vibrant environment. 2021 VISUM's focused on applying those methodologies in fashion. Recently, there has been an increase of interest within the scientific community in applying computer vision methodologies to the fashion domain. That is highly motivated by fashion being one of the world's largest industries presenting a rapid development in e-commerce mainly since the COVID-19 pandemic. Computer Vision for Fashion enables a wide range of innovations, from personalized recommendations to outfit matching. The competition enabled students to apply the knowledge acquired in the summer school to a real-world problem. The ambition was to foster research and development in fashion outfit complementary product retrieval by leveraging vast visual and textual data with domain knowledge. For this, a new fashion outfit dataset (acquired and curated by FARFETCH) for research and benchmark purposes is introduced. Additionally, a competitive baseline with an original negative sampling process for triplet mining was implemented and served as a starting point for participants. The top 3 performing methods are described in this paper since they constitute the reference state-of-the-art for this particular problem. To our knowledge, this is the first challenge in fashion outfit complementary product retrieval. Moreover, this joint project between academia and industry brings several relevant contributions to disseminating science and technology, promoting economic and social development, and helping to connect early-career researchers to real-world industry challenges.

The second edition of the International Workshop on Multimodal Conversational AI puts forward a diverse set of contributions that aim to brainstorm this new field. Conversational agents are now becoming a commodity as this technology is being applied to a wide range of domains. Healthcare, assisting technologies, e-commerce, information seeking, are some of the domains where multimodal conversational AI is being explored. The wide use of multimodal conversational agents exposes the many challenges in achieving more natural, human-like, and engaging conversational agents. The research contributions of the Workshop actively address several of relevant challenges: How to include assistive-technologies in dialog systems? How can agents engage in negotiation in dialogs? How to handle the embodiment of conversational agents?

Recently, conversational systems have seen a significant rise in demand due to modern commercial applications using systems such as Amazon's Alexa, Apple's Siri, Microsoft's Cortana and Google Assistant. The research on multimodal chatbots is a widely underexplored area, where users and the conversational agent communicate by natural language and visual data. Conversational agents are now becoming a commodity as a number of companies push for this technology. The wide use of these conversational agents exposes the many challenges in achieving more natural, human-like, and engaging conversational agents. The research community is actively addressing several of these challenges: how are visual and text data related in user utterances? How to interpret the user intent? How to encode multimodal dialog status? What are the ethical and legal aspects of conversational AI? The Multimodal Conversational AI workshop will be a forum where researchers and practitioners share their experiences and brainstorm about success and failures in the topic. It will also promote collaboration to strengthen the conversational AI community at ACM Multimedia.

Transfer learning is a process that allows reusing a learning machine trained on a problem to solve a new problem. Transfer learning studies on shallow architectures show low performance as they are generally based on hand-crafted features obtained from experts. It is therefore interesting to study transference on deep architectures, known to directly extract the features from the input data. A Stacked Denoising Autoencoder (SDA) is a deep model able to represent the hierarchical features needed for solving classification problems. In this paper we study the performance of SDAs trained on one problem and reused to solve a different problem not only with different distribution but also with a different tasks. We propose two different approaches: 1) unsupervised feature transference, and 2) supervised feature transference using deep transfer learning. We show that SDAs using the unsupervised feature transference outperform randomly initialized machines on a new problem. We achieved 7% relative improvement on average error rate and 41% on average computation time to classify typed uppercase letters. In the case of supervised feature transference, we achieved 5.7% relative improvement in the average error rate, by reusing the first and second hidden layer, and 8.5% relative improvement for the average error rate and 54% speed up w.r.t the baseline by reusing all three hidden layers for the same data. We also explore transfer learning between geometrical shapes and canonical shapes, we achieved 7.4% relative improvement on average error rate in case of supervised feature transference approach.

In this work we study the impact of a set of bag-of-features strategies for the recognition of cancer in gastroen-terology images. By using the SIFT descriptor, we analyzed the importance and performance impact of term weighting functions for the construction of visual vocabularies. Further analyzes were conducted in order to ascertain the robustness of multiclass decomposition rules for Support Vector Machines with different kernels. Our study was extended by tailoring a decomposition rule that explores prior knowledge according the four grades of the Singh taxonomy (SDR). We found that SDR coupled with a frequency term weight function attained the best overall results (80%) when trained with an intersection kernel. It also outperformed standard decomposition rules when using a χ2kernel and attained competitive performances with a linear kernel.

Ordinal data classification (ODC) has a wide range of applications in areas where human evaluation plays an important role, ranging from psychology and medicine to information retrieval. In ODC the output variable has a natural order; however, there is not a precise notion of the distance between classes. The recently proposed method for ordinal data, Kernel Discriminant Learning Ordinal Regression (KDLOR), is based on Linear Discriminant Analysis (LDA), a simple tool for classification. KDLOR brings LDA to the forefront in the ODC held, motivating further research. This paper compares three LDA based algorithms for ODC. The first method uses the generic framework of Frank and Hall for ODC instantiated with a kernel version of LDA. Similarly, the second method is based on the also generic Data Replication framework for ODC instantiated with the same kernel version of LDA. Both the Frank and Hall and Data Replication methods address the ODC problem by the use of a base binary classifier. Finally, the third method under comparison is KDLOR. The experiments are carried out on synthetic and real datasets. A comparison between the performances of the three systems is made based on tstatistics. The performance and running time complexity of the methods do not support any advantage of KDLOR over the other two methods.

While ordinal classification problems are common in many situations, induction of ordinal decision trees has not evolved significantly. Conventional trees for regression settings or nominal classification are commonly induced for ordinal classification problems. On the other hand a decision tree consistent with the ordinal setting is often desirable to aid decision making in such situations as credit rating. In this work we extend a recently proposed strategy based on constraints defined globally over the feature space. We propose a bootstrap technique to improve the accuracy of the baseline solution. Experiments in synthetic and real data show the benefits of our proposal.

Ordinal classification is a form of multiclass classification for which there is an inherent order between the classes, but not a meaningful numeric difference between them. The performance of such classifiers is usually assessed by measures appropriate for nominal classes or for regression. Unfortunately, these do not account for the true dimension of the error. The goal of this work is to show that existing measures for evaluating ordinal classification models suffer from a number of important shortcomings. For this reason, we propose an alternative measure defined directly in the confusion matrix. An error coefficient appropriate for ordinal data should capture how much the result diverges from the ideal prediction and how "inconsistent" the classifier is in regard to the relative order of the classes. The proposed coefficient results from the observation that the performance yielded by the Misclassification Error Rate coefficient is the benefit of the path along the diagonal of the confusion matrix. We carry out an experimental study which confirms the usefulness of the novel metric.

Ordinal classification is a form of multi-class classification where there is an inherent ordering between the classes, but not a meaningful numeric difference between them. Although conventional methods, designed for nominal classes or regression problems, can be used to solve the ordinal data problem, there are benefits in developing models specific to this kind of data. This paper introduces a new rationale to include the information about the order in the design of a classification model. The method encompasses the inclusion of consistency constraints between adjacent decision regions. A new decision tree and a new nearest neighbour algorithms are then designed under that rationale. An experimental study with artificial and real data sets verifies the usefulness of the proposed approach.

Support vector machines (SVMs) were initially proposed to solve problems with two classes. Despite the myriad of schemes for multiclassification with SVMs proposed since then, little work has been done for the case where the classes are ordered. Usually one constructs a nominal classifier and a posteriori defines the order. The definition of an ordinal classifier leads to a better generalisation. Moreover, most of the techniques presented so far in the literature can generate ambiguous regions. All-at-Once methods have been proposed to solve this issue. In this work we devise a new SVM methodology based on the unimodal paradigm with the All-at-Once scheme for the ordinal classification.

In this work we consider the problem of binary classification where the classifier may abstain instead of classifying each observation, leaving the critical items for human evaluation. This article motivates and presents a novel method to learn the reject region on complex data. Observations are replicated and then a single binary classifier determines the decision plane. The proposed method is an extension of a method available in the literature for the classification of ordinal data. Our method is compared with standard techniques on synthetic and real datasets, emphasizing the advantages of the proposed approach.

Breast cancer conservative treatment (BCCT) is considered the gold standard of breast cancer treatment. However, aesthetic results are heterogeneous and difficult to evaluate in a standardised way. The limited reproducibility of subjective aesthetic evaluation in BCCT forced the research on objective methods. A recent computer system was developed to objectively and automatically evaluate the aesthetic result of BCCT. In this system, the detection of the breast contour on the digital photograph of the patient is necessary to extract the features subsequently used in the evaluation process. In this paper we extend an algorithm based on the shortest path on a graph to detect automatically the breast contour. The advantage of graph algorithms is that they are guaranteed to find the global optimum of the problem; the difficulty is that they make it hard to enforce shape constraints. We define and compare different techniques to introduce the a priory knowledge of the mammary contour. Experimental results show that the proposed techniques consistently outperform the base method.