Social networks have become venues where people can share and spread hate speech, especially when the platforms allow users to remain anonymous. Hate speech can have significant social and cultural effects, especially when it targets specific groups of people in terms of religion, race, ethnicity, culture or a specific social situation such as immigrants and refugees. In this study, we propose a hate speech detection model, BERTurk-DualCL, using a mixed objective with contrastive learning loss that is combined with the traditional cross-entropy loss used for classification. In addition, we study the effects of paralinguistic features, namely emojis and hashtags, on the performance of our model. We trained and evaluated our model on tweets in four different topics with heated discussions from two separate datasets, ranging from discussions about migrants to the Israel-Palestine conflict. Our multi-domain model outperforms comparable results in literature and the average results of four domain-specific models, achieving a macro-F1 score of 81.04% and 58.89% on two- and five-class tasks respectively.

Learning a discriminative semantic space using unlabelled and noisy data remains unaddressed in a multi-label setting. We present a contrastive self-supervised learning method which is robust to data noise, grounded in the domain of variational methods. The method (VCL) utilizes variational contrastive learning with beta-divergence to learn robustly from unlabelled datasets, including uncurated and noisy datasets. We demonstrate the effectiveness of the proposed method through rigorous experiments with multi-label datasets in the face understanding domain, including one where the system is pretrained with web collected face images. Experiments include linear evaluation and fine-tuning scenarios, in addition to verification and face attribute learning tests, showing that the model learns effective embedding representations. In almost all tested scenarios, VCL surpasses the performance of state-of-the-art self-supervised methods.

Abstract Abstractive summarization is an approach to document summarization that is not limited to selecting sentences from the document but can generate new sentences as well. We address the two main challenges in abstractive summarization: how to evaluate the performance of a summarization model and what is a good training objective. We first introduce new evaluation measures based on the semantic similarity of the input and corresponding summary. The similarity scores are obtained by the fine-tuned BERTurk model using either the cross-encoder or a bi-encoder architecture. The fine-tuning is done on the Turkish Natural Language Inference and Semantic Textual Similarity benchmark datasets. We show that these measures have better correlations with human evaluations compared to Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores and BERTScore. We then introduce a deep reinforcement learning algorithm that uses the proposed semantic similarity measures as rewards, together with a mixed training objective, in order to generate more natural summaries in terms of human readability. We show that training with a mixed training objective function compared to only the maximum-likelihood objective improves similarity scores.

We propose a new model for learning to rank two images with respect to their relative strength of expression for a given attribute. We address this problem – called relative attribute learning — using a vision transformer backbone. The embedded representations of the two images to be compared are extracted and used for comparison with a ranking head, in an end-to-end fashion. The results demonstrate the strength of vision transformers and their suitability for relative attributes classification. Our proposed approach outperforms the state-of-the-art by a large margin, achieving 90.40% and 98.14% mean accuracy over the attributes of LFW-10 and Pubfig datasets.

We address the problem of uncertainty quantification in the domain of face attribute classification, using Evidential Deep Learning (EDL) framework. The proposed EDL approach leverages the strength of Convolution Neural Networks (CNN), with the objective of representing the uncertainty in the output predictions. Predominantly, the softmax/sigmoid activation functions are applied to map the output logits of the CNN to target class probabilities in multi-class classification problems. By replacing the standard softmax/sigmoid output of a CNN with the parameters of the evidential distribution, EDL learns to represent the uncertainty in its predictions. The proposed approach is evaluated on CelebA and LFWA datasets. The quantitative and qualitative analysis demonstrate the suitability and strength of EDL to estimate the uncertainty in the output predictions without hindering the accuracy of CNN-based models.

We address the problem of web supervised learning, in particular for face attribute classification. Web data suffers from image set noise, due to unrelated images that may be retrieved in response to the query. We propose a semi-supervised pseudo-labeling approach where the embedding space distribution is learnt via variational contrastive learning. We use 40 Gaussian sampling heads for the 40 attributes in the CelebA dataset and apply supervised contrastive learning over a limited amount of labelled data, to address the multi-label face attribute classification problem. Soft pseudo-labeling is then used to label the unlabelled data at attribute level, followed by two-stage domain adaptation. We show that the proposed method using noisy web data brings improvements in accuracy over supervised multi-label face attribute classification in all experimental settings (over 2% points for very low-data setting). We suggest that learning the embedding distribution and the subsequent soft pseudo-labeling according to the nearest neighbors help in overcoming the noise in the unlabeled data.

Social media posts containing hate speech are reproduced and redistributed at an accelerated pace, reaching greater audiences at a higher speed. We present a machine learning system for automatic detection of hate speech in Turkish, along with a hate speech dataset consisting of tweets collected in two separate domains. We first adopted a definition for hate speech that is in line with our goals and amenable to easy annotation; then designed the annotation schema for annotating the collected tweets. The Istanbul Convention dataset consists of tweets posted following the withdrawal of Turkey from the Istanbul Convention. The Refugees dataset was created by collecting tweets about immigrants by filtering based on commonly used keywords related to immigrants. Finally, we have developed a hate speech detection system using the transformer architecture (BERTurk), to be used as a baseline for the collected dataset. The binary classification accuracy is 77% when the system is evaluated using 5-fold cross-validation on the Istanbul Convention dataset and 71% for the Refugee dataset. We also tested a regression model with 0.66 and 0.83 RMSE on a scale of [0-4], for the Istanbul Convention and Refugees datasets.

Hearings of witnesses and defendants play a crucial role when reaching court trial decisions. Given the high-stakes nature of trial outcomes, developing computational models that assist the decision-making process is an important research venue. In this article, we address the identification of deception in real-life trial data. We use a dataset consisting of videos collected from public court trials. We explore the use of verbal and non-verbal modalities to build a multimodal deception detection system that aims to discriminate between truthful and deceptive statements provided by defendants and witnesses. In particular, three complementary modalities (visual, acoustic and linguistic) are evaluated for the classification of deception at the subject level. The final classifier is obtained by combining the three modalities via score-level classification, achieving 83.05 percent accuracy in subject-level deceit detection. To place our results in perspective, we present a human deception detection study where we evaluate the human capability of detecting deception using different modalities and compare the results to the developed system. The results show that our system outperforms the average non-expert human capability of identifying deceit.

The amount of training data that is required to train a classifier scales with the dimensionality of the feature data. In hyperspectral remote sensing (HSRS), feature data can potentially become very high dimensional. However, the amount of training data is oftentimes limited. Thus, one of the core challenges in HSRS is how to perform multiclass classification using only relatively few training data points. In this letter, we address this issue by enriching the feature matrix with synthetically generated sample points. These synthetic data are sampled from a Gaussian mixture model (GMM) fitted to each class of the limited training data. Although the true distribution of features may not be perfectly modeled by the fitted GMM, we demonstrate that a moderate augmentation by these synthetic samples can effectively replace a part of the missing training samples. Doing so, the median gain in classification performance is 5% on two datasets. This performance gain is stable for variations in the number of added samples, which makes it easy to apply this method to real-world applications.

As biometric applications are gaining popularity, there is increased concern over the loss of privacy and potential misuse of biometric data held in central repositories. We present a biometric authentication framework that constructs a multi-biometric template by layering multiple biometrics of a user, such that it is difficult to separate the individual layers. Thus, the framework uses the biometrics of the user to conceal them among one another. The resulting biometric template is also cancelable if the system is implemented with cancelable biometrics, such as voice. We present a realization of this idea combining two or three different fingerprints of the user, using four different methods of template construction. Three of the methods use less and less information of the constituent biometrics, so as to lower the risk of leakage and cross-link rates. Results are evaluated on publicly available Finger Verification Championship (FVC) 2000, 2002 and NIST fingerprint databases. With the FVC databases, we obtain 2.1%, 3.9% and 3.4% Equal Error Rate on average using the three proposed methods, while the state-of-the-art commercial system achieves 1.9%. Furthermore, we show low cross-link rates under 63% under different scenarios, while genuine identification rates are 100%, with such a small gallery of 55 templates.

Abstract Sentiment analysis has attracted a lot of research interest in recent years, especially in the context of social media. While most of this research has focused on English, there is ample data and interest in the topic for many other languages, as well. In this article, we propose a comprehensive sentiment analysis system for Turkish. We cover different levels of sentiment analysis such as aspect, sentence, and document levels as well as some linguistic issues such as conjunction and intensification in Turkish sentiment analysis. Our system is evaluated on Turkish movie reviews and the obtained accuracies range from sixty per cent to seventy-nine per cent in ternary and binary classification tasks at different levels of analysis.

Error Correcting Output Coding (ECOC) is a multiclass classification technique in which multiple binary classifiers are trained according to a preset code matrix such that each one learns a separate dichotomy of the classes. While ECOC is one of the best solutions for multi-class problems, one issue which makes it suboptimal is that the training of the base classifiers is done independently of the generation of the code matrix. In this paper, we propose to modify a given ECOC matrix to improve its performance by reducing this decoupling. The proposed algorithm uses beam search to iteratively modify the original matrix, using validation accuracy as a guide. It does not involve further training of the classifiers and can be applied to any ECOC matrix. We evaluate the accuracy of the proposed algorithm (BeamECOC) using 10-fold cross-validation experiments on 6 UCI datasets, using random code matrices of different sizes, and base classifiers of different strengths. Compared to the random ECOC approach, BeamECOC increases the average cross-validation accuracy in 83.3% of the experimental settings involving all datasets, and gives better results than the state-of-the-art in 75% of the scenarios. By employing BeamECOC, it is also possible to reduce the number of columns of a random matrix down to 13% and still obtain comparable or even better results at times.

Effective spatial-spectral pixel description is of crucial significance for the classification of hyperspectral remote sensing images. Attribute profiles are considered as one of the most prominent approaches in this regard, since they can capture efficiently arbitrary geometric and spectral properties. Lately though, the advent of deep learning in its various forms has also led to remarkable classification performances by operating directly on hyperspectral input. In this letter, we explore the collaboration potential of these two powerful feature extraction approaches. Specifically, we propose a new strategy for hyperspectral image classification, where attribute filtered images are stacked and provided as input to convolutional neural networks. Our experiments with two real hyperspectral remote sensing data sets show that the proposed strategy leads to a performance improvement, as opposed to using each of the involved approaches individually.

Although plant recognition has become an increasingly popular research topic, it remains nonetheless a scientific and technical challenge. Besides all the difficulties of classic object recognition, such as illumination, viewpoint and scale variations, plants can additionally exhibit visual changes depending on their age and condition, thus demanding a specialized approach. In this paper, we present two descriptors based on mathematical morphology; the first consists of the computation of morphological covariance on the leaf contour profile and the second is an extension of the recently introduced circular covariance histogram, capturing leaf venation characteristics. The effectiveness of both descriptors has been validated with the ImageClef'12 plant identification dataset.

In this paper, we present the main results of the BioSecure Signature Evaluation Campaign (ESRA'2011). The objective of ESRA'2011 is to evaluate through two different tasks the resistance of different online signature systems to skilled forgeries categorized automatically according to their quality. Task 1 aims at studying with only coordinate time functions the influence of acquisition conditions (digitizing tablet vs. PDA) on systems' performance. The two BioSecure Data Sets DS2 and DS3 make this possible, since they contain data from the same 382 people, acquired respectively on a digitizer and on a PDA. Task 2 then aims at assessing the contribution of the five time functions available on a digitizer (coordinates, pressure, pen inclination) on systems' resistance to different qualities of skilled forgeries. Results of the 13 systems involved in this competition are reported and analyzed for both tasks in this paper. We observe that the best system in terms of performance on forgeries of "bad" quality is not necessarily the most resistant to an increased quality of skilled forgeries. Also, we note that mobile conditions are still threatening independently of the quality of forgeries. Finally, when adding pen inclination time functions to pressure and coordinates, we find that the gap between systems in terms of performance is wider than when only pen coordinates and pressure are considered.

We present an offline signature verification system based on a signature's local histogram features. The signature is divided into zones using both the Cartesian and polar coordinate systems and two different histogram features are calculated for each zone: histogram of oriented gradients (HOG) and histogram of local binary patterns (LBP). The classification is performed using Support Vector Machines (SVMs), where two different approaches for training are investigated, namely global and user-dependent SVMs. User-dependent SVMs, trained separately for each user, learn to differentiate a user's signature from others, whereas a single global SVM trained with difference vectors of query and reference signatures' features of all users, learns how to weight dissimilarities. The global SVM classifier is trained using genuine and forgery signatures of subjects that are excluded from the test set, while user dependent SVMs are separately trained for each subject using genuine and random forgeries. The fusion of all classifiers (global and user-dependent classifiers trained with each feature type), achieves a 15.41 % equal error rate in skilled forgery test, in the GPDS 160 signature database without using any skilled forgeries in training.

We present a probabilistic framework for the mathematical expression recognition problem. The developed system is flexible in that its grammar can be extended easily thanks to its graph grammar which eliminates the need for specifying rule precedence. It is also optimal in the sense that all possible interpretations of the expressions are expanded without making early commitments or hard decisions. In this paper, we give an overview of the whole system and describe in detail the graph grammar and the parsing process used in the system, along with some preliminary results on character, structure and expression recognition performances.

We present a content-based image retrieval system for plant image retrieval, intended especially for the house plant identification problem. A plant image consists of a collection of overlapping leaves and possibly flowers, which makes the problem challenging. We studied the suitability of various well-known color, shape and texture features for this problem, as well as introducing some new texture matching techniques and shape features. Feature extraction is applied after segmenting the plant region from the background using the max-flow min-cut technique. Results on a database of 380 plant images belonging to 78 different types of plants show promise of the proposed new techniques and the overall system: in 55% of the queries, the correct plant image is retrieved among the top-15 results. Furthermore, the accuracy goes up to 73% when a 132-image subset of well-segmented plant images are considered.

We describe a new approach for evaluating page segmentation algorithms. Unlike techniques that rely on OCR output, our method is region-based: the segmentation output, described as a set of regions together with their types, output order etc., is matched against the pre-stored set of ground-truth regions. Misclassifications, splitting, and merging of regions are among the errors that are detected by the system. Each error is weighted individually for a particular application and a global estimate of segmentation quality is derived. The system can be customized to benchmark specific aspects of segmentation (e.g., headline detection) and according to the type of error correction that might follow (e.g., re-typing). Segmentation ground-truth files are quickly and easily generated and edited using GroundsKeeper, an X-Window based tool that allows one to view a document, manually draw regions (arbitrary polygons) on it, and specify information about each region (e.g., type, parent).

We present a system for recognizing off-line, cursive, English text, guided in part by global characteristics (style) of the handwriting. We introduce a new method for segmenting words into letters, based on minimizing a cost function. Segmented letters are normalized with a novel algorithm that scales different parts of a letter separately removing much of the variation in the writing. We use a neural network for letter recognition and use the output of the network as posterior probabilities of letters in the word recognition process. We found that using a hidden Markov Model for word recognition is less successful than assuming an independent process for our small set of test words. In our experiments with several hundred words, written by 7 writers, 96% of the test words were correctly segmented, 52% were correctly recognized, and 70% were in the top three choices.>