The integration of Large Language Models (LLMs) into medicine presents both great opportunities and significant challenges, particularly in ensuring these models are accurate, reliable, and safe. While LLMs have shown impressive capabilities in understanding and generating human language, their application in the medical domain requires careful evaluation due to the critical nature of medical applications which are inherently linked to patient life and health. Current evaluations of LLMs in medicine are often fragmented and insufficient, with a lack of standardized performance metrics, limited use of real patient data, and insufficient attention to important applications, such as documentation, education, and research. Furthermore, traditional NLP-based evaluations are often inadequate for assessing the text generated by LLMs. Therefore, a robust evaluation is essential to ensure the responsible and effective use of LLMs in medical settings, and to address the inherent challenges associated with their implementation. This paper explores the various dimensions of LLM evaluation in the medical domain, proposes a new taxonomy for categorizing medical applications, and discusses directions for future research in this critical area.

Quantitative algorithmic trading and machine learning have transformed financial markets by introducing advanced computational techniques for market analysis, decision-making, risk management, and order execution. This paper explores Deep Reinforcement Learning (DRL) for developing day trading strategies within the Ibovespa futures market, the world's most traded stock index futures contract. We chose the Ibovespa market for its high liquidity and volatility, which present unique challenges and opportunities for trading. We propose a novel methodology for the development, evaluation, and deployment of DRL-powered strategies. This methodology involves model generation, approval, selection, and execution, supported by comprehensive metrics for assessing returns, risk, and stability. We introduce the FUT-DRL trading strategy, a new approach that models the trading problem as a Partially Observable Markov Decision Process using DRL agents and the Deep Q-Network algorithm to navigate market complexities. Our empirical study, utilizing a diverse five-year dataset under varied conditions, shows that FUT-DRL outperforms traditional strategies, such as trend-following and mean - reversion, achieving an annualized return over 100 % and a Probabilistic Sharpe Ratio of 0.96. This research advances the integration of machine learning with quantitative trading, offering a robust framework for strategy development, and highlighting DRL's effectiveness in the volatile Ibovespa market, demonstrating the potential of adaptive and intelligent systems to evolve trading strategies in rapidly changing financial markets.

This paper explores using algorithmic recourse as a tool in mental healthcare. Algorithmic recourse provides explanations and recommendations to individuals who want to reverse a machine learning prediction and has been widely used in various domains such as finance and marketing. However, its application in mental healthcare has been restricted. This paper addresses this by examining the potential benefits and challenges of using algorithmic recourse in mental healthcare, specifically in how changing one's behavior may affect their quality of life and well-being. The paper proposes a new classification-based framework for algorithmic recourse in mental healthcare. The proposed framework considers both observed and latent variables to account for the individuality of individuals and provides a more comprehensive understanding of mental health outcomes. The research results can provide valuable insights for future work and help bridge the gap between machine learning and mental healthcare.

Nowadays, Recommender Systems have played a crucial role in several entertainment scenarios by making personalised recommendations and guiding the entire users’ journey from their first interaction. Recent works have addressed it as a Contextual Bandit by providing a sequential decision model to explore items not tried yet (or not tried enough) or exploit the best options learned so far. However, this work noticed these current algorithms are limited to naive non-personalised approaches in the first interactions of a new user, offering random or most popular items. Through experiments in three domains, we identify a negative impact of these first choices. Our study indicates that the bandit performance is directly related to the choices made in the first trials. Then, we propose a new approach to balance exploration and exploitation in the first interactions and handle these drawbacks. This approach is based on the Active Learning theory to catch more information about the new users and improve their long-term experience. Our idea is to explore the potential information gain of items that can also please the user’s taste. This method is named Warm-Starting Contextual Bandits, and it statistically outperforms 10 benchmarks in the literature in the long run.

Nowadays, most e-commerce and entertainment services have adopted interactive Recommender Systems (RS) to guide the entire journey of users into the system. This task has been addressed as a Multi-Armed Bandit problem where systems must continuously learn and recommend at each iteration. However, despite the recent advances, there is still a lack of consensus on the best practices to evaluate such bandit solutions. Several variables might affect the evaluation process, but most of the works have only been concerned about the accuracy of each method. Thus, this work proposes an interactive RS framework named iRec. It covers the whole experimentation process by following the main RS guidelines. The iRec provides three modules to prepare the dataset, create new recommendation agents, and simulate the interactive scenario. Moreover, it also contains several state-of-the-art algorithms, a hyperparameter tuning module, distinct evaluation metrics, different ways of visualizing the results, and statistical validation.

This work is a companion reproducibility paper that presents a framework to reproduce our previous experiments and results reported in Werneck et al. (2021). In that previous paper, we introduced a systematic mapping process of points-of-interest (POI) recommendation methods and provided a uniform evaluation methodology based on metrics covering different aspects besides accuracy. Due to the lack of reproducible and extensible benchmarks, our work introduces a reproducibility framework for POI methods based on a collection of Python software libraries and a Docker image. Our proposal is composed of: (1) a package to perform a protocol that reproduces our systematic mapping process Werneck et al. (2021), containing all collected data, insightful views on current advances and opened challenges; and (2) an extensible benchmark to perform a protocol to reproduce experimental evaluations on POI recommendation, considering different datasets, metrics, and the strongest baselines in the literature. This work also demonstrates all processes required to instantiate its framework. Moreover, our work can be considered at least weakly reproducible, since we were able to reproduce the results of the previous paper, leading us to the same conclusions.

In the last few years, there was a growth regarding the use of computational methods in the field of finance, especially to negotiations in the stock market. In this paper, we aim to bring new ideas and approaches to the development of automated trading or bots based on historical data of financial series. Our model, named Pattern Searcher, was inspired in unsupervised learning methods and evolutionary optimization. Given a trading agent with its predefined parameters, the method uses the power of Genetic Algorithm (GA) to search, within a set of financial indicators, for the region that provides a higher positive financial return. This implementation exhibited desirable properties compared to some Machine Learning methods, such as the simplification of the system flow and the generation of rules that humans can clearly understand. Besides, we have generated strategy portfolios, composed by the strategies derived from the Pattern Searcher method, which were also optimized via GA. The system was able to generate very profitable trading agents and portfolios on the Brazilian stock market, surpassing important benchmarks.

In this study we investigate the potential of using Deep Reinforcement Learning (DRL) to day trade stocks, taking into account the constraints imposed by the stock market, such as liquidity, latency, slippage and transaction costs. More specifically, we use a Deep Deterministic Policy Gradient (DDPG) algorithm to solve a series of asset allocation problems in order to define the percentage of capital that must be invested in each asset at each period, executing exclusively day trade operations. DDPG is a model-free, off-policy actor-critic method that can learn policies in high-dimensional and continuous action and state spaces, like the ones normally found in financial market environments. The proposed day trading system was tested in B3 - Brazil Stock Exchange, an important and understudied market, especially considering the application of DRL techniques to alpha generation. A series of experiments were performed from the beginning of 2017 until the end of 2019 and compared with ten benchmarks, including Ibovespa, the most important Brazilian market index, and the stock portfolios suggested by the main Brazilian banks and brokers during these years. The results were evaluated considering return and risk metrics and showed that the proposed method outperformed the benchmarks by a huge margin. The best results obtained by the algorithm had a cumulative percentage return of 311% in three years, with an annual average maximum drawdown around 19%.

Forecasting financial time series is a problem studied by researchers from different fields, who are looking for effective ways to achieve financial gains. Over time, many authors conducted studies on the possible predictability of the series through different statistical tests, and recently several papers explore the application of machine learning algorithms to have better predictions. In this paper we analyzed real data of 11 time series related to Brazilian stocks, focusing on the statistical characteristics of the series and the use of an LSTM neural network to classify future values. We analyzed the results of 5 different variance ratio tests and their relationship with the neural network classification performance. This paper proposes the application of statistical tests in the LSTM training set to highlight previously those series that have more temporal dependence and, therefore, possibly better forecast results. The results showed that 5 out of 11 stocks rejected the random walk hypothesis through the variance ratio tests and that these same stocks obtained the best performances in terms of classification and financial return.

Advances in Recommender Systems (RSs) have been focused on improving the system's accuracy. However, accuracy alone is not enough to assess the practical effects. In real scenarios, diversity has been identified as a key dimension of recommendation utility. Thus, the main researches are focused in improve both, accuracy and diversity. This challenge remains an apparent dilemma that remains open and can boost sales by offering consumers both their mainstream and specific tastes. For this reason, we propose an approach to handle the accuracy-diversity dilemma. Our approach, based on a Particle Swarm Optimization (PSO), is a post-processing method to re-rank items from traditional RSs in order to improve diversity without accuracy losses. Experimental results in entertainment and e-commerce scenarios show that our strategy can improve users satisfaction. We improve the diversity up to 70% without significant accuracy losses.

Nowadays, it is possible to note many machine learning techniques being applied to predict financial time series. However, recent studies indicate that the performance of such techniques can be strongly affected by data representation. In this manuscript we propose a combination of two machine learning algorithms to detect trends in stock market prices. In this approach, Boltzmann Restricted Machines are used as the latent feature extractor and Support Vector Machines work as the classifier. We performed tests with real data of five assets from the Brazilian Stock Market, BM&FBOVESPA. The results obtained with the proposed combination were better when compared to those ones reached by Support Vector Machines only. This suggests that the proposed approach can be suitable for the considered application.

Before the advent of computers and Internet, the stock market investors perform their operations based mainly on intuition. With the growth of investments and online stock trading, a continued search for better tools to improve the prediction of stock market trends has become necessary in order to increase profits and reduce risks. In this work we propose and evaluate some algorithmic trading (algotrading) strategies, based on an Ensemble of artificial neural networks (ANN), to support the decision of stock market's investors. Thirty different ANN models, using different input sets of price, volume and technical indicators, were analyzed for different stock symbols (i.e., companies from different economy sectors) of the main Brazilian Stock Market-BM& FBovespa. Moreover, ensembles that combine the best ANN models were modeled and validated through different experiments. The results confirm the potential of the proposed strategies for algotrading.

Predictions on stock market prices are a great challenge due to the fact that it is an immensely complex, chaotic and dynamic environment. There are many studies from various areas aiming to take on that challenge and Machine Learning approaches have been the focus of many of them. There are many examples of Machine Learning algorithms been able to reach satisfactory results when doing that type of prediction. This article studies the usage of LSTM networks on that scenario, to predict future trends of stock prices based on the price history, alongside with technical analysis indicators. For that goal, a prediction model was built, and a series of experiments were executed and theirs results analyzed against a number of metrics to assess if this type of algorithm presents and improvements when compared to other Machine Learning methods and investment strategies. The results that were obtained are promising, getting up to an average of 55.9% of accuracy when predicting if the price of a particular stock is going to go up or not in the near future.

The aim of this study was to model and use machine learning techniques to maximize the chance of a market maker be executed successfully in a stock market, that is, when their bid and ask orders are filled at the desired prices. In this context, a binary ensemble classifier was created to decide whether, at a specific time, is or not propitious to start a new market making process. Conducting the study over a large volume of data for high-frequency traders, we showed that the new proposed ensemble classifier was able to improve the efficiency of the isolated models and the precision of the models are better than random decision makers.

The volume of online transactions has increased considerably in the recent years. Consequently, the number of fraud cases has also increased, causing billion dollar losses each year worldwide. Therefore, it is mandatory to employ mechanisms that are able to assist in fraud detection. In this work, it is proposed the use of Genetic Programming (GP) to identify frauds (charge back) in electronic transactions, more specifically in online credit card operations. A case study, using a real dataset from one of the largest Latin America electronic payment systems, has been conducted in order to evaluate the proposed algorithm. The presented algorithm achieves good performance in fraud detection, obtaining gains up to 17% with regard to the actual company baseline. Moreover, several classification problems, with considerably different datasets and domains, have been used to evaluate the performance of the algorithm. The effectiveness of the algorithm has been compared with other methods, widely employed for classification. The results show that the proposed algorithm achieved good classification effectiveness in all tested instances.

Artificial Neural Networks (ANN) have been frequently applied to reduce risks and maximize the net returns in different types of algorithm trading. Using a real dataset, and aiming to support the Market Making process in High-Frequency Trading, this work investigates the use of a multilayer perceptron (MLP) to predict positive oscillations in short time periods (5, 10 or 15 minutes). The statistical analysis of our results showed that a neural network is more effective in short-term oscillations (5 minutes) when compared with the results obtained in longer periods (10 or 15 minutes). The result is important because it allows to insert a higher quantity of limit orders once they will be placed more frequently, which increases the market liquidity. It contextualizes a new contribution in the High-Frequency Trading field, where this work proposes a new trigger to start a market making process.

Text-based social media channels, such as Twitter, produce torrents of opinionated data about the most diverse topics and entities. The analysis of such data (aka. sentiment analysis) is quickly becoming a key feature in recommender systems and search engines. A prominent approach to sentiment analysis is based on the application of classification techniques, that is, content is classified according to the attitude of the writer. A major challenge, however, is that Twitter follows the data stream model, and thus classifiers must operate with limited resources, including labeled data and time for building classification models. Also challenging is the fact that sentiment distribution may change as the stream evolves. In this paper we address these challenges by proposing algorithms that select relevant training instances at each time step, so that training sets are kept small while providing to the classifier the capabilities to suit itself to, and to recover itself from, different types of sentiment drifts. Simultaneously providing capabilities to the classifier, however, is a conflicting-objective problem, and our proposed algorithms employ basic notions of Economics in order to balance both capabilities. We performed the analysis of events that reverberated on Twitter, and the comparison against the state-of-the-art reveals improvements both in terms of error reduction (up to 14%) and reduction of training resources (by orders of magnitude).

The classification and treatment of different kinds of traffic in computer networks have become essential on the implementation of Quality of Service (QoS) policies. However, the correct setup of the traffic shaping mechanism is a challenge in terms of performance and robustness. This paper proposes a methodology for optimizing operational parameters of traffic shaping in network environments, using evolutionary algorithms and artificial neural networks. The real data captured from a large network infrastructure has been used for validation.

Web multimedia content has reached much importance lately. One of the most important content types is online video, as demonstrated by the success of platforms such as YouTube. The growth in the volume of available online video is also observed in corporate scenarios, such as TV station. This paper evaluates a set of corporate online videos hosted by Sambatech, a company that holds the largest platform for online multimedia content distribution in Latin America. We propose a novel analytical approach for video recommendation, focusing on video objects being consumed. After modeling this service, we characterize the contents from multiple sources, and propose techniques for multimedia content recommendation. Experimental results indicate that the proposed method is very promising, which had obtained almost 70 in precision. We also perform distinct evaluations using different approaches from literature, such as the state-of-the-art technique for item recommendation.

Due to the growing popularity of the Web, there is an increasing number of people who perform e-business transactions. On the other hand, this popularity has also attracted the attention of criminals, raising the number of frauds on the Web and associated financial losses, which reach billions of dollars per year. This paper proposes a KDD-based methodology to detect fraud in e-payment systems. In order to evaluate this methodology we defined the concept of economic efficiency and applied it to an actual dataset of one of the largest Latin American electronic payment systems. The results show a very good performance, providing gains of up to 46.5% in comparison with the strategy currently employed by the company.

The popularization of Web has given rise to new services every day, demanding mechanisms to ensure the credibility of these services. Since now, little has been done to measure and understand the credibility of this complex Web environment, which itself is a major research challenge. From the challenges related to the task of assigning a credibility value to an online service in Web 2.0 applications, we propose a framework for the design, implementation and evaluation of credibility models. We call a credibility model a function capable of assigning a credibility value to a transaction of a Web application, considering different criteria of this service and its supplier. To validate this framework and models, we perform experiments using an actual dataset, from which we evaluated different credibility models using distinct types of information sources, and it allows to compare and evaluate these credibility models. The obtained results are very good, showing representative gains, when compared to a baseline and also with a known state-of-the-art approach. The results confirm that the credibility framework can be used to enforce trust to users of services on the Web.

The popularization of Web has given rise to new services every day, demanding mechanisms to ensure the credibility of these services. Since now, little has been done to measure and understand the credibility of this complex Web environment, which itself is a major research challenge. From the challenges related to the task of assigning a credibility value to an online service in Web 2.0 applications, we propose a framework for the design, implementation and evaluation of credibility models. To validate the framework, we perform experiments using an actual dataset, from which we evaluated different credibility models using distinct types of information sources, and it allows to compare and evaluate these credibility models. The results show that the credibility framework is applicable and is capable of supporting decision making by users of Web services.

Electronic markets have early emerged as an important topic inside e-commerce research. An e-market is a digital ecosystem intended to provide their users with online services that will facilitate information exchange and transactions. This work presents a characterization and analysis of fixed-price online negotiations. Using actual data from a Brazilian marketplace, we analyze selling practices, considering seller profiles and selling strategies. There are important factors that can be considered when analyzing selling practices, such as the seller's reputation and experience, offer's price, duration, among others. We evaluate which factors impact on the success of selling practices in e-markets, which can be used to support seller's decision and recommend selling practices. Moreover, we investigate some important hypotheses about selling practices in online marketplaces, which allow us to state interesting conclusions, such as: a seller profile can achieve success or not in a trade, depending on the adopted strategy; the offer's price and how it is being advertised are two important success factors.

E-commerce is growing at an exponential rate. In the last decade, there has been an explosion of online commercial activity enabled by World Wide Web (WWW). These days, many consumers are less attracted to online auctions, preferring to buy merchandise quickly using fixed-price negotiations. Sales at Amazon.com, the leader in online sales of fixed-price goods, rose 37% in the first quarter of 2008. At eBay, where auctions make up 58% of the site's sales, revenue rose 14%. In Brazil, probably by cultural influence, online auctions are not been popular. This work presents a characterization and analysis of fixed-price online negotiations. Using actual data from a Brazilian marketplace, we analyze seller practices, considering seller profiles and strategies. We show that different sellers adopt strategies according to their interests, abilities and experience. Moreover, we confirm that choosing a selling strategy is not simple, since it is important to consider the seller's characteristics to evaluate the applicability of a strategy. The work also provides a comparative analysis of some selling practices in Brazil with popular worldwide marketplaces.

Online auction services have reached great popularity and revenue over the last years. A key component for this success is the seller. Few studies proposed analyzing how the seller and the auction configuration affect the negotiation results. In this work we propose a methodology to characterize online auctions by the seller's perspective. This methodology is based on: (1) recognizing the characteristics of the variables related to the auction results and (2) capturing the correlation among these variables to identify seller profiles and selling strategies. We applied our methodology to a real case study, using an eBay dataset, to validate two hypotheses about sellers and their practices. These results are useful to understand the complex mechanisms that guide ending prices, success (or failure), and the attraction of bids in online auctions, which can support decision strategies for buyers and sellers.

Due to the increasing amount of information being stored and accessible through the Web, Automatic Document Classification (ADC) has become an important research topic. ADC usually employs a supervised learning strategy, where we first build a classification model using pre-classified documents and then use it to classify unseen documents. One major challenge in building classifiers is dealing with the temporal evolution of the characteristics of the documents and the classes to which they belong. However, most of the current techniques for ADC do not consider this evolution while building and using the models. Previous results show that the performance of classifiers may be affected by three different temporal effects (class distribution, term distribution and class similarity). Further, it is shown that using just portions of the pre-classified documents, which we call contexts, for building the classifiers, result in better performance, as a consequence of the minimization of the aforementioned effects.

The great success of the Internet has raised new challenges in terms of applications and the satisfaction of their users. In fact, there is strong evidence that a significant part of the user behavior depends on its satisfaction. Users reactions may affect the load of a server, establishing successive interactions where the user behavior affects the system behavior and vice-versa. It is important to understand this interactive process to design systems more suited to user requirements. In this work we study and explain how this reactive interaction is performed by users and how it affects the system's performance. We perform experiments using a real server under a TPC-W-based workload generated using a reactive version of httperf. We also simulate different workload configurations in order to evaluate the effects on the system's load. The results show that accounting for reactivity causes a significant impact on the server's performance in terms of throughput and response time, raising the possibility of performance improvement of Web systems by considering reactivity

Designing systems with better performance and scalability is a real need to fulfill the user demands and generate profitable Web services. Being able to mimic user behavior and the workload they generate on the servers is fundamental to evaluate the performance of systems and their improvements. One aspect that is usually neglected by workload generators is the user reactivity, that is, how the users react to variable server response time. Further, it is not clear how the reactivity-related changes in the user generated workload affect the server and how these dependences converge. This paper addresses this problem by proposing, implementing, and validating a workload generator that accounts for reactivity while interacting with servers. Our workload generator is used, for instance, to generate workloads based on a TPC-W benchmark. These workloads are used to assess the impacts of reactivity on the performance of a Web application. The results show significant changes in terms of throughput and response time for the experiments, raising the possibility of improving the performance of Web systems considering user reactivity.