Technology is getting increasingly complicated. If complex technologies have the potential to cause harm, people may need protection. Such legal protection is increasingly available, but it is only effective if it can also be enforced in courts. If people do not understand what is happening, for instance, with their personal data, they may not go to court at all. When people go to court, they may encounter another problem: if also courts and judges have a limited understanding of how the complex technologies work, this can affect the right to a fair trial. In this paper, it is argued that a fair trial requires that courts and judges need to have sufficient understanding of the technology in cases on which they are ruling. Since not all judges can be trained to have deep understanding of technology, other ways to address this are proposed.

The EU General Data Protection Regulation (GDPR) contains several data subject rights, but for many of these rights it is not entirely clear how they should work in practice, especially in digital environments. Most data subject rights apply to personal data obtained directly or indirectly from the data subject. This is often personal data that data subjects already are familiar with, i.e., things they already know about themselves. Unclear, however, is to what extent ascribed personal data, such as inferred data and categories or profiles in which data subjects are placed by data controllers, are within the scope of these rights. Such ascribed personal data often concerns novel information, generated by data controllers, and includes insights into how controllers view and assess them, which may have practical and legal impact on data subjects. Given these characteristics, the ascribed personal data may be much more interesting to data subjects, so it appears beneficial, from the policy perspective, to have this novel information included in the scope of data subject rights. If data subject rights do not apply to inferred data and profiles, invoking these rights is unlikely to be informative and provide meaningful information for data subjects, particularly in complex, digital environments. However, if data subject rights do apply to inferred data and profiles, the scope of these rights may be hard to delineate and they may quickly interfere with rights and freedoms of others, including trade secrets of data controllers and privacy rights of other data subjects. In this article, we investigate the implications of applying data subject rights to inferred data and profiles. For each data subject right in the GDPR, we assess which types of personal data could and perhaps should be in scope, based on grammatical and teleological legal analyses as well as practical considerations. While the area of data subject rights received significant academic attention in the past years, our article contributes to the discussion by providing a systematic, holistic framework to consider the scope of the rights in relation to ascribed data.

According to the fairness principle in Article 5.1a of the EU General Data Protection Regulation (GDPR), data controllers must process personal data fairly. However, the GDPR fails to explain what is fairness and how it should be achieved. In fact, the GDPR focuses mostly on procedural fairness: if personal data are processed in compliance with the GDPR, for instance, by ensuring lawfulness and transparency, such processing is assumed to be fair. Because some forms of data processing can still be unfair, even if all the GDPR's procedural rules are complied with, we argue that substantive fairness is also an essential part of the GDPR's fairness principle and necessary to achieve the GDPR's goal of offering effective protection to data subjects. Substantive fairness is not mentioned in the GDPR and no guidance on substantive fairness is provided. In this paper, we provide elements of substantive fairness derived from EU consumer law, competition law, non-discrimination law, and data protection law that can help interpret the substantive part of the GDPR's fairness principle. Three elements derived from consumer protection law are good faith, no detrimental effects, and autonomy (e.g., no misleading or aggressive practices). We derive the element of abuse of dominant position (and power inequalities) from competition law. From other areas of law, we derive non-discrimination, vulnerabilities, and accuracy as elements relevant to interpreting substantive fairness. Although this may not be a complete list, cumulatively these elements may help interpret Article 5.1a GDPR and help achieve fairness in data protection law.

This article proposes a new visual analysis method of hazardous events to be used in contract risk management. We present our research work for the creation of an ontological extension of the Onassis Ontology to manage, analyse and visualise risk data. Onassis is an openly accessible ontology that we earlier designed to structure contract automation data. Onassis and its extension for risk management contribute to the development of trustworthy Intelligent Contracts (iContracts). They allow for the creation of explicit data out of usually im plicit contractual information and legal processes on which it is possible to perform cross-referencing analysis with other collections of data. The ontological model that resulted from our study additionally contributes to the disambiguation of the bow-tie method structure, the primary method for analysing and visualising haz ardous events. To achieve this, we use the following methodology. We visualise the bow-tie method in an ontology and then investigate the presence of taxonomic ambiguities or even errors in its structure. The results present an enriched version of bow-tie conceptualisation, in which entities and relationships are translated into openly-accessible and ready-to-use ontological terms, whereas risk analysis becomes visible.

Computer Systems, Imagery and Media

Artificial intelligence (AI) offers opportunities but also challenges for biomedical research and healthcare. This position paper shares the results of the international conference "Fair medicine and AI" (online 3-5 March 2021). Scholars from science and technology studies (STS), gender studies, and ethics of science and technology formulated opportunities, challenges, and research and development desiderata for AI in healthcare. AI systems and solutions, which are being rapidly developed and applied, may have undesirable and unintended consequences including the risk of perpetuating health inequalities for marginalized groups. Socially robust development and implications of AI in healthcare require urgent investigation. There is a particular dearth of studies in human-AI interaction and how this may best be configured to dependably deliver safe, effective and equitable healthcare. To address these challenges, we need to establish diverse and interdisciplinary teams equipped to develop and apply medical AI in a fair, accountable and transparent manner. We formulate the importance of including social science perspectives in the development of intersectionally beneficent and equitable AI for biomedical research and healthcare, in part by strengthening AI health evaluation.

The principle of equal treatment (i.e., all people have the right to be treated equally) is protected by non-discrimination provisions in national constitutions across the EU as well as the EU Charter of Fundamental Rights (CFEU). These provisions specify which grounds (e.g., gender, race, religion) are prohibited to use as the basis for making decisions on people, such as offering a person a job. In the data economy, in which large amounts of personal data are collected and analyzed, it has become possible to make decisions on people on the basis of all kinds of grounds, also grounds that are not protected in anti-discrimination law (e.g., zip code, shoe size, wealth). Even though mostly unintentional, patterns revealed by sophisticated data analysis can turn out to be discriminatory, either directly or indirectly. Particularly indirect discrimination (i.e., discrimination by proxy) can be hard to discover and enforce. From a substantive perspective, these technological developments also raise the question which discrimination grounds should be protected, since discrimination grounds are in flux and not harmonized across the EU. In this paper, through legal comparison, discrimination grounds across EU national constitutions and the CFEU are compared, to identify overlaps and differences. This overview is then used to start the discussion on the extent to which current legislation is still appropriate in the data economy or should perhaps be reconsidered.

The increasing use of digital technologies by governments and companies raises numerous questions regarding the regulation of these technologies, particularly regarding the rights and legal protections citizens are entitled to. The focus is mostly on the application and potential modification of existing (fundamental) rights. However, the debate and legal research in this area lacks a broader discussion on which new rights citizens should have in the digital era. Only now and then new concepts surface, such as the ‘right to be forgotten’. This article deals with the question which new, additional rights could be imagined in the digital era if we were to draft them right now, from scratch, rather than being tied to a set of existing fundamental rights. In order to start a broader legal debate on this, various new rights for citizens in the digital area are proposed.

The right of access in Article 15 of the EU General Data Protection Regulation (GDPR) is essential for empowering data subjects when exercising other data subject rights. In the context of automated decision-making, including profiling, Article 15(1)(h) provides a right to meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject. In this research, the notions of ‘meaningful information’, ‘the logic involved’, and ‘the significance and the envisaged consequences’ are analysed. Apart from a legal analysis, empirical research was carried out in 30 countries (27 EU and 3 EFTA EEA Member States), consisting of a survey amongst all Data Protection Authorities (DPAs) and interviews with experts of privacy organisations. Several types of potentially meaningful information that could be in scope of right of access requests were assessed, as well as several types of information on the consequences for data subjects. Even though respondents indicate that most of these types of information should be provided upon access requests, the findings show that most of these types of information are rarely or not at all provided in practice. Providing such information strongly depends on the willingness of data controllers to cooperate, as they may balance this against rights and freedoms of others, including intellectual property and trade secrets. Particularly trade secrets are invoked in practice to block or restrict access requests, despite the GDPR stating that these considerations should not lead to a refusal of the request to provide all information to the data subject.

Many free online services, including search engines and social media, use business models based on the collecting and processing of personal data of its users. The user data are analysed, leased or sold to generate profits. Basically, the users are not paying for the services with subscription fees or any kind of monetary payment, but with their personal data. In this paper, we argue that these business models, treating personal data as a commodity, are problematic under EU data protection law, which disqualifies personal data as a commodity. Both under the EU Charter of Fundamental Rights and the General Data Protection Regulation (GDPR), the legal rights to data protection are inalienable. This is at odds with the actual trade in personal data in the data economy, since the ‘payment’ cannot be a transfer of ownership of personal data. It could be argued that the ‘payment’ is not a transfer of ownership of personal data, but rather a transfer of personal data rights, i.e., granting a right to collect and process the data. However, even from that perspective, users would retain inalienable rights to stop or restrict the data processing, as the GDPR does not allow mandating data subject rights to others. Because the legal basis for the processing of personal data is often consent, people can invoke their data subject rights (and thus withdraw their ‘payment’) at any time and at will after having received (access to) online services. This causes considerable legal uncertainty in transactions, particularly on the side of data controllers, and may not contribute to the EU's envisioned data economy.

In healthcare, gender and sex considerations are crucial because they affect individuals' health and disease differences. Yet, most algorithms deployed in the healthcare context do not consider these aspects and do not account for bias detection. Missing these dimensions in algorithms used in medicine is a huge point of concern, as neglecting these aspects will inevitably produce far from optimal results and generate errors that may lead to misdiagnosis and potential discrimination. This paper explores how current algorithmic-based systems may reinforce gender biases and affect marginalized communities in healthcare-related applications. To do so, we bring together notions and reflections from computer science, queer media studies, and legal insights to better understand the magnitude of failing to consider gender and sex difference in the use of algorithms for medical purposes. Our goal is to illustrate the potential impact that algorithmic bias may have on inadvertent discriminatory, safety, and privacy-related concerns for patients in increasingly automated medicine. This is necessary because by rushing the deployment of AI technologies that do not account for diversity, we risk having an even more unsafe and inadequate healthcare delivery. By promoting the account for privacy, safety, diversity, and inclusion in algorithmic developments with health-related outcomes, we ultimately aim to inform the Artificial Intelligence (AI) global governance landscape and practice on the importance of integrating gender and sex considerations in the development of algorithms to avoid exacerbating existing or new prejudices.

Online price discrimination (OPD) or personalized pricing has triggered many debates in the existing literature, due to its potential adverse effects on (trust in) markets and certain consumer groups. The implicit assumption underpinning these debates is the actual use of OPD in practice, although empirical research has not provided evidence of its common use. The aim of this study is to explore company perspectives regarding OPD and possible explanations as to why this practice is not widespread. Semi-structured interviews were held with 14 data scientists, sales managers, marketing directors and policy experts from Dutch companies. The findings indicate that companies are reluctant towards using OPD, particularly selective price increases. They rather seek other forms of data application for personalized marketing. Next to economic, technological, legal, and ethical considerations, consumer backlash was another key factor in the decision to engage in OPD. While companies do not seem convinced that the current regulatory initiatives are sufficiently effective, they support self-regulation and ethical codes. However, as more covert or indirect forms of OPD are expected, continued attention and scrutiny by regulators is warranted.

Online and social media platforms employ automated recognition methods to presume user preferences, sensitive attributes such as race, gender, sexual orientation, and opinions. These opaque methods can predict behaviors for marketing purposes and influence behavior for profit, serving attention economics but also reinforcing existing biases such as gender stereotyping. Although two international human rights treaties include explicit obligations relating to harmful and wrongful stereotyping, these stereotypes persist online and offline. By identifying how inferential analytics may reinforce gender stereotyping and affect marginalized communities, opportunities for addressing these concerns and thereby increasing privacy, diversity, and inclusion online can be explored. This is important because misgendering reinforces gender stereotypes, accentuates gender binarism, undermines privacy and autonomy, and may cause feelings of rejection, impacting people's self-esteem, confidence, and authenticity. In turn, this may increase social stigmatization. This study brings into view concerns of discrimination and exacerbation of existing biases that online platforms continue to replicate and that literature starts to highlight. The implications of misgendering on Twitter are investigated to illustrate the impact of algorithmic bias on inadvertent privacy violations and reinforcement of social prejudices of gender through a multidisciplinary perspective, including legal, computer science, and critical feminist media-studies viewpoints. An online pilot survey was conducted to better understand how accurate Twitter's gender inferences of its users’ gender identities are. This served as a basis for exploring the implications of this social media practice.