Abstract The release of ChatGPT to the general public has sparked discussions about the dangers of artificial intelligence (AI) among the public. The European Commission’s draft of the AI Act has further fueled these discussions, particularly in relation to the definition of AI and the assignment of risk levels to different technologies. Security concerns in AI systems arise from the need to protect against potential adversaries and to safeguard individuals from AI decisions that may harm their well-being. However, ensuring secure and trustworthy AI systems is challenging, especially with deep learning models that lack explainability. This paper proposes the concept of Controllable AI as an alternative to Trustworthy AI and explores the major differences between the two. The aim is to initiate discussions on securing complex AI systems without sacrificing practical capabilities or transparency. The paper provides an overview of techniques that can be employed to achieve Controllable AI. It discusses the background definitions of explainability, Trustworthy AI, and the AI Act. The principles and techniques of Controllable AI are detailed, including detecting and managing control loss, implementing transparent AI decisions, and addressing intentional bias or backdoors. The paper concludes by discussing the potential applications of Controllable AI and its implications for real-world scenarios.

Abstract It is commonly understood that the resilience of critical information technology (IT) systems based on artificial intelligence (AI) must be ensured. In this regard, we consider resilience both in terms of IT security threats, such as cyberattacks, as well as the ability to robustly persist under uncertain and changing environmental conditions, such as climate change or economic crises. This paper explores the relationship between resilience and sustainability with regard to AI systems, develops fields of action for resilient AI, and elaborates direct and indirect influences on the achievement of the United Nations Sustainable Development Goals. Indirect in this case means that a sustainability effect is reached by taking resilience measures when applying AI in a sustainability-relevant application area, for example precision agriculture or smart health.

Procurement is a critical step in the setup of systems, as reverting decisions made at this point is typically time-consuming and costly. Especially Artificial Intelligence (AI) based systems face many challenges, starting with unclear and unknown side parameters at design time of the systems, changing ecosystems and regulations, as well as problems of overselling capabilities of systems by vendors. Furthermore, the AI Act puts forth a great deal of additional requirements for operators of critical AI systems, like risk management and transparency measures, thus making procurement even more complex. In addition, the number of providers of AI systems is drastically increasing. In this paper we provide guidelines for the procurement of AI based systems that support the decision maker in identifying the key elements for the procurement of secure AI systems, depending on the respective technical and regulatory environment. Furthermore, we provide additional resources for utilizing these guidelines in practical procurement.

Malware authors often rely on code obfuscation to hide the malicious functionality of their software, making detection and analysis more difficult. One of the most advanced techniques for binary obfuscation is virtualization-based obfuscation, which converts the functionality of a program into the bytecode of a randomly generated virtual machine which is embedded into the protected program. To enable the automatic detection and analysis of protected malware, new deobfuscation techniques against virtualization-based obfuscation are constantly being developed and proposed in the literature.

Data quality is of paramount importance for the smooth functioning of modern data-driven AI applications with machine learning as a core technology. This is also true for medical AI , where malfunctions due to "dirty data" can have particularly dramatic harmful implications. Consequently, data cleansing is an important part in improving the usability of (Big) Data for medical AI systems. However, it should not be overlooked that data cleansing can also have negative effects on data quality if not performed carefully. This paper takes an interdisciplinary look at some of the technical and legal challenges of data cleansing against the background of European medical device law, with the key message that technical and legal aspects must always be considered together in such a sensitive context.

The protection of private information is a crucial issue in data-driven research and business contexts. Typically, techniques like anonymisation or (selective) deletion are introduced in order to allow data sharing, e. g. in the case of collaborative research endeavours. For use with anonymisation techniques, the k-anonymity criterion is one of the most popular, with numerous scientific publications on different algorithms and metrics. Anonymisation techniques often require changing the data and thus necessarily affect the results of machine learning models trained on the underlying data. In this work, we conduct a systematic comparison and detailed investigation into the effects of different k-anonymisation algorithms on the results of machine learning models. We investigate a set of popular k-anonymisation algorithms with different classifiers and evaluate them on different real-world datasets. Our systematic evaluation shows that with an increasingly strong k-anonymity constraint, the classification performance generally degrades, but to varying degrees and strongly depending on the dataset and anonymisation method. Furthermore, Mondrian can be considered as the method with the most appealing properties for subsequent classification.

The security and privacy of our online communications heavily relies on the entity authentication mechanisms provided by SSL. Those mechanisms in turn heavily depend on the trustworthiness of a large number of companies and governmental institutions for attestation of the identity of SSL services providers. In order to offer a wide and unobstructed availability of SSL-enabled services and to remove the need to make a large amount of trust decisions from their users, operating systems and browser manufactures include lists of certification authorities which are trusted for SSL entity authentication by their products. This has the problematic effect that users of such browsers and operating systems implicitly trust those certification authorities with the privacy of their communications while they might not even realize it. The problem is further complicated by the fact that different software vendors trust different companies and governmental institutions, from a variety of countries, which leads to an obscure distribution of trust. To give insight into the trust model used by SSL this thesis explains the various entities and technical processes involved in establishing trust when using SSL communications. It furthermore analyzes the number and origin of companies and governmental institutions trusted by various operating systems and browser vendors and correlates the gathered information to a variety of indexes to illustrate that some of these trusted entities are far from trustworthy. Furthermore it points out the fact that the number of entities we trust with the security of our SSL communications keeps growing over time and displays the negative effects this might have as well as shows that the trust model of SSL is fundamentally broken.

Today's capability of fast Internet-wide scanning allows insights into the Internet ecosystem, but the on-going transition to the new Internet Protocol version 6 (IPv6) makes the approach of probing all possible addresses infeasible, even at current speeds of more than a million probes per second. As a consequence, the exploitation of frequent patterns has been proposed to reduce the search space. Current patterns are manually crafted and based on educated guesses of administrators. At the time of writing, their adequacy has not yet been evaluated. In this paper, we assess the idea of pattern-based scanning for the first time, and use an experimental set-up in combination with three real-world data sets. In addition, we developed a pattern-based algorithm that automatically discovers patterns in a sample and generates addresses for scanning based on its findings. Our experimental results confirm that pattern-based scanning is a promising approach for IPv6 reconnaissance, but also that currently known patterns are of limited benefit and are outperformed by our new algorithm. Our algorithm not only discovers more addresses, but also finds implicit patterns. Furthermore, it is more adaptable to future changes in IPv6 addressing and harder to mitigate than approaches with manually crafted patterns.

Ever since the Snowden revelations regarding mass surveillance, the role of privacy protection in commodity communication software has gained increasing awareness in the general public. Still, during the last years many new messengers were developed for Android, where often privacy was not considered to be a key issue. Due to the widespread use of these apps even in corporate environments this opens up attack vectors that can result in advanced persistent threats. In this paper we analyze the most prominent messenger apps with respect to privacy concepts, focusing not only on the transmission layer regarding the support of encrypted communication, but also attacks targeting the communication metadata, e.g. detecting the existence of communication between users, as well as providing an enumeration of all users of a service. Furthermore, device theft and loss is a major issue regarding the protection of user privacy. Thus, we also analyzed, whether the messages are stored in a secure way on the device itself, or if control over the physical device allows access to the message data. In order to analyze the possible usability of these messengers as means for targeted surveillance of users by the provider (or an entity controlling it), we also analyzed the rights and privileges the respective apps need in order to be able to install and work. Here, major differences could be detected, with several apps claiming privileges that could not be explained with the normal mode of operation, thus posing a serious risk for the privacy of the respective user base.

While the resilience of software-only code obfuscation remains unclear and ultimately depends only on available resources and patience of the attacker, hardware-based software protection approaches can provide a much higher level of protection against program analysis. Almost no systematic research has been done on the interplay between hardware and software based protection mechanism. In this paper, we propose modifications to Intel's AES-NI instruction set in order to make it suitable for application in software protection scenarios and demonstrate its integration into a control flow obfuscation scheme. Our novel approach provides strong hardware-software binding and restricts the attack context to pure dynamic analysis - two major limiting factors of reverse engineering - to delay a successful attack against a program.

In recent years mobile messaging and VoIP applications for smartphones have seen a massive surge in popularity, which has also sparked the interest in research related to the security of these applications. Various security researchers and institutions have performed in-depth analyses of specific applications or vulnerabilities. This paper gives an overview of the status quo in terms of security for a number of selected applications in comparison to a previous evaluation conducted two years ago, as well as performing an analysis on some new applications. The evaluation methods mostly focus on known vulnerabilities in connection with authentication and validation mechanisms but also describe some newly identified attack vectors. The results show a predominantly positive trend for new applications, which are mostly being developed with robust security and privacy features, while some of the older applications have shown little to no progress in this regard or have even introduced new vulnerabilities in recent versions.

As malicious software gets increasingly sophisticated and resilient to detection, new concepts for the identification of malicious behavior are developed by academia and industry alike. While today's malware detectors primarily focus on syntactical analysis (i.e., signatures of malware samples), the concept of semantic-aware malware detection has recently been proposed. Here, the classification is based on models that represent the underlying machine and map the effects of instructions on the hardware. In this paper, we demonstrate the incompleteness of these models and highlight the threat of malware, which exploits the gap between model and machine to stay undetectable. To this end, we introduce a novel concept we call covert computation, which implements functionality in side effects of microprocessors. For instance, the flags register can be used to calculate basic arithmetical and logical operations. Our paper shows how this technique could be used by malware authors to hide malicious code in a harmless-looking program. Furthermore, we demonstrate the resilience of covert computation against semantic-aware malware scanners.

InnoDB is a powerful open-source storage engine for MySQL that gained much popularity during the recent years. This paper proposes methods for forensic analysis of InnoDB databases by analyzing the redo logs, primarily used for crash recovery within the storage engine. This new method can be very useful in forensic investigations where the attacker got admin privileges, or was the admin himself. While such a powerful attacker could cover tracks by manipulating the log files intended for fraud detection, data cannot be changed easily in the redo logs. Based on a prototype implementation, we show methods for recovering Insert, Delete and Update statements issued against a database.

Digital forensics is the application of techniques to recover, reconstruct and analyze data from a computer or a similar system in order to gather digital evidence (e.g. on a suspicious employee or for law enforcement). Guidelines and standards for forensic investigations exist (e.g. NIST SP800-86), but do not cover Enterprise Rights Management (ERM), where data is usually encrypted and therefore inaccessible without knowing the cryptographic key. This paper explores forensic techniques for ERM systems and develops application specific guidelines for forensic investigations targeting Microsoft Active Directory Rights Management Services (RMS) and Adobe LiveCycle Rights Management. Moreover, we illustrate the important role of database forensics for investigations in ERM systems and finally show that with Microsoft's ERM solution no secure, centrally-managed revocation of specific documents in order to prevent digital forensics is feasible.

The Interdependency Modeling Tool and Simulation (INMOTOS) project is aimed to develop a tool for modeling and assessment of interdependent business- and contingency plans and risks affecting them. In the scope of that project a methodology had to be created that enables the modeling of highly complex business processes, their structures and interdependencies, as well as threats and countermeasures. A time-based simulation of the impact of possible threats is required as well as a risk assessment by using multiple different impact calculations. The methodology shall be kept simple and flexible to enable modeling of a wide range of different business scenarios. For the fundamental basics the Risk-Oriented Process Evaluation (ROPE) methodology [7] was chosen due to its high flexibility. This paper describes the adaptations and enhancements that are applied on the ROPE methodology to refine it to the INMOTOS methodology.

In the event of large natural and artificial disasters, it is of vital importance to provide all sorts of data to the relief organizations (fire department, red cross,...) to enhance their effectivity. Still, some of this data (e.g. regarding personal information on health status) may be considered private. k-anonymity can be utilized to mitigate the risks resulting from disclosure of such data, however, sometimes it is not possible to achieve a suitable size for k in order to completely anonymize the data without interfering with rescue operations. Still, this data will be sensitive after the disaster recovery is finished. Thus we aim at protecting the data by devising an intrinsic fingerprinting-scheme that allows to detect the source of eventually disclosed information afterwards. Our approach uses the properties directly derived from the anonymization process to generate unique fingerprints for every data set.

Today's database management systems implement sophisticated access control mechanisms to prevent unauthorized access and modifications. This is, as an example, an important basic requirement for SOX (Sarbanes--Oxley Act) compliance, whereby every past transaction has to be traceable at any time. However, malicious database administrators may still be able to bypass the security mechanisms to make hidden modifications to the database.

This paper examines QR Codes and how they can be used to attack both human interaction and automated systems. As the encoded information is intended to be machine readable only, a human cannot distinguish between a valid and a maliciously manipulated QR code. While humans might fall for phishing attacks, automated readers are most likely vulnerable to SQL injections and command injections. Our contribution consists of an analysis of the QR Code as an attack vector, showing different attack strategies from the attackers point of view and exploring their possible consequences.