Large Language Models (LLMs) are increasingly deployed within agentic systems—collections of interacting, LLM-powered agents that execute complex, adaptive workflows using memory, tools, and dynamic planning. While enabling powerful new capabilities, these systems also introduce unique forms of uncertainty stemming from probabilistic reasoning, evolving memory states, and fluid execution paths. Traditional software observability and operations practices fall short in addressing these challenges.This paper presents our vision of AgentOps: a comprehensive framework for observing, analyzing, optimizing, and automating operation of agentic AI systems. We identify distinct needs across four key roles—developers, testers, site reliability engineers (SREs), and business users—each of whom engages with the system at different points in its lifecycle. We present the AgentOps Automation Pipeline, a six-stage process encompassing behavior observation, metric collection, issue detection, root cause analysis, optimized recommendations, and runtime automation. Throughout, we emphasize the critical role of automation in managing uncertainty and enabling self-improving AI systems—not by eliminating uncertainty, but by taming it to ensure safe, adaptive, and effective operation.

Blockchain databases were designed to improve trust in centralized ecosystems, which dominate the market today, by introducing tamper-evidence features on top of a classical database. Compared to decentralized ledger technologies, blockchain databases are easier to use, and they can significantly reduce the operational and development costs. However, the existing blockchain databases do not equip multiple parties with tools to efficiently control common data written to the ledger. This paper describes Orion, a new open source blockchain database that introduces multi-sig and proof capabilities with extensive key-level access control, which enable parties to mutually control and validate a value written to the database. These unique capabilities, together with additional blockchain properties, provide users with features such as tamper-evidence, provenance, data lineage, authenticity, and non-repudiation, while using a standard data model and transactional APIs. We found our technology to be extremely useful in improving the integrity of a system and reducing mistakes, disputes, and fraud.

Software developers nowadays not infrequently face a challenge of solving problems that essentially sum up to finding a sequence of deterministic actions leading from a given initial state to a goal. This is the problem of deterministic planning, one of the most basic and well studied problems in artificial intelligence. Two of the best known approaches to deterministic planning are the black box approach, in which a programmer implements a successor generator, and the model-based approach, in which a user describes the problem symbolically, e.g., in PDDL. While the black box approach is usually easier for programmers who are not experts in AI to understand, it does not scale up without informative heuristics. We propose an approach that we baptize as semi-black box (SBB) that combines the strength of both. SBB is implemented as a set of Java classes, which a programmer can inherit from when implementing a successor generator. Using the known characteristics of these classes, we then automatically derive heuristics for the problem. Our empirical evaluation shows that these heuristics allow the planner to scale up significantly better than the traditional black box approach.

Software metrics are easy to define, but not so easy to justify. It is hard to prove that a metric is valid, i.e., that measured numerical values imply anything on the vaguely defined, yet crucial software properties such as complexity and maintainability. This paper employs statistical analysis and tests to check some plausible assumptions on the behavior of software and metrics measured for this software in retrospective on its versions evolution history. Among those are the reliability assumption implicit in the application of any code metric, and the assumption that the magnitude of change, i.e., increase or decrease of its size, in a software artifact is correlated with changes to its version number. Putting a suite of 36 metrics to the trial, we confirm most of the assumptions on a large repository of software artifacts. Surprisingly, we show that a substantial portion of the reliability of some metrics can be observed even in random changes to architecture. Another surprising result is that Boolean-valued metrics tend to flip their values more often in minor software version increments than in major increments.

Reverse engineering is an essential part of the modernization process that enables the evolution of existing software assets. The extraction of state machines out of existing code is an important aspect of the reverse engineering process. However, none of the reverse engineering tools fully support an automatic extraction of state machines. In our work we investigated the process of manual extraction of hierarchical state machines from the source code of an embedded C application and identified the steps of the process that can be automated. We learned that manual creation of state machines out of code is a very complicated task mostly because of the large amount of potential states that can be created by a relatively small amount of global variables. To reduce the complexity of this task we developed a methodology to decompose the code into smaller parts of functionally related elements. We showed how this technique and other system analysis mechanisms provided by the System Grokking technology can automate steps of the state machine extraction process.