Realizing the vision of using AI agents to automate critical IT tasks depends on the ability to measure and understand effectiveness of proposed solutions. We introduce ITBench, a framework that offers a systematic methodology for benchmarking AI agents to address real-world IT automation tasks. Our initial release targets three key areas: Site Reliability Engineering (SRE), Compliance and Security Operations (CISO), and Financial Operations (FinOps). The design enables AI researchers to understand the challenges and opportunities of AI agents for IT automation with push-button workflows and interpretable metrics. IT-Bench includes an initial set of 102 real-world scenarios, which can be easily extended by community contributions. Our results show that agents powered by state-of-the-art models resolve only 11.4% of SRE scenarios, 25.2% of CISO scenarios, and 25.8% of FinOps scenarios (excluding anomaly detection). For FinOps-specific anomaly detection (AD) scenarios, AI agents achieve an F1 score of 0.35. We expect ITBench to be a key enabler of AI-driven IT automation that is correct, safe, and fast. IT-Bench, along with a leaderboard and sample agent implementations, is available at https://github.com/ibm/itbench.

By 2027, over 50 % of enterprises are expected to adopt industry cloud platforms [1], driving potential EBITDA value of $3 trillion by 2030 [2]. In this landscape, software providers rely on Infrastructure-as-a-Service (IaaS) providers to access tailored virtualized resources based on usage. Optimizing resource utilization is crucial to reducing operating costs and maintaining quality standards for SaaS and IaaS providers. This creates an essential need for dynamic scaling mechanisms to adjust resources according to workload variations. The Kubernetes resource Horizontal Pod Autoscaler (HPA) has limitations in scaling applications. However, AI-based algorithms, particularly Reinforcement Learning (RL), offer promising solutions. AI-based methods excel in overcoming fixed parameter constraints, handling sudden load spikes, and supporting custom parameters. We present an RL-based framework for auto scaling applications, demonstrating results from experimental evaluation.

Tagging of faces present in a photo or video at shot level has multiple applications related to indexing and retrieval. Face clustering, which aims to group similar faces corresponding to an individual, is a fundamental step of face tagging. We present a progressive method of applying easy-to-hard grouping technique that applies increasingly sophisticated feature descriptors and classifiers on reducing number of faces from each of the iteratively generated clusters. Our primary goal is to design a cost effective solution for deploying it on low-power devices like mobile phones. First, the method initiates the clustering process by applying K-Means technique with relatively large K value on simple LBP features to generate the first set of high precision clusters. Multiple clusters generated for each individual (low recall) are then progressively merged by applying linear and non-linear subspace modelling strategies on custom selected sophisticated features like Gabor filter, Gabor Jets, and Spin LGBP (Local Gabor Binary Patterns) with spatially spinning bin support for histogram computation. Our experiments on the standard face databases like YouTube Faces, YouTube Celebrities, Indian Movie Face database, eNTERFACE, Multi-Pie, CK+, MindReading and internally collected mobile phone samples demonstrate the effectiveness of proposed approach as compared to state-of-the-art methods and a commercial solution on a mobile phone.

Facial expression recognition is one of the open problems in computer vision. Robust neutral face recognition in real time is a major challenge for various supervised learning-based facial expression recognition methods. This is due to the fact that supervised methods cannot accommodate all appearance variability across the faces with respect to race, pose, lighting, facial biases, and so on, in the limited amount of training data. Moreover, processing each and every frame to classify emotions is not required, as user stays neutral for majority of the time in usual applications like video chat or photo album/web browsing. Detecting neutral state at an early stage, thereby bypassing those frames from emotion classification would save the computational power. In this paper, we propose a light-weight neutral versus emotion classification engine, which acts as a pre-processer to the traditional supervised emotion classification approaches. It dynamically learns neutral appearance at key emotion (KE) points using a statistical texture model, constructed by a set of reference neutral frames for each user. The proposed method is made robust to various types of user head motions by accounting for affine distortions based on a statistical texture model. Robustness to dynamic shift of KE points is achieved by evaluating the similarities on a subset of neighborhood patches around each KE point using the prior information regarding the directionality of specific facial action units acting on the respective KE point. The proposed method, as a result, improves emotion recognition (ER) accuracy and simultaneously reduces computational complexity of the ER system, as validated on multiple databases.

In a Facial Expression Recognition (FER) system, appropriate representation of facial features from relevant face regions play crucial role in robust detection of facial actions units (AUs) under realistic conditions like wide range of illumination variations, presence of tracking errors, inter person expression variations, partial occlusion of faces, etc. In this work, we perform an in-depth analysis of state-of-the-art FER techniques to further understand their performance gaps under realistic conditions. We propose an appropriate Region Of Interest (ROI) selection strategy for each AU and also an appropriately designed robust Local Binary Pattern (LBP) based descriptor that applies spatially spinning bin support for histogram computation. We show that the proposed solutions are capable of addressing performance gaps seen in existing approaches. The ROI strategy present here gives a better trade-off in eliminating inter AU correlations while modeling AUs and minimizes the constraints on the accuracy of facial feature localization. The proposed spin support based feature descriptor provides unique representation for AUs by encoding both appearance and geometry of the facial features in its description and results in a better detection accuracy. We compare the performance of the proposed solutions with the key state-of-the-art techniques and show clear improvement on benchmark databases like CK+, ISL, FACS, JAFFE, MultiPie, MindReading and also on an internally collected real-world data.