The basic objective of my research work is to address the challenging problem of recognizing object states in a visual context by integrating data-driven and symbolic approaches. In particular, I focus on the Zero-shot variation of this task. The contributions made so far include the development of novel methods that exhibit state-of-the-art (SOTA) performance, the creation of a new object states dataset, the formulation of novel problems, the successful integration of low-level and high-level approaches, and comprehensive analyses that highlight the specific challenges posed by the problem.

Object State Classification (OSC) is a critical task in computer vision, enabling systems to understand the functional state of objects. This work proposes a novel end-to-end architecture for OSC that leverages the inherent relationship between object classification and state recognition. Our approach first classifies the object and then uses object-specific attention mechanisms to focus on relevant features for state classification. This two-stage design allows the model to effectively capture object-state dependencies while maintaining modularity and flexibility. We conduct an extensive ablation study to analyze the impact of key parameters, such as attention mechanisms and loss weighting, and evaluate our method against three baselines across four benchmark datasets. Experimental results demonstrate that our approach outperforms competing methods by a significant margin, achieving state-of-the-art performance.

We investigate the problem of Object State Classification (OSC) in the context of zero-shot learning. Specifically, we propose the first method for Zero-shot Object-agnostic State Classification (OaSC) that, given an image, infers the state of a single object without relying on the knowledge or the estimation of the object class. In that direction, we capitalize on Knowledge Graphs (KGs) for structuring and organizing external knowledge, which, in combination with visual information, enable effective inference of the states of objects that have not been encountered in the training set. Having this unique property, a significant strength of our method is that it can handle an Open Set of object classes. We investigate the performance of OaSC in various datasets and settings, against several hypotheses and in comparison with state-of-the-art approaches for object attribute classification. OaSC outperforms these methods significantly across all benchmarks.1