Human-Object Interaction (HOI) detection aims to localize human-object pairs and recognize their interactions astriplets. Recent advancements in pre-trained vision-language model (VLM) have improved zero-shot HOI detection, enabling identification of unseen triplets. However, existing methods leverage the VLM as an additional encoder only for interaction prediction, not for human/object detection. This limitation hinders their ability to detect unseen objects. Furthermore, the additional encoder increases both model size and computational cost. This paper proposes a novel HOI detection framework ECI-HOI, which unleashes potentials of the pre-trained VLM for the zero-shot HOI detection by leveraging it for both of the sub-tasks. We first employ CLIP as a single image encoder, reducing redundancy in the network architecture. In addition, we propose an instance selector and a HO pair decoder to effectively harmonize the human/object detection and the interaction prediction in zero-shot manner. We evaluate our model under various settings on HICO-DET and our two new testsets: out-of-distribution image testset and novel object testset. Our model outperforms the state-of-the-art models while reducing the model size by more than 50%, especially achieving a + 1 0.01 mAP improvement under the unseen object setting on HICO-DET. The results on the proposed datasets highlight the zero-shot performance of our model on more challenging settings.

Image acquisition in many biomedical imaging modalities is corrupted by Poisson noise followed by additive Gaussian noise. While total variation and related regularization methods for solving biomedical inverse problems are known to yield high quality reconstructions in most situations, such methods mostly use log-likelihood of either Gaussian or Poisson noise models, and rarely use mixed Poisson-Gaussian (PG) noise model. There is a recent work which deals with exact PG likelihood and total variation regularization. This method adapts the primal-dual approach involving gradients steps on the PG log-likelihood, with step size limited by the inverse of its Lipschitz constant. This leads to limitations in the convergence speed. Although the alternating direction method of multipliers (ADMM) algorithm does not have such step size restrictions, ADMM has never been applied for this problem, for the possible reason that PG log-likelihood is quite complex. In this paper, we develop an ADMM based optimization for roughness minimizing image restoration under PG log-likelihood. We achieve this by first developing a novel iterative method for computing the proximal solution of PG log-likelihood, deriving the termination conditions for this iterative method, and then integrating into a provably convergent ADMM scheme. We experimentally demonstrate that the proposed method outperform primal-dual method in most of the cases.

Total Variation (TV) and related extensions have been popular in image restoration due to their robust performance and wide applicability. While the original formulation is still relevant after two decades of extensive research, its extensions that combine derivatives of first and second orders are now being explored for better performance, with examples being Combined Order TV (COTV) and Total Generalized Variation (TGV). As an improvement over such multi-order convex formulations, we propose a novel non-convex regularization functional which adaptively combines Hessian-Schatten (HS) norm and first order TV (TV1) functionals with spatially varying weight. This adaptive weight itself is controlled by another regularization term; the total cost becomes the sum of this adaptively weighted HS-TV1 term, the regularization term for the adaptive weight, and the data-fitting term. The reconstruction is obtained by jointly minimizing w.r.t. the required image and the adaptive weight. We construct a block coordinate descent method for this minimization with proof of convergence, which alternates between minimization w.r.t. the required image and the adaptive weights. We derive exact computational formula for minimization w.r.t. the adaptive weight, and construct an ADMM algorithm for minimization w.r.t. to the required image. We compare the proposed method with existing regularization methods, and a recently proposed Deep GAN method using image recovery examples including MRI reconstruction and microscopy deconvolution.