In this work, we introduce Camouflage Anything, a novel and robust approach to generate camouflaged datasets. To the best of our knowledge, we are the first to apply Controlled Out-painting and Representation Engineering for generating realistic camouflaged images with an objective to hide any segmented object coming from a generic or salient database. Our proposed method uses a novel control design to out-paint a given segmented object, with a camouflaged background. We also uncover the role of representation engineering in enhancing the quality of generated camouflage datasets. We address the limitations of existing metrics FID and KID in capturing the "camouflage quality", by proposing a novel metric namely, CamOT. CamOT uses Optimal Transport between foreground & background Gaussian Mixture Models (GMM) of concerned camouflaged object to assign an image quality score. Furthermore, we conduct LoRA-based fine-tuning of the robust BiRefNet baseline with our generated camouflaged datasets, leading to notable improvements in camouflaged object segmentation accuracy. The experimental results showcase the efficacy and potential of Camouflage Anything, outperforming existing methods in camouflaged generation tasks.

Few Shot or Single Shot Segmentation of an object part is a challenging problem owing to various factors involving scarcity of part segmentation labels and varying scale and pose of the labeled object part (support-part) in relation to query object whole segment (query-whole). Prior attempts have been made to segment the query object part using additional language guidance with object part labels. Owing to lack of proper definition of an object part or lack of part label descriptions in many scenarios (e.g. an engine part), many real-world situations demand segmenting object parts with guidance from only a visual prompt. We propose to solve the problem of few shot part segmentation using only visual prompt by modeling it as a graph labeling problem. To this end, we introduce a novel super-pixel guided DiNOv2 feature based graph modeling for the whole and part of the support object (visual prompt) and the whole of the query object. We rely on an iterative graph partitioning strategy based on reverse guidance from query to support and converge to the most optimal query part segmentation. We showcase the efficacy of our proposed method in the most exhaustive part segmentation database, ADE20K-Part-234.

Deep hierarchical models for feature learning have emerged as an effective technique for object representation and classification in recent years. Though the features learnt using deep models have shown lot of promise towards achieving invariance to data transformations, this primarily comes at the expense of using much larger training data and model size. In the proposed work we devise a novel technique to incorporate rotation invariance, while training the deep model parameters. The convolution weight parameters in the network architecture are constrained to exhibit circular symmetry resulting in “rotation equivariance” of output feature maps. Rotation invariance is further achieved by max-pooling of the feature maps later in the hierarchy. We also show that by incorporating circular symmetry constraint into the training loss function, rotation invariance can be achieved with-in deep neural network framework with much lesser training data and model parameters. Our experiment results evaluated on rotated MNIST dataset further objectively validate the contribution.

Accurate segmentation of humans from live videos is an important problem to be solved in developing immersive video experience. We propose to extract the human segmentation information from color and depth cues in a video using multiple modeling techniques. The prior information from human skeleton data is also fused along with the depth and color models to obtain the final segmentation inside a graph-cut framework. The proposed method runs real time on live videos using single CPU and is shown to be quantitatively outperforming the methods that directly fuse color and depth data.

Conventional Image Retargeting methods aim to preserve the salient regions in an image using As Similar as Possible (ASAP) energy formulation or As Rigid as Possible (ARAP) energy formulation. ASAP energy formulation preserves the shape of the salient object while the scale of salient object can get distorted in the retargeted image. On the contrary, ARAP energy formulation preserves the scale of the salient object while the shape can get compromised. We propose a novel technique in which the necessity to preserve similarity and rigidity of the salient object is taken into consideration in a single energy formulation. The concept of object centric saliency is also introduced to improve the quality of image retargeting. The subjective and objective results on retargetMe database demonstrate the advantage of the proposed method than using ASAP or ARAP energy alone.

This paper proposes a novel approach for real-time video summarization on mobile using Dictionary Learning, Global Camera Motion analysis and Colorfulness. A dictionary is represented as a distinct set of events that are described as spatio-temporal features. Uniqueness measure is predicted based on the correlation scores of the dictionary elements whereas the quality measure is estimated using Global Camera Motion analysis and Colorfulness. Our proposed technique combines the uniqueness measure and the quality measure to predict the interestingness. Experiments indicate that our method outperforms state-of-the-arts in terms of computation speed while retaining the similar subjective quality.

We propose a method for interactive object segmentation using a single touch provided on the foreground object. The `extent' of the foreground object is estimated by a random walk technique designed on the salient edge representation of the image. The final image segmentation is performed under graph-cut framework. The accuracy of proposed method is demonstrated against the state-of the art interactive image segmentation techniques on GSC and BSD500 image datasets.

Facial expression recognition is one of the open problems in computer vision. Robust neutral face recognition in real time is a major challenge for various supervised learning-based facial expression recognition methods. This is due to the fact that supervised methods cannot accommodate all appearance variability across the faces with respect to race, pose, lighting, facial biases, and so on, in the limited amount of training data. Moreover, processing each and every frame to classify emotions is not required, as user stays neutral for majority of the time in usual applications like video chat or photo album/web browsing. Detecting neutral state at an early stage, thereby bypassing those frames from emotion classification would save the computational power. In this paper, we propose a light-weight neutral versus emotion classification engine, which acts as a pre-processer to the traditional supervised emotion classification approaches. It dynamically learns neutral appearance at key emotion (KE) points using a statistical texture model, constructed by a set of reference neutral frames for each user. The proposed method is made robust to various types of user head motions by accounting for affine distortions based on a statistical texture model. Robustness to dynamic shift of KE points is achieved by evaluating the similarities on a subset of neighborhood patches around each KE point using the prior information regarding the directionality of specific facial action units acting on the respective KE point. The proposed method, as a result, improves emotion recognition (ER) accuracy and simultaneously reduces computational complexity of the ER system, as validated on multiple databases.

In a Facial Expression Recognition (FER) system, appropriate representation of facial features from relevant face regions play crucial role in robust detection of facial actions units (AUs) under realistic conditions like wide range of illumination variations, presence of tracking errors, inter person expression variations, partial occlusion of faces, etc. In this work, we perform an in-depth analysis of state-of-the-art FER techniques to further understand their performance gaps under realistic conditions. We propose an appropriate Region Of Interest (ROI) selection strategy for each AU and also an appropriately designed robust Local Binary Pattern (LBP) based descriptor that applies spatially spinning bin support for histogram computation. We show that the proposed solutions are capable of addressing performance gaps seen in existing approaches. The ROI strategy present here gives a better trade-off in eliminating inter AU correlations while modeling AUs and minimizes the constraints on the accuracy of facial feature localization. The proposed spin support based feature descriptor provides unique representation for AUs by encoding both appearance and geometry of the facial features in its description and results in a better detection accuracy. We compare the performance of the proposed solutions with the key state-of-the-art techniques and show clear improvement on benchmark databases like CK+, ISL, FACS, JAFFE, MultiPie, MindReading and also on an internally collected real-world data.

As electronic gadgets become more user friendly, we find natural interaction with the gadgets becoming increasingly popular. Haptic devices have become immensely popular and touch and gesture based interfaces are the logical extension to natural user interaction. We propose to use facial expressions for natural interaction, especially for gadgets like smart phones and tablets where it is more appropriate to capture human faces for user interaction. We present a use-case scenario of an eBook Reader application wherein the user performs certain facial expressions naturally to control the device while using this application. We detect, recognize and interpret these facial expressions for natural interaction with the application. More broadly we want to fit such facial expression based natural interaction to the entire device, not limiting it to a particular application. As we move forward, devices will have to be personalized to comprehend subtle and implicit cues from the user instead of waiting for an explicit input.

Detection of salient motion in a video involves determining which motion is attended to by the human visual system in the presence of background motion that consists of complex visuals that are constantly changing. Salient motion is marked by its predictability compared to the more complex unpredictable motion of the background such as fluttering of leaves, ripples in water, dispersion of smoke, and others. We introduce a novel approach to detect salient motion based on the concept of “observability” from the output pixels, when the video sequence is represented as a linear dynamical system. The group of output pixels with maximum saliency is further used to model the holistic dynamics of the salient region. The pixel saliency map is bolstered by two region-based saliency maps, which are computed based on the similarity of dynamics of the different spatiotemporal patches in the video with the salient region dynamics, in a global as well as a local sense. The resulting algorithm is tested on a set of challenging sequences and compared to state-of-the-art methods to showcase its superior performance on grounds of its computational efficiency and ability to detect salient motion.

We formulate the problem of salient object detection in images as an automatic labeling problem on the vertices of a weighted graph. The seed (labeled) nodes are first detected using Markov random walks performed on two different graphs that represent the image. While the global properties of the image are computed from the random walk on a complete graph, the local properties are computed from a sparse k-regular graph. The most salient node is selected as the one which is globally most isolated but falls on a locally compact object. A few background nodes and salient nodes are further identified based upon the random walk based hitting time to the most salient node. The salient nodes and the background nodes will constitute the labeled nodes. A new graph representation of the image that represents the saliency between nodes more accurately, the "pop-out graph" model, is computed further based upon the knowledge of the labeled salient and background nodes. A semisupervised learning technique is used to determine the labels of the unlabeled nodes by optimizing a smoothness objective label function on the newly created "pop-out graph" model along with some weighted soft constraints on the labeled nodes.

We formulate the problem of salient region detection in images as Markov random walks performed on images represented as graphs. While the global properties of the image are extracted from the random walk on a complete graph, the local properties are extracted from a k-regular graph. The most salient node is selected as the one which is globally most isolated but falls on a compact object. The equilibrium hitting times of the ergodic Markov chain holds the key for identifying the most salient node. The background nodes which are farthest from the most salient node are also identified based on the hitting times calculated from the random walk. Finally, a seeded salient region identification mechanism is developed to identify the salient parts of the image. The robustness of the proposed algorithm is objectively demonstrated with experiments carried out on a large image database annotated with “ground-truth” salient regions.

We present a robust salient region detection framework based on the color and orientation distribution in images. The proposed framework consists of a color saliency framework and an orientation saliency framework. The color saliency framework detects salient regions based on the spatial distribution of the component colors in the image space and their remoteness in the color space. The dominant hues in the image are used to initialize an expectation-maximization (EM) algorithm to fit a Gaussian mixture model in the hue-saturation (H-S) space. The mixture of Gaussians framework in H-S space is used to compute the inter-cluster distance in the H-S domain as well as the relative spread among the corresponding colors in the spatial domain. Orientation saliency framework detects salient regions in images based on the global and local behavior of different orientations in the image. The oriented spectral information from the Fourier transform of the local patches in the image is used to obtain the local orientation histogram of the image. Salient regions are further detected by identifying spatially confined orientations and with the local patches that possess high orientation entropy contrast. The final saliency map is selected as either color saliency map or orientation saliency map by automatically identifying which of the maps leads to the correct identification of the salient region. The experiments are carried out on a large image database annotated with ldquoground-truthrdquo salient regions, provided by Microsoft Research Asia, which enables us to conduct robust objective level comparisons with other salient region detection algorithms.