Image editing using generative models has recently advanced through GAN- and diffusion-based techniques. While the current image manipulation methods shows considerable performance in general image editing, their effectiveness drops when extended to fashion attribute editing. This is due to multiple challenges such as category-specific and overlapping attributes, inherent entanglement in real-world datasets that leads to degraded editing quality and unintended attribute shifts. To address these challenges, we propose FAE-Net (Fashion Attribute Editing Network), a latent diffusion framework that leverages disentangled latent projections for precise and reliable attribute manipulation. Our method first disentangles the latent projections to mitigate the inherent entanglement in the data and then conditions the diffusion model with those projections to improve the manipulation control in the presence of overlapping attributes. The attribute presence detector in FAE-Net handles category-specific attributes and prevents invalid attribute manipulations during inference. Extensive experiments on three large-scale datasets demonstrate that our proposed method achieves more controllable, disentangled, and faithful attribute editing compared to state-of-the-art methods.

Automatic fashion attribute recognition enables retailers to address an array of applications. Usually, fashion attributes are manually input in the system by retailers, which is a time-consuming and an error prone process. To alleviate this, several existing works use traditional CNN-based backbones to recognize attributes. These backbones generate attribute embeddings that are entangled in the feature space. Existing methods that generate disentangled attribute embedding do not explicitly specify the location of attributes, and often extract features from irrelevant regions. This directly impacts the quality of downstream tasks. To alleviate this problem, we have proposed a novel framework to extract location-aware attribute representation using localization maps created from fashion landmarks. These localization maps highlight regions of interest in an image, aiding localized attribute feature extraction. Moreover, we have proposed a novel fusion module to effectively select important features from the global representation of an image to enhance the local features of the attribute. These attribute embeddings are then used in downstream applications such as attribute recognition, hierarchical taxonomy classification, and retrieval with two large-scale datasets. Using the proposed model, we observe improvement in performance from the state-of-the-art by a significant margin.

Recommendation systems improve users' online shopping experience by recommending relevant items from a large pool of items in different categories. Fashion recommendation systems apart from recommending individual fashion items also recommend fashion outfits. In this work, we consider the problem of the outfit compatibility prediction task, an integral part of the fashion outfit recommendation system. A compatibility prediction module determines whether all the items in an outfit are visually compatible with each other and match the user's preferences. Existing approaches can be grouped based on the representation scheme: (i) pair-wise and (ii) set or sequence. Pair-wise representation does not consider the outfit as a whole, and the sequence representation approaches are sensitive to the ordering of the items. Further, these methods do not explicitly capture the visual relationship between the items. We propose a novel method for the personalized outfit-compatible prediction task. The proposed method represents the outfit as a graph and uses a dot-attention graph neural network to capture the visual relationship between items. The graph read-out layer generates the final outfit embedding. A novel approach is proposed to model the user's preference for different styles. The final outfit compatibility score is generated by computing the similarity between outfit embedding and user embedding. Experimental results and ablation study on the Polyvore-U dataset, highlight the effectiveness of the proposed method.

Scene graph captures rich semantic information of an image by representing objects and their relationships as nodes and edges of a graph. Recent works have demonstrated that scene graph representation improves the performance of various computer vision tasks such as image retrieval, action recognition, visual question answering. Computationally efficient scene graph generation methods are required to leverage scene graphs in various real-world applications (e.g., autonomous driving, robotics). A typical scene graph generation model consists of two modules: (i) object detector and (ii) scene graph classifier. The scene graph classifier module predicts the object category and object-object relationships. The presence of a quadratic number of potential edges poses a major challenge in the scene graph classification task. Detecting the relationship between each object pair using the traditional approach is computationally intensive and non-scalable. To address this issue, we propose a novel module named EdgeNet that directly predicts the set of relevant edges and helps to prune out a significant number of unrelated object pairs, thereby improving the effectiveness and efficiency of the scene graph classifier. The proposed EdgeNet is a generic module and can be plugged into an existing scene graph classifier. Experimental results highlight the effectiveness and efficiency of the proposed approach on the Visual Genome dataset.

As humans, we inherently perceive images based on their predominant features, and ignore noise embedded within lower bit planes. On the contrary, Deep Neural Networks are known to confidently misclassify images corrupted with meticulously crafted perturbations that are nearly imperceptible to the human eye. In this work, we attempt to address this problem by training networks to form coarse impressions based on the information in higher bit planes, and use the lower bit planes only to refine their prediction. We demonstrate that, by imposing consistency on the representations learned across differently quantized images, the adversarial robustness of networks improves significantly when compared to a normally trained model. Present state-of-the-art defenses against adversarial attacks require the networks to be explicitly trained using adversarial samples that are computationally expensive to generate. While such methods that use adversarial training continue to achieve the best results, this work paves the way towards achieving robustness without having to explicitly train on adversarial samples. The proposed approach is therefore faster, and also closer to the natural learning process in humans.

Deep learning models have shown impressive performance across a spectrum of computer vision applications including medical diagnosis and autonomous driving. One of the major concerns that these models face is their susceptibility to adversarial attacks. Realizing the importance of this issue, more researchers are working towards developing robust models that are less affected by adversarial attacks. Adversarial training method shows promising results in this direction. In adversarial training regime, models are trained with mini-batches augmented with adversarial samples. Fast and simple methods (e.g., single-step gradient ascent) are used for generating adversarial samples, in order to reduce computational complexity. It is shown that models trained using single-step adversarial training method (adversarial samples are generated using non-iterative method) are pseudo robust. Further, this pseudo robustness of models is attributed to the gradient masking effect. However, existing works fail to explain when and why gradient masking effect occurs during single-step adversarial training. In this work, (i) we show that models trained using single-step adversarial training method learn to prevent the generation of single-step adversaries, and this is due to over-fitting of the model during the initial stages of training, and (ii) to mitigate this effect, we propose a single-step adversarial training method with dropout scheduling. Unlike models trained using existing single-step adversarial training methods, models trained using the proposed single-step adversarial training method are robust against both single-step and multi-step adversarial attacks, and the performance is on par with models trained using computationally expensive multi-step adversarial training methods, in white-box and black-box settings.