Domain Generalization (DG) aims to learn from multiple known source domains a model that can generalize well to unknown target domains. One of the key approaches in DG is training an encoder which generates domain-invariant representations. However, this approach is not applicable in Federated Domain Generalization (FDG), where data from various domains are distributed across different clients. In this paper, we introduce a novel approach, dubbed Federated Learning via On-server Matching Gradient (FedOMG), which can efficiently leverage domain information from distributed domains. Specifically, we utilize the local gradients as information about the distributed models to find an invariant gradient direction across all domains through gradient inner product maximization. The advantages are two-fold: 1) FedOMG can aggregate the characteristics of distributed models on the centralized server without incurring any additional communication cost, and 2) FedOMG is orthogonal to many existing FL/FDG methods, allowing for additional performance improvements by being seamlessly integrated with them. Extensive experimental evaluations on various settings demonstrate the robustness of FedOMG compared to other FL/FDG baselines. Our method outperforms recent SOTA baselines on four FL benchmark datasets (MNIST, EMNIST, CIFAR-10, and CIFAR-100), and three FDG benchmark datasets (PACS, VLCS, and OfficeHome). The reproducible code is publicly available~\footnote[1]{\url{https://github.com/skydvn/fedomg}}.

Weakly supervised visual grounding aims to predict the region in an image that corresponds to a specific linguistic query, where the mapping between the target object and query is unknown in the training stage. The state-of-the-art method uses a vision language pre-training model to acquire heatmaps from Grad-CAM, which matches every query word with an image region, and uses the combined heatmap to rank the region proposals. In this paper, we propose two simple but efficient methods for improving this approach. First, we propose a target-aware cropping approach to encourage the model to learn both object and scene level semantic representations. Second, we apply dependency parsing to extract words related to the target object, and then put emphasis on these words in the heatmap combination. Our method surpasses the previous SOTA methods on RefCOCO, RefCOCO+, and RefCOCOg by a notable margin.

This paper presents a patch-based approach for crowd density estimation in public scenes. We formulate the problem of estimating density in a structured learning framework applied to random decision forests. Our approach learns the mapping between patch features and relative locations of all objects inside each patch, which contribute to generate the patch density map through Gaussian kernel density estimation. We build the forest in a coarse-to-fine manner with two split node layers, and further propose a crowdedness prior and an effective forest reduction method to improve the estimation accuracy and speed. Moreover, we introduce a semi-automatic training method to learn the estimator for a specific scene. We achieved state-of-the-art results on the public Mall dataset and UCSD dataset, and also proposed two potential applications in traffic counts and scene understanding with promising results.

While online approaches for multi-person tracking are still hard to deal with occlusions, many tracking methods use a global data association approach to find person trajectories on a long segment of consecutive frames. Such offline approaches often cause long output latencies, which are unsuitable for camera-based surveillance systems. Based on an observation that the current position of an object should be estimated from both its past and future positions that are observed in a short period of time, we propose a novel object tracking method based on a dynamic programming framework. This method iteratively associates and integrates track lets obtained by visual tracking which are observed just before and after occlusions. We always produce outputs at a constant short delay time, and requires only the standard HOG detector for high performance. Our tracking method can run at 10-60 fps on a single CPU core, and attains the state-of-the-art performance for the Town Center dataset.

We propose an image segmentation method that divides an image into foreground and background regions when the approximate color distributions for these regions are given.Our approach was inspired by global consistency measures that directly evaluate the similarity between a given distribution and the distribution of the resulting segmentation, which were recently proposed in order to overcome the limitations of traditional pixelwise (local) consistency measures.The main feature of our proposal is that it uses two (foreground and background) input distributions, which increases the robustness compared to previous studies.To achieve this, we formulated a new mathematical model that describes the consistencies between the two input distributions and the segmentation, in which weighting parameters for the two distribution matching terms are set to be approximately proportional to the size of the foreground and background areas.We call this dual distribution matching (DDM).We also derived an optimization method that uses graph cuts.Experimental results that show the effectiveness of our method and comparisons between local and global consistency measures are presented.

This paper addresses the problem of image segmentation with a reference distribution. Recent studies have shown that segmentation with global consistency measures outperforms conventional techniques based on pixel-wise measures. However, such global approaches require a precise distribution to obtain the correct extraction. To overcome this strict assumption, we propose a new approach in which the given reference distribution plays a guiding role in inferring the latent distribution and its consistent region. The inference is based on an assumption that the latent distribution resembles the distribution of the consistent region but is distinct from the distribution of the complement region. We state the problem as the minimization of an energy function consisting of global similarities based on the Bhattacharyya distance and then implement a novel iterated distribution matching process for jointly optimizing distribution and segmentation. We evaluate the proposed algorithm on the GrabCut dataset, and demonstrate the advantages of using our approach with various segmentation problems, including interactive segmentation, background subtraction, and co-segmentation.

This paper addresses the problem of interactive image segmentation with a user-supplied object bounding box. The underlying problem is the classification of pixels into foreground and background, where only background information is provided with sample pixels. Many approaches treat appearance models as an unknown variable and optimize the segmentation and appearance alternatively, in an expectation maximization manner. In this paper, we describe a novel approach to this problem: the objective function is expressed purely in terms of the unknown segmentation and can be optimized using only one minimum cut calculation. We aim to optimize the trade-off of making the foreground layer as large as possible while keeping the similarity between the foreground and background layers as small as possible. This similarity is formulated using the similarities of distant pixel pairs. We evaluated our algorithm on the GrabCut dataset and demonstrated that high-quality segmentations were attained at a fast calculation speed.

In this paper, we have developed a robust object-based watermarking algorithm using the scale-invariant feature transform (SIFT) features in conjunction with a new data embedding method based on discrete cosine transform (DCT). The message is embedded in DCT spaces of randomly generated blocks in the selected object region. To recognize the object region after being distorted, its SIFT features are registered in advance. In the detection scheme, we firstly detect the object region by using feature matching. The transformation parameters are then calculated, and the message can be detected. Experimental results demonstrated that our proposed algorithm is very robust to geometrical distortions such as JPEG compression, scaling, rotation, shearing, aspect ratio change, image filtering, and so on.