Understanding and reasoning over text within visual contexts poses a significant challenge for Vision-Language Models (VLMs), given the complexity and diversity of real-world scenarios.To address this challenge, text-rich Visual Question Answering (VQA) datasets and benchmarks have emerged for high-resource languages like English.However, a critical gap persists for low-resource languages such as Korean, where the lack of comprehensive benchmarks hinders robust model evaluation and comparison.To bridge this gap, we introduce KRETA, a benchmark for Korean Reading and rEasoning in Text-rich VQA Attuned to diverse visual contexts.KRETA facilitates an in-depth evaluation of both visual text understanding and reasoning capabilities, while also supporting a multifaceted assessment across 15 domains and 26 image types.Additionally, we introduce a semiautomated VQA generation pipeline specifically optimized for text-rich settings, leveraging refined stepwise image decomposition and a rigorous seven-metric evaluation protocol to ensure data quality.While KRETA is tailored for Korean, we hope our adaptable and extensible pipeline will facilitate the development of similar benchmarks in other languages, thereby accelerating multilingual VLM research.The code and dataset for KRETA are available at https://github.com/tabtoyou/KRETA.

Online action detection, which aims to identify an ongoing action from a streaming video, is an important subject in real-world applications. For this task, previous methods use recurrent neural networks for modeling temporal relations in an input sequence. However, these methods overlook the fact that the input image sequence includes not only the action of interest but background and irrelevant actions. This would induce recurrent units to accumulate unnecessary information for encoding features on the action of interest. To overcome this problem, we propose a novel recurrent unit, named Information Discrimination Unit (IDU), which explicitly discriminates the information relevancy between an ongoing action and others to decide whether to accumulate the input information. This enables learning more discriminative representations for identifying an ongoing action. In this paper, we further present a new recurrent unit, called Information Integration Unit (IIU), for action anticipation. Our IIU exploits the outputs from IDN as pseudo action labels as well as RGB frames to learn enriched features of observed actions effectively. In experiments on TVSeries and THUMOS-14, the proposed methods outperform state-of-the-art methods by a significant margin in online action detection and action anticipation. Moreover, we demonstrate the effectiveness of the proposed units by conducting comprehensive ablation studies.

From a streaming video, online action detection aims to identify actions in the present. For this task, previous methods use recurrent networks to model the temporal sequence of current action frames. However, these methods overlook the fact that an input image sequence includes background and irrelevant actions as well as the action of interest. For online action detection, in this paper, we propose a novel recurrent unit to explicitly discriminate the information relevant to an ongoing action from others. Our unit, named Information Discrimination Unit (IDU), decides whether to accumulate input information based on its relevance to the current action. This enables our recurrent network with IDU to learn a more discriminative representation for identifying ongoing actions. In experiments on two benchmark datasets, TVSeries and THUMOS-14, the proposed method outperforms state-of-the-art methods by a significant margin. Moreover, we demonstrate the effectiveness of our recurrent unit by conducting comprehensive ablation studies.

Recent temporal action proposal generation approaches have suggested integrating segment- and snippet score-based methodologies to produce proposals with high recall and accurate boundaries. In this paper, different from such a hybrid strategy, we focus on the potential of the snippet score-based approach. Specifically, we propose a new snippet score-based method, named Snippet Relatedness-based Generator (SRG), with a novel concept of “snippet relatedness”. Snippet relatedness represents which snippets are related to a specific action instance. To effectively learn this snippet relatedness, we present “pyramid non-local operations” for locally and globally capturing long-range dependencies among snippets. By employing these components, SRG first produces a 2D relatedness score map that enables the generation of various temporal intervals reliably covering most action instances with high overlap. Then, SRG evaluates the action confidence scores of these temporal intervals and refines their boundaries to obtain temporal action proposals. On THUMOS-14 and ActivityNet-1.3 datasets, SRG outperforms state-of-the-art methods for temporal action proposal generation. Furthermore, compared to competing proposal generators, SRG leads to significant improvements in temporal action detection.

In this paper, we present a novel grid encoding model for content-aware image retargeting. In contrast to previous approaches such as vertex-based and axis-aligned grid encoding models, our approach takes each horizontal/vertical distance between two adjacent vertices as an optimization variable. Upon this difference-based encoding scheme, every vertex position of a target grid is subsequently determined after optimizing the one-dimensional values. Our quad edge-based grid model has two major advantages for image retargeting. First, the model enables a grid optimization problem to be developed in a simple quadratic program while ensuring the global convexity of objective functions. Second, due to the independency of variables, spatial regularizations can be applied in a locally adaptive manner to preserve structural components. Based on this model, we propose three quadratic objective functions. Note that, in our work, their linear combination guides a grid deformation process to obtain a visually comfortable retargeting result by preserving salient regions and structural components of an input image. Comparative evaluations have been conducted with ten existing state-of-the-art image retargeting methods, and the results show that our method built upon the quad edge-based model consistently outperforms other previous methods both on qualitative and quantitative perspectives.

In this paper, we propose a fast method based on deep neural networks to detect and recognize Korean characters in traffic guide signs. To detect character candidates quickly, we first employ a region proposal network (RPN) which is in this paper ResNet-18, being relatively shallow. We also apply the Inception architecture to residual blocks for reducing parameters of the network. After character candidates are detected, we classify them into 709 Korean characters by using a classification network (CLSN). Similar to the RPN, our CLSN consists of residual blocks with the Inception architecture. In experiments, we achieved 97.69 % of accuracy at 5.9fps on both detection and recognition of Korean characters in traffic guide signs.

In edge-preserving image smoothing, edge blurriness and structural edge attenuation have been common problems. L0smoothing successfully solves these two problems by adopting L0norm of gradients. However, a weak structural edge diminishing problem still exists because L0penalty first removes small nonzero gradients. In order to address this problem, we propose superpixel-guided adaptive image smoothing by introducing an adaptive parameter into L0smoothing framework. The adaptive smoothing parameter is efficiently computed in a cascade manner. In the first stage, we allocate smoothing parameters to the pixels consisting of details. More importantly, we then exploit similarities between a pixel and its surrounding superpixels for assigning smoothing parameters to the rest pixels. Experimental results demonstrate that our proposed method efficiently preserves structural edges regardless of their scales compared to previous methods.

Recent salient object detection algorithms often involve a segmentation step to produce saliency maps preserving boundaries. However, over-segmented results that many segmentation methods produce confuse to describe object boundaries. In this paper, we present a novel salient object detection algorithm which produces reliable salient object candidates. First, the input image is processed by Higher Order Statistics (HOS) based L0smoothing to highlight strong edges and reduce texture. We then apply image segmentation to the HOS L0smoothed image to produce improved results in which the number of over-segmented regions is greatly reduced. Second, we propose shape-aware region merging with a novel region scale measure. Finally, a saliency map from the merging result is generated by taking two simple saliency cues. Extensive experiments on a challenging saliency dataset indicate that our algorithm has comparable performance against state-of-the-arts.