• A novel LIC model uses a predictor that automatically determines realism levels • We propose a two-stage training framework for stable training of the LIC model • Optimal realism levels determine the quality of distortion and perception of the image • We achieved significant results in perceptual metrics Learned image compression (LIC) methods have demonstrated high-quality reconstruction, and even surpassed the traditional codec algorithms at rate-distortion performance. However, reconstructing natural and realistic textures remains a challenge. The main problem is the distortion-realism trade-off. Realism deteriorates quickly if we minimize distortion, and vice versa. The existing methods manually determine and apply a realism level, which is a degree of the trade-off, to an entire image, resulting in time-consuming training and suboptimal quality. In this paper, we propose a novel LIC model equipped with a predictor that automatically determines realism levels for regions of the image. Thus, we reconstruct images using the optimal realism levels. Moreover, we developed a two-stage training framework for stable training of the proposed model. The experimental results show that the proposed model achieved significant perceptual quality while the PSNR is comparable to the existing methods.

Recent efforts have explored leveraging visible light images to enrich texture details in infrared (IR) super-resolution. However, this direct adaptation approach often becomes a double-edged sword, as it improves texture at the cost of introducing noise and blurring artifacts. Such imperfections are inherent in the spatial domain of visible images and are accentuated during the imaging process. Enhancing IR image quality by integrating rich texture details from visible images, while minimizing noise transfer, presents a challenging research avenue. To address these challenges, we propose the Texture and Noise Dual Adaptation SRGAN (DASRGAN), an innovative framework specifically engineered for robust IR super-resolution model adaptation. DASRGAN operates on the synergy of two key components: (1) Texture-Oriented Adaptation (TOA) to refine texture details meticulously, and (2) Noise-Oriented Adaptation (NOA), dedicated to minimizing noise transfer. Specifically, TOA uniquely integrates a specialized discriminator, incorporating a prior extraction branch, and employs a Sobel-guided adversarial loss to align texture distributions effectively. Concurrently, NOA utilizes a noise adversarial loss to distinctly separate the generative and Gaussian noise pattern distributions during adversarial training. Our extensive experiments confirm DASRGAN’s superiority. Comparative analyses against leading methods across multiple benchmarks and upsampling factors reveal that DASRGAN sets new state-of-the-art performance standards. Code are available at https://github.com/yongsongH/DASRGAN . • DASRGAN improves infrared image super-resolution using visible textures and noise control through dual adaptations. • Texture-Oriented Adaptation uses Sobel-based discriminator and texture alignment loss for detail enhancement. • Noise-Oriented Adaptation employs domain-specific loss to suppress noise propagation between modalities. • DASRGAN achieves state-of-the-art performance in infrared super-resolution benchmarks across metrics.

Infrared image super-resolution (IRSR) is challenging due to weak structures and textures. While Mamba-based state-space models (SSMs) efficiently model long-range dependencies, their inherent block-wise processing disrupts spatial consistency, limiting direct IRSR applicability. We propose IRSRMamba, a novel Mamba-based framework overcoming this limitation via tailored structural and textural preservation. Integrated into a Mamba backbone, our key innovations are: 1) Wavelet Transform Feature Modulation (WTFM), enhancing multi-scale frequency-aware feature extraction to mitigate block-induced coherence loss; and 2) an SSMs-based Semantic Consistency Loss, enforcing cross-block alignment to restore fragmented context. IRSRMamba achieves superior global-local fusion, structural coherence, and fine-detail preservation. Experiments show state-of-the-art PSNR, SSIM, and perceptual quality on IR benchmarks, as well as robust generalization to remote sensing. This work establishes Mamba-based architectures as highly promising for high-fidelity IR image enhancement. Code is available at https://github.com/yongsongH/IRSRMamba.

In recent years, neural network-driven image compression (NIC) has gained significant attention. Some works adopt deep generative models such as GANs and diffusion models to enhance perceptual quality (realism). A critical obstacle of these generative NIC methods is that each model is optimized for a single bit rate. Consequently, multiple models are required to compress images to different bit rates, which is impractical for real-world applications. To tackle this issue, we propose a variable-rate generative NIC model. Specifically, we explore several discriminator designs tailored for the variable-rate approach and introduce a novel adversarial loss. Moreover, by incorporating the newly proposed multi-realism technique, our method allows the users to adjust the bit rate, distortion, and realism with a single model, achieving ultra-controllability. Unlike existing variable-rate generative NIC models, our method matches or surpasses the performance of state-of-the-art single-rate generative NIC models while covering a wide range of bit rates using just one model.

Japanese historical documents provide valuable information. Character recognition is a critical technology for the digitalization of historical documents. Sample imbalance is a significant obstacle in recognizing Japanese historical characters, kuzushiji. Thousands of kuzushiji only have less than a few samples. Thus, recognition performance deteriorates greatly in kuzushiji with a few samples. In this study, we propose a framework for transferring knowledge of character parts from font to kuzushiji. The pretraining learns character parts from synthesized font images. However, fine-tuning to kuzushiji is more complex. We propose calculating a mean squared error loss between feature vectors of kuzushiji and font images, resulting in consistent feature vectors in kuzushiji and font. Consequently, we can perform zero-shot recognition for kuzushiji using the font images of zero-sampled kuzushiji. The experimental results show that the proposed method recognized zero-sampled kuzushiji at approximately 48% accuracy. Consequently, we significantly expand the number of recognizable kuzushiji.

Scene-text image synthesis techniques that aim to naturally compose text instances on background scene images are very appealing for training deep neural networks due to their ability to provide accurate and comprehensive annotation information. Prior studies have explored generating synthetic text images on two-dimensional and three-dimensional surfaces using rules derived from real-world observations. Some of these studies have proposed generating scene-text images through learning; however, owing to the absence of a suitable training dataset, unsupervised frameworks have been explored to learn from existing real-world data, which might not yield reliable performance. To ease this dilemma and facilitate research on learning-based scene text synthesis, we introduce DecompST, a real-world dataset prepared from some public benchmarks, containing three types of annotations: quadrilateral-level BBoxes, stroke-level text masks, and text-erased images. Leveraging the DecompST dataset, we propose a Learning-Based Text Synthesis engine (LBTS) that includes a text location proposal network (TLPNet) and a text appearance adaptation network (TAANet). TLPNet first predicts the suitable regions for text embedding, after which TAANet adaptively adjusts the geometry and color of the text instance to match the background context. After training, those networks can be integrated and utilized to generate the synthetic dataset for scene text analysis tasks. Comprehensive experiments were conducted to validate the effectiveness of the proposed LBTS along with existing methods, and the experimental results indicate the proposed LBTS can generate better pretraining data for scene text detectors. Our dataset and code are made available at: https://github.com/iiclab/DecompST.

Scene text erasing, which replaces text regions with reasonable content in natural images, has drawn significant attention in the computer vision community in recent years. There are two potential subtasks in scene text erasing: text detection and image inpainting. Both subtasks require considerable data to achieve better performance; however, the lack of a large-scale real-world scene-text removal dataset does not allow existing methods to realize their potential. To compensate for the lack of pairwise real-world data, we made considerable use of synthetic text after additional enhancement and subsequently trained our model only on the dataset generated by the improved synthetic text engine. Our proposed network contains a stroke mask prediction module and background inpainting module that can extract the text stroke as a relatively small hole from the cropped text image to maintain more background content for better inpainting results. This model can partially erase text instances in a scene image with a bounding box or work with an existing scene-text detector for automatic scene text erasing. The experimental results from the qualitative and quantitative evaluation on the SCUT-Syn, ICDAR2013, and SCUT-EnsText datasets demonstrate that our method significantly outperforms existing state-of-the-art methods even when they are trained on real-world data.

We propose a GAN-based image compression method working at extremely low bitrates below 0.1bpp. Most existing learned image compression methods suffer from blur at extremely low bitrates. Although GAN can help to reconstruct sharp images, there are two drawbacks. First, GAN makes training unstable. Second, the reconstructions often contain unpleasing noise or artifacts. To address both of the drawbacks, our method adopts two-stage training and network interpolation. The two-stage training is effective to stabilize the training. Moreover, the network interpolation utilizes the models in both stages and reduces undesirable noise and artifacts, while maintaining important edges. Hence, we can control the trade-off between perceptual quality and fidelity without re-training models. The experimental results show that our model can reconstruct high quality images. Furthermore, our user study confirms that our reconstructions are preferable to state-of-the-art GAN-based image compression model. Our source code is available at https://github.com/iwa-shi/fidelity_controllable_compression.

Blue and chub mackerels are often caught at the same time, and their market prices are different. Thus, humans need to classify them by their hands. Therefore, the demand for automatic sorting machines of these mackerels using image processing technique is increasing. Classification for blue and chub mackerels is a challenging task due to their quite similar appearance. In this paper, we propose a method for classifying blue and chub mackerels by image processing technique. The difference of the two mackerels appears on the whole and the local specific parts. Therefore, we use global and local features that are extracted from the whole and the specific part in fish images, respectively. Experimental results showed that the proposed method is superior to the methods using only global or local feature.

In this paper, we address a problem of data augmentation for character recognition. Particularly, we focus on incorporating variation in glyph into data augmentation of character images, which is a simple approach for data augmentation. Generally, existing methods increase data size by distorting images, whereas the proposed method applies noise injection into glyphs, resulting in data with radical variation in glyph. The proposed method exploits public database of glyphs for kanji and augments glyphs by injecting noise into glyphs. Then, we generate images of kanji automatically by deploying stroke images on the augmented glyphs. We carried out experiments for kanji character recognition using augmented data. The results show the effectiveness of the proposed method.

Various indoor navigation methods have been developed recently, but digitalized data of indoor map is not always available. Therefore, an indoor navigation framework using an image of information board has been proposed. In this method, the process to extract map regions from the image of an information board is necessary to be done by hands beforehand, and the process to estimate passageway regions is important because its information is used in map matching. However, the method of passageway discrimination is very heuristic, which is intended for a specific type of floor maps. Therefore, in this paper, we propose a semi-automatic method to extract map regions from the image of information board with simple user's operation. We use GrabCut method and Snakes method for the extraction method. In GrabCut method, we detect closed regions to prevent the degradation of accuracy when conducting GrabCut to the downsizing image. The proposed method can extract a map region with few deficits in short calculation time. In addition, we propose a machine learning based method to classify passageway regions and other regions from a segment image. We confirmed that the proposed methods are effective and promising by experiments.

This paper presents a novel method for structural data recognition using a large number of graph models. Broadly, existing methods for structural data recognition have two crucial problems: 1) only a single model is used to capture structural variation, 2) naive classification rules are used, such as nearest neighbor method. In this paper, we propose to strengthen both capturing structural variation and the classification ability. The proposed method constructs a large number of graph models and trains decision tree classifiers with the models. There are two contributions of this paper. The first contribution is a novel graph model which can be constructed by straightforward calculation. This calculation enables us to construct many models in feasible time. The second contribution is a novel approach to capture structural variation. We construct a large number of our models in a boosting framework so that we can capture structural variation comprehensively. Consequently, we are able to perform structural data recognition with the powerful classification ability and comprehensive structural variation. In experiments, we show that the proposed method achieves significant results and outperforms the existing methods.