Edge-based computer vision models running on compact, resource-limited devices benefit greatly from using unprocessed, detail-rich RAW sensor data instead of processed RGB images. Training these models, however, necessitates large labeled RAW datasets, which are costly and often impractical to obtain. Thus, converting existing la- beled RGB datasets into sensor-specific RAW images becomes crucial for effective model training. In this paper, we introduce ReRAW, an RGB-to-RAW conversion model that achieves state-of-the-art reconstruction performance across five diverse RAW datasets. This is accomplished through ReRAW’s novel multi-head architecture predicting RAW image candidates in gamma space. The performance is further boosted by a stratified sampling-based training data selection heuristic, which helps the model better reconstruct brighter RAW pixels. We finally demonstrate that pretraining compact models on a combination of high-quality synthetic RAW datasets (such as generated by ReRAW) and ground-truth RAW images for downstream tasks like object detection, outperforms both standard RGB pipelines, and RAW fine-tuning of RGB-pretrained models for the same task. The code is available at: https://github.com/ SonyResearch/ReRAW

Current deep learning approaches in computer vision primarily focus on RGB data sacrificing information. In contrast, RAW images offer richer representation, which is crucial for precise recognition, particularly in challenging conditions like low-light environments. The resultant demand for comprehensive RAW image datasets contrasts with the labor-intensive process of creating specific datasets for individual sensors. To address this, we propose a novel diffusion-based method for generating RAW images guided by RGB images. Our approach integrates an RGB-guidance module for feature extraction from RGB inputs, then incorporates these features into the reverse diffusion process with RGB-guided residual blocks across various resolutions. This approach yields high-fidelity RAW images, enabling the creation of camera-specific RAW datasets. Our RGB2RAW experiments on four DSLR datasets demonstrate state-of-the-art performance. Moreover, RAW-Diffusion demonstrates exceptional data efficiency, achieving remarkable performance with as few as 25 training samples or even fewer. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets, revealing its effectiveness for object detection in RAW imagery, significantly reducing the amount of required RAW images. The code is available at https://github.com/SonyResearch/RAW-Diffusion.

Image recognition models that work in challenging environments (e.g., extremely dark, blurry, or high dynamic range conditions) must be useful. However, creating training datasets for such environments is expensive and hard due to the difficulties of data collection and annotation. It is desirable if we could get a robust model without the need for hard-to-obtain datasets. One simple approach is to apply data augmentation such as color jitter and blur to standard RGB (sRGB) images in simple scenes. Unfortunately, this approach struggles to yield realistic images in terms of pixel intensity and noise distribution due to not considering the non-linearity of Image Signal Processors (ISPs) and noise characteristics of image sensors. Instead, we propose a noise-accounted RAW image augmentation method. In essence, color jitter and blur augmentation are applied to a RAW image before applying non-linear ISP, resulting in realistic intensity. Furthermore, we introduce a noise amount alignment method that calibrates the domain gap in the noise property caused by the augmentation. We show that our proposed noise-accounted RAW augmentation method doubles the image recognition accuracy in challenging environments only with simple training data.

Image Signal Processors (ISPs) play important roles in image recognition tasks as well as in the perceptual quality of captured images. In most cases, experts make a lot of effort to manually tune many parameters of ISPs, but the parameters are sub-optimal. In the literature, two types of techniques have been actively studied: a machine learning-based parameter tuning technique and a DNN-based ISP technique. The former is lightweight but lacks expressive power. The latter has expressive power, but the computational cost is too heavy on edge devices. To solve these problems, we propose "DynamicISP," which consists of multiple classical ISP functions and dynamically controls the parameters of each frame according to the recognition result of the previous frame. We show our method successfully controls the parameters of multiple ISP functions and achieves state-of-the-art accuracy with low computational cost in single and multi-category object detection tasks.

This paper presents Cellular Fuzzy Oriented Classifier Evolution (CFORCE), a generic method for constructing fuzzy rules to divide an image into two segments: object and background. In CFORCE, a pair of fuzzy classification rule sets for object and background is defined as a processing unit, and the identical units are allocated in each pixel of an input image. Each unit computes matching degree of each pixel with object and background class iteratively with considering the matching degree of neighbor units. The algorithm has mainly two features: 1) designing the fuzzy rules using Fuzzy Oriented Classifier Evolution (FORCE) which develops fuzzy rules represented as directed graphs flexibly and automatically by Genetic Algorithm, and 2) performing iterative segmentation with considering spatial relationship between pixels besides local features. In natural image segmentation, many pixels are overlapped between different clusters. Therefore, considering the spatial relationship is important to classify the overlapped pixels correctly. We applied CFORCE to three different object segmentation, and showed that CFORCE extracted object regions successfully.

Fuzzy ORiented Classifier Evolution (FORCE) is a graph-based genetic fuzzy system which we have previously proposed. FORCE constructs fuzzy classification rules automatically by evolving directed graphs composed of fuzzy conditions using Genetic Algorithm. In this work, to improve FORCE about efficiency of rule optimization and expressiveness of Membership Functions (MFs), we introduce two new ideas into FORCE: Edge Mutation (EM) and Parameter tunable MFs (PMFs). EM changes node connections with considering current graph structure to develop rules efficiently unlike the original mutation changing them just randomly. PMFs are MFs characterized by real-coded parameters optimized using uniform and non-uniform mutation. PMFs improve the expressiveness of MFs, which are represented by combination of user-defined parameters in the previous work. We tested the improved FORCE with 21 classification data sets in comparison with our previous model and a common method, and experimental results showed the proposed ideas improved classification accuracy of FORCE.