The multi-instance multi-label (MIML) problem is a new supervised learning paradigm that has emerged to efficiently represent complex data. Therefore, various similarity-based algorithms have been proposed, but existing algorithms commonly measure similarity by considering only the structural relationships in the feature space without utilizing information from the label space. As these approaches do not adequately reflect the complex properties of MIML data, it is essential to improve the accuracy of MIML classification by utilizing information from both feature and label spaces. Thus, we propose a new algorithm, T-MDML: triplet-based multiple distance metric learning for MIML. T-MDML defines a distance metric by learning a global property shared by the entire label space and a label-specific property for each label. In addition, we simultaneously consider the structural characteristics of features and label space to extract label correlation and incorporate it into the optimization process. In experiments, we demonstrate the efficiency of our label correlation estimation method and verify its performance by applying it to MIMLkNN. We also demonstrate T-MDML’s relative superiority over existing MIML algorithms, as well as its scalability when applied to similarity-based MIML methods.

When an image is seen on an optical see-through augmented reality (AR) display, the light from the display is mixed with the background light from the environment. This can severely limit the available contrast in AR, which is often orders of magnitude below that of traditional displays. Yet, the presented images appear sharper and show more details than the reduction in physical contrast would indicate. In this work, we hypothesize two effects that are likely responsible for the enhanced perceived contrast in AR: background discounting, which allows observers focused on the display plane to partially discount the light from the environment; and supra-threshold contrast perception, which explains the differences in contrast perception across luminance levels. In a series of controlled experiments on an AR high-dynamic-range multi-focal haploscope testbed, we found no statistical evidence supporting the effect of background discounting on contrast perception. Instead, the increase of visibility in AR is better explained with models of supra-threshold contrast perception. Our findings can be generalized to incorporate an image input, and this model serves to design better algorithms and hardware for display systems affected by additive light, such as AR.

The perception of visual content in optical-see-through augmented reality (AR) devices is affected by the light coming from the environment. This additional light interacts with the content in a non-trivial manner because of the illusion of transparency, different focal depths, and motion parallax. To investigate the impact of environment light on display artifact visibility (such as blur or color fringes), we created the first subjective quality dataset targeted toward augmented reality displays. Our study consisted of 6 scenes, each affected by one of 6 distortions at two strength levels, seen against one of 3 background patterns shown at 2 luminance levels: 432 conditions in total. Our dataset shows that environment light has a much smaller masking effect than expected. Further, we show that this effect cannot be explained by compositing of the AR-content with the background using optical blending models. As a consequence, we demonstrate that existing video quality metrics perform worse than expected when predicting the perceived magnitude of degradation in AR displays, motivating further research.

Holographic near-eye displays are a promising technology to solve long-standing challenges in virtual and augmented reality display systems. Over the last few years, many different computer-generated holography (CGH) algorithms have been proposed that are supervised by different types of target content, such as 2.5D RGB-depth maps, 3D focal stacks, and 4D light fields. It is unclear, however, what the perceptual implications are of the choice of algorithm and target content type. In this work, we build a perceptual testbed of a full-color, high-quality holographic near-eye display. Under natural viewing conditions, we examine the effects of various CGH supervision formats and conduct user studies to assess their perceptual impacts on 3D realism. Our results indicate that CGH algorithms designed for specific viewpoints exhibit noticeable deficiencies in achieving 3D realism. In contrast, holograms incorporating parallax cues consistently outperform other formats across different viewing conditions, including the center of the eyebox. This finding is particularly interesting and suggests that the inclusion of parallax cues in CGH rendering plays a crucial role in enhancing the overall quality of the holographic experience. This work represents an initial stride towards delivering a perceptually realistic 3D experience with holographic near-eye displays.

The evolution of computer-generated holography (CGH) algorithms has prompted significant improvements in the performances of holographic displays. Nonetheless, they start to encounter a limited degree of freedom in CGH optimization and physical constraints stemming from the coherent nature of holograms. To surpass the physical limitations, we consider polarization as a new degree of freedom by utilizing a novel optical platform called metasurface. Polarization-multiplexing metasurfaces enable incoherent-like behavior in holographic displays due to the mutual incoherence of orthogonal polarization states. We leverage this unique characteristic of a metasurface by integrating it into a holographic display and exploiting polarization diversity to bring an additional degree of freedom for CGH algorithms. To minimize the speckle noise while maximizing the image quality, we devise a fully differentiable optimization pipeline by taking into account the metasurface proxy model, thereby jointly optimizing spatial light modulator phase patterns and geometric parameters of metasurface nanostructures. We evaluate the metasurface-enabled depolarized holography through simulations and experiments, demonstrating its ability to reduce speckle noise and enhance image quality.

De novo assembly, which discovers the entire nucleotide sequence by reconstructing the reads resulting from next-generation sequencing, is a subject that must be studied for genetic information analysis. The recombination of reads is performed in several steps, but gaps that cannot be resolved occur even after scaffolding. Gap-filling is performed as the last assembly stage to fill the unidentified regions called gaps, significantly improving overall assembly performance. We propose a gap-filling method using hybrid reads to resolve gaps based on sequence similarity estimation and graph searches. The proposed method consists of three key steps: extracting the candidate sequence, estimating similarity, and filling the gaps based on the graph. Hybrid reads extract sequences with more accurate information, and candidate sequences corresponding to noise are effectively removed based on the similarity estimation. In conclusion, a graph search using statistical information derives a final sequence that guarantees high coverage, resolves gaps, reduces misassemblies, and improves accuracy.

Big data analysis can enhance police activities and promote improvements in police policy. Police agencies in various countries recognize the need for big data-based policing and are engaged in various research and field support activities. However, big data analysis has rarely been applied in actual patrolling for crime prevention. Therefore, this study established a novel model that uses big data to efficiently respond to emergency reports of crimes. First, we extracted hotspots that were regions of crime incidence based on the correlation between community environment data and crime data. Second, we used a new optimization method based on a genetic algorithm to generate a patrol sequence that required the shortest travel time to the crime area. Finally, we verified the efficiency of the patrol routes using the real-time traffic data of a map API. The proposed big data–based model can be applied to existing patrolling systems and can enhance the efficiency of all police departments.

Holographic displays have gained unprecedented attention as next-generation virtual and augmented reality applications with recent achievements in the realization of a high-contrast image through computer-generated holograms (CGHs). However, these holograms show a high energy concentration in a limited angular spectrum, whereas the holograms with uniformly distributed angular spectrum suffer from a severe speckle noise in the reconstructed images. In this study, we claim that these two physical phenomena attributed to the existing CGHs significantly limit the support of accommodation cues, which is known as one of the biggest advantages of holographic displays. To support the statement, we analyze and evaluate various CGH algorithms with contrast gradients - a change of contrast over the change of the focal diopter of the eye - simulated based on the optical configuration of the display system and human visual perception models. We first introduce two approaches to improve monocular accommodation response in holographic viewing experience; optical and computational approaches to provide holographic images with sufficient contrast gradients. We design and conduct user experiments with our prototype of holographic near-eye displays, validating the deficient support of accommodation cues in the existing CGH algorithms and demonstrating the feasibility of the proposed solutions with significant improvements on accommodative gains.

Since 1960s, digital holography (DH) has been developed significantly as an exquisite optical sensing technique with its computational interpretation. Recent advances in the image sensor and the coherent light source have suggested the possibility of using DH in portable form that can be utilized in industrial fields. Field of view (FOV) should be sufficiently wide and the system be simple for successful landing of DH on the industrial market. To embody the possibility, in this paper, we present a method for implementing off-axis reflective DH using a single diffraction grating. Beam splitter and mirror in the conventional off-axis DH are replaced with a single diffraction grating. This replacement enables the implementation of simple interferometer while maintaining the performance of the holography. In addition, we apply the multiplexed illumination method to enhance space-bandwidth product of off-axis hologram. Using the wavelength dependence of the diffraction grating, the FOV can be doubled without additional optical element. We demonstrate the proposed method by three holographic imaging experiments with two laser diodes. The prototype provides lateral resolution of 19.69$\mu$m with the FOV of 6.85 mm × 8.72 mm. The FOV is 1.9 times wider than the image sensor area of 6.85 mm × 4.59 mm. Overall size of the prototype is 12 cm × 17 cm × 5 cm, showing the suitability of implementation to the various industrial fields.

Over the past century, as display evolved, people have demanded more realistic and immersive experiences in theaters. Here, we present a tomographic projector for a volumetric display system that accommodates large audiences while providing a uniform experience. The tomographic projector combines high-speed digital micromirror and three spatial light modulators to refresh projection images at 7200 Hz. With synchronization of the tomographic projector and wearable focus-tunable eyepieces, the presented system can reconstruct 60 focal planes for volumetric representation right in front of audiences. We demonstrate proof of concept of the proposed system by implementing a miniaturized theater environment. Experimentally, we show that this system has wide expressible depth range with focus cues from 25 cm to optical infinity with sufficient tolerance while preserving high resolution and contrast. We also confirm that the proposed system provides uniform experience in a wide range of viewing zone through simulation and experiment. Additionally, the tomographic projector has capability to equalize vergence state that varies in conventional stereoscopic 3D theater according to viewing position as well as interpupillary distance. This study is concluded with thorough discussion about tomographic projectors in terms of challenges and research issues.

Abstract This paper presents an automatic method for producing stipple renderings from photographs, following the style of professional hedcut illustrations. For effective depiction of image features, we introduce a novel dot placement algorithm which adapts stipple dots to the local shapes. The core idea is to guide the dot placement along ‘feature flow’ extracted from the feature lines, resulting in a dot distribution that conforms to feature shapes. The sizes of dots are adaptively determined from the input image for proper tone representation. Experimental results show that such feature‐guided stippling leads to the production of stylistic and feature‐emphasizing dot illustrations.