Extracting and exploiting edge information is important for various computer vision tasks. According to the physical characteristics, edges are further categorized into reflectance, illumination, normal, and depth discontinuities. Previous studies for edge discontinuity classification have achieved impressive performance in extracting discontinuities, but the capacity of classification remains limited. This paper proposes TriDecTr, a novel network comprising a transformer-based encoder to improve semantic understanding and a tri-directional decoder to explore relationships among categories. Extensive experiments demonstrate that TriDecTr achieves state-of-the-art performance on the BSDS-RIND dataset with 0.531 in ODS, 0.571 in OIS, and 0.461 in AP. Moreover, TriDecTr significantly narrows the performance gap between illumination edges and the other categories from 0.191 to 0.118 in ODS.

For immersive media services such as volumetric video, 3-dimensional (3D) data such as point cloud or mesh is commonly used. The size of such data is significantly larger than that of 2-dimensional (2D) video, and its characteristics are also different from those of conventional video. Recently, MPEG standardized video-based point cloud (V-PCC) and is now developing for video-based dynamic mesh coding (V-DMC), which changes connectivity information of vertices over time. Encoding of mesh parts is divided into generation of decimated mesh and compression of displacement for mesh reconstruction. The displacement data is being compressed using a conventional video CODEC in the form of YUV. However, in YUV of displacement, there is no spatial coherence which is commonly observed in general video. Therefore, currently adopted intra-prediction based video encoding in V-DMC, where prediction is performed in unit of blocks within one frame, is not able to exploit the characteristic of displacement data and brings low compression. This paper proposes a conversion technique for displacement data to increase compression efficiency. Displacement YUVs in a group-of-frames (GOF) are changed into a single micro image (MI) to secure better coherence. In order to maximize compression efficiency, pixels in MI are reordered in a Morton-order. This Morton-ordered MI (MoMI) performs decomposition of frequencies, and ensures a smooth area composed of zero pixels as much as possible. Experimental results show that the proposed MoMI gives better spatial coherency and improves the compression efficiency up to 22.64%

Light field (LF), a new approach in three-dimensional image processing, has been actively used in various applications in recent years. LF is based on a large amount of data and this always leads to LF compression (LFC) issues. Pseudo-sequence (PS)-based LFC converts a LF into a video sequence and compresses it through a video codec, whereas synthesis-based LFC (SYN-LFC) synthesizes the rest from some of the LF to reduce the number of bits. SYN-LFC is superior to PS-based LFC at low bitrates. However, its competitiveness decreases at high bitrates due to the inefficient compression of residuals. This paper maximizes the advantages of SYN-LFC by increasing the compression efficiency of residuals. To exploit the characteristic of the residual in favor of compression, this paper compresses the residual in the form of a micro image (MI). The conversion of residuals to MI has the effect of gathering similar residuals of each viewpoint, which increases the spatial coherence. However, the conventional MI conversion does not reflect the geometric characteristics of LF at all. To tackle this problem, this paper proposes the re-ordered micro image (RoMI), which is a novel MI conversion that takes advantage of the geometric characteristics of LF, thereby maximizing the spatial coherence and compression efficiency. To compress MI-type residuals, JPEG2000, an image-level codec, is used. It is highly suitable for RoMI with spatial coherence beyond the block level. In the experimental results, the proposed RoMI shows average improvements of 30.29% and 14.05% in the compression efficiency compared to the existing PS-based LFC and SYN-LFC methods, respectively.

Recently, research on six-degree-of-freedom virtual reality (VR) systems based on a light field (LF) has been actively conducted. The LF-based approach is photorealistic and less prone to errors than the existing three-dimensional (3D) modeling-based approach. On the other hand, because the amount of data is very large for LF-based immersive virtual reality, it is important to manage the data efficiently. This paper makes two major contributions to data management for rendering acceleration in this context. First, the GPUs commonly used for high-speed rendering require optimization of the data transfer process between the host memory and the device memory. This paper proposes a simple but effective way to maximize data reuse by finding the proper size of a transfer unit depending on the target applications and the system capability constraints. Also, view synthesis and data transfer are performed in a pipelined manner to hide the data movement time. Second, for the LF rendering of a wide space, data management in the storage-host-GPU hierarchy is attempted for the first time. As the size of the virtual space increases, the required amount of LF data grows rapidly and exceeds the allocatable host memory capacity. Because the loading of data stored in storage is a very slow process, a system-level LF rendering structure that carefully considers the characteristics of the storage-host-GPU memory hierarchy should be designed. In this paper, progressive LF updating is proposed, and it prevents rendering stalls caused by slow storage data loading. Experimental results show that a 360-degree view with a$9000 \times 2048$resolution can be rendered in an average of 2.57 ms from LF data with a$4096 \times 2048$resolution, achieving performance close to 400 frames per second.

Conventional captured-image-based virtual reality (VR) systems have three degrees of freedom (DoFs), where only rotational user motion is tracked for view rendering. This is a major cause of the reduced sense of reality. To increase user immersion levels akin to the real world, 3-DoF+ VR systems that support not only rotational but also translational view changes have been proposed. The light-field (LF) approach is suitable for this type of 3-DoF+ VR because it renders a view from a free view position by simply combining lights. Many previous systems have limited scalability because they assume a single LF for the acquisition and representation of light. One recent work connects multiple LFs at a physical intersection to increase the scalability. However, these fixed connection points limit the renderable view range and the layout of multiple LFs. Furthermore, in conventional single- or multiple-LF systems, the representable ranges of the view positions are highly dependent on the input field-of-view (FOV) of the camera used. In order to realize a wide view exploration range, the above-mentioned limitations must be overcome. This paper proposes a flexible connection scheme for multiple-LF systems taking advantage of the constant radiance of rays in LF theory. The proposed flexibly connectable LF system is able to widen the range of the renderable view position under reasonable conditions of the camera FOV. A light-field unit (LFU) which uses the proposed flexible connection is implemented. The LFU has a square-shaped structure and is thus easily stackable. This offers the advantage of dramatically expanding the scope of view explorations. The proposed LFU achieves 3-DoF+ VR with good quality as well as high scalability. Its cost-performance outcome is also better compared to those in previous works.

Conventional 3DoF (Degree of Freedom) systems provide only rotational view transformation. 6DoF VR systems offer a more realistic VR by supporting view transformation for translational movement. This paper proposes ray-space 360 which supports both rotational and translational view transformation simultaneously. Ray-space360 is an extension of ray-space [1-3] to an omnidirectional view in order to support a new camera structure in a square form for ray-space360 configuration. Ray-space360 also provides a way to use the intersection to connect independent ray-spaces. Thanks to its low complexity, the proposed system can provide real-time service with a low performance processor such as a mobile processor. This paper presents the result of 4DoF ray-space360 which provides view transformation of yaw, pitch, x and z axes. The ray-space360 can be extended to 6DoF simply by stacking cameras.