Reconstructing visual stimulus representation is a significant task in neural decoding. Until now, most studies have considered functional magnetic resonance imaging (fMRI) as the signal source. However, fMRI-based image reconstruction methods are challenging to apply widely due to the complexity and high cost of acquisition equipment. Taking into account the advantages of the low cost and easy portability of electroencephalogram (EEG) acquisition equipment, we propose a novel image reconstruction method based on EEG signals in this article. First, to meet the high recognizability of visual stimulus images in a fast-switching manner, we construct a visual stimuli image dataset and obtain the corresponding EEG dataset through EEG signals collection experiment. Second, we introduce the deep visual representation model (DVRM), comprising a primary encoder and a subordinate decoder, to reconstruct visual stimuli representation. The encoder is designed based on residual-in-residual dense blocks to learn the distribution characteristics between EEG signals and visual stimulus images. Meanwhile, the decoder is designed using a deep neural network to reconstruct the visual stimulus representation from the learned deep visual representation. The DVRM can accommodate the deep and multiview visual features of the human natural state, resulting in more precise reconstructed images. Finally, we evaluate the DVRM based on the quality of the generated images using our EEG dataset. The results demonstrate that the DVRM exhibits an excellent performance in learning deep visual representation from EEG signals, generating reconstructed representation of images that are realistic and highly resemble the original images.

Spotting camouflaged objects that are visually assimilated into the background is tricky for both object detection algorithms and humans who are usually confused or cheated by the perfectly intrinsic similarities between the foreground objects and the background surroundings. To tackle this challenge, we aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries. We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner, essentially a global loop-based connection among the multi-scale resolutions. To design better feedback feature flow and avoid the feature corruption caused by recurrent path, an iterative feedback strategy is proposed to impose more constraints on each feedback connection. Extensive experiments on four challenging datasets demonstrate that our HitNet breaks the performance bottleneck and achieves significant improvements compared with 29 state-of-the-art methods. In addition, to address the data scarcity in camouflaged scenarios, we provide an application example to convert the salient objects to camouflaged objects, thereby generating more camouflaged training samples from the diverse salient object datasets. Code will be made publicly available.

Classical microfacet theory suffers from energy loss on materials with high roughness due to the single bounce assumption of most microfacet models. When roughness is high, there is a large chance of multiple scattering occurring among the microfacets of the surface. Without explicitly modelling for this behaviour, rough surfaces appear darker than they should. To address this issue, we present a novel method to estimate the multiple scattering contribution from a second light bounce. Our method is inspired by Zipin’s geometric construction approach, which simplifies the calculation of the light transport inside a V-groove cavity. Our experimental results demonstrate that our method is visually pleasing, physically plausible, and artifact-free compared to recent multiple scattering works. Additionally, the low computational cost makes our model suitable for real-time rendering.

Deep learning approaches often require huge datasets to achieve good generalization. This complicates its use in tasks like image-based medical diagnosis, where the small training datasets are usually insufficient to learn appropriate data representations. For such sensitive tasks it is also important to provide the confidence in the predictions. Here, we propose a way to learn and use probabilistic labels to train accurate and calibrated deep networks from relatively small datasets. We observe gains of up to 22% in the accuracy of models trained with these labels, as compared with traditional approaches, in three classification tasks: diagnosis of hip dysplasia, fatty liver, and glaucoma. The outputs of models trained with probabilistic labels are calibrated, allowing the interpretation of its predictions as proper probabilities. We anticipate this approach will apply to other tasks where few training instances are available and expert knowledge can be encoded as probabilities.

Chenyang Huang, Amine Trabelsi, Xuebin Qin, Nawshad Farruque, Lili Mou, Osmar Zaïane. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Fuzzy control rules can be converted into mathematical functions through fuzzy set theory and fuzzy reasoning theory, which can be easily combined with other physical laws to realize control strategies through computer software. At the same time, the fuzzy rules designed by expert experience can effectively control complex controlled objects, after actual debugging, its robustness and adaptability can easily meet the requirements. Combined with the fuzzy self-tuning PID control strategy, a fuzzy PID control system of the vehicle inverter is designed. Through the analysis of the inverter control system in the specific project, the mathematical model of the entire control system is obtained, and MATLAB simulation is carried out. The simulation results show that the fuzzy PID control system greatly improves the steady-state performance of the system, the output waveform distortion is small, and the harmonic content is low, which better meets the control requirements of the vehicle inverter.

Deep Convolutional Neural Networks have been adopted for salient object detection and achieved the state-of-the-art performance. Most of the previous works however focus on region accuracy but not on the boundary quality. In this paper, we propose a predict-refine architecture, BASNet, and a new hybrid loss for Boundary-Aware Salient object detection. Specifically, the architecture is composed of a densely supervised Encoder-Decoder network and a residual refinement module, which are respectively in charge of saliency prediction and saliency map refinement. The hybrid loss guides the network to learn the transformation between the input image and the ground truth in a three-level hierarchy -- pixel-, patch- and map- level -- by fusing Binary Cross Entropy (BCE), Structural SIMilarity (SSIM) and Intersection-over-Union (IoU) losses. Equipped with the hybrid loss, the proposed predict-refine architecture is able to effectively segment the salient object regions and accurately predict the fine structures with clear boundaries. Experimental results on six public datasets show that our method outperforms the state-of-the-art methods both in terms of regional and boundary evaluation measures. Our method runs at over 25 fps on a single GPU. The code is available at: https://github.com/NathanUA/BASNet.

Although semi-dense Simultaneous Localization and Mapping (SLAM) has been becoming more popular over the last few years, there is a lack of efficient methods for representing and processing their large scale point clouds. In this paper, we propose using 3D line segments to simplify the point clouds generated by semi-dense SLAM. Specifically, we present a novel incremental approach for 3D line segment extraction. This approach reduces a 3D line segment fitting problem into two 2D line segment fitting problems and takes advantage of both images and depth maps. In our method, 3D line segments are fitted incrementally along detected edge segments via minimizing fitting errors on two planes. By clustering the detected line segments, the resulting 3D representation of the scene achieves a good balance between compactness and completeness. Our experimental results show that the 3D line segments generated by our method are highly accurate. As an application, we demonstrate that these line segments greatly improve the quality of 3D surface reconstruction compared to a feature point based baseline.

In this paper, we propose a novel real-time method for tracking planar edge templates. This method tracks an edge template by estimating its homography transformations with respect to the sampled edge pixels detected from the incoming frames. Particularly, we define a cost function based on a new feature map of the to-be-tracked edge template and optimize it by a Lucas-Kanade-like algorithm. The feature map is defined as the fourth root of the distance transform. Our method operates on just edges so that it is good at tracking those low textured targets, such as hollow targets (mug rim), thin targets (cable, ring) and non-Lambertian objects (disc). We validate and compare our method with four other methods on five newly collected real-world video sequences. The results achieves the lowest overall average error (1.58 pixels) and also outperforms others in terms of success rate. The per frame processing time of about 30 ms proves that our method is acceptable in realtime applications. The code and dataset are publicly available at: http://webdocs.cs.ualberta.ca/~xuebin/.

This paper presents a novel boundary based semiautomatic tool, ByLabel, for accurate image annotation. Given an image, ByLabel first detects its edge features and computes high quality boundary fragments. Current labeling tools require the human to accurately click on numerous boundary points. ByLabel simplifies this to just selecting among the boundary fragment proposals that ByLabel automatically generates. To evaluate the performance of By-Label, 10 volunteers, with no experiences of annotation, labeled both synthetic and real images. Compared to the commonly used tool LabelMe, ByLabel reduces image-clicks and time by 73% and 56% respectively, while improving the accuracy by 73% (from 1.1 pixel average boundary error to 0.3 pixel). The results show that our ByLabel outperforms the state-of-the-art annotation tool in terms of efficiency, accuracy and user experience. The tool is publicly available: http://webdocs.cs.ualberta.ca/~vis/ bylabel/.

This letter presents a novel approach for extracting accurate outlines of individual buildings from very high-resolution (0.1-0.4 m) optical images. Building outlines are defined as polygons here. Our approach operates on a set of straight line segments that are detected by a line detector. It groups a subset of detected line segments and connects them to form a closed polygon. Particularly, a new grouping cost is defined first. Second, a weighted undirected graph G(V,E) is constructed based on the endpoints of those extracted line segments. The building outline extraction is then formulated as a problem of searching for a graph cycle with the minimal grouping cost. To solve the graph cycle searching problem, the bidirectional shortest path method is utilized. Our method is validated on a newly created data set that contains 123 images of various building roofs with different shapes, sizes, and intensities. The experimental results with an average intersection-over-union of 90.56% and an average alignment error of 6.56 pixels demonstrate that our approach is robust to different shapes of building roofs and outperforms the state-of-the-art method.

This paper presents a novel real-time method for tracking salient closed boundaries from video image sequences. This method operates on a set of straight line segments that are produced by line detection. The tracking scheme is coherently integrated into a perceptual grouping framework in which the visual tracking problem is tackled by identifying a subset of these line segments and connecting them sequentially to form a closed boundary with the largest saliency and a certain similarity to the previous one. Specifically, we define a new tracking criterion which combines a grouping cost and an area similarity constraint. The proposed criterion makes the resulting boundary tracking more robust to local minima. To achieve real-time tracking performance, we use Delaunay Triangulation to build a graph model with the detected line segments and then reduce the tracking problem to finding the optimal cycle in this graph. This is solved by our newly proposed closed boundary candidates searching algorithm called “Bidirectional Shortest Path (BDSP)”. The efficiency and robustness of the proposed method are tested on real video sequences as well as during a robot arm pouring experiment.

This paper presents a novel feature grouping based framework for building facade recognition from aerial images. A combination of Maximally Stable Extremal Regions (MSERs and steered Determinant-of-Hessian (steered-DoH) are proposed to detect different shapes of blobs from images. Then we employ local parallelogram grouped by these repetitive and evenly distributed blobs to form an point-based regularity measurement. Building facade regions are indicated by these local parallelograms. In our work, we use Delaunay Triangulation (DT) to guide the search of local parallelograms. Our approach can handle images with large range of resolution. Vertical and horizontal assumptions of facades are not required. The experimental results conducted on images with different resolutions and different types of facades demonstrate superior performance on facade recognition both in terms of speed and accuracy (F1- score over 80%) over state-of-the-art methods.

This paper presents a novel deep hierarchical representation and segmentation approach for high resolution remote sensing image understanding. An information extraction approach using deep hierarchical exploitation for remote sensing image is presented. The key idea is that we adopt a fast scanning image segmentation within a deep hierarchical feature representation framework, using a deep learning technique to split and merge over-segmented regions until they form meaningful objects. The contribution is to develop an effective procedure for multi-scale image representation to address the issue of information uncertainty in practical applications. We test our method on two optical high resolution remote sensing image datasets and produce promising experimental results in the form of multiple layer outputs, which confirm the effectiveness and robustness of the proposed procedure.

Road damage detection from high-resolution remote sensing image is critical for natural disaster investigation and disaster relief. In a disaster context, the pair of pre-disaster and post-disaster road data for change detection are difficult to obtain due to the mismatch of different data sources, especially for rural areas where the pre-disaster data (i.e. remote sensing imagery or vector map) are hard to obtain. In this study, a knowledge-based method for road damage detection solely from post-disaster high-resolution remote sensing image is proposed. The road centerline is firstly extracted based on the preset road seed points. Then, features such as road brightness, standard deviation, rectangularity, aspect ratio are selected form a knowledge model. Finally, under the guidance of the road centerline, the post-disaster roads are extracted and the damaged roads were detected by applying the knowledge model. The newly developed method is evaluated using a WorldView-1 image over Wenchuan, China acquired three days after the earthquake in May 15, 2008. The results show that the producer's accuracy (PA) and user's accuracy (UA) reached about 90% and 85% respectively, indicating that the proposed method is effective for road damage detection. This approach also significantly reduces the need for pre-disaster remote sensing data.

This paper presents a new method to detect damaged buildings caused by earthquake from high spatial resolution remote sensing image. We found that the probability of multiple gradient orientations is greater in a local area within a damaged building than that in a local area within an intact building. Therefore, a new feature (Local Gradient Orientation Entropy, LGOE) was put forward to determine whether a building was damaged. First, gradient information was obtained by Prewitt gradient operator. Second, the gradient orientation entropy of one pixel was calculated in a local 3 ×3 window. Last, average LGOE value within a building boundary was counted. In general, damaged buildings have higher LGOE values because of their irregular texture. Therefore, an optimum LGOE threshold value (LGOET) was set to detect damaged buildings. The experiment results of Yushu earthquake using a Quickbird image demonstrated that our method was effective. Of the total 101 buildings, 87 were detected correctly, the overall accuracy was 86.14%, and the overall kappa coefficient is 72.25%.

This letter presents a new approach for rapid automatic building extraction from very high resolution (VHR) optical satellite imagery. The proposed method conducts building extraction based on distinctive image primitives such as lines and line intersections. The optimized framework consists of three stages: First, a developed edge-preserving bilateral filter is adopted to reduce noise and enhance building edge contrast for preprocessing. Second, a state-of-the-art line segment detector called EDLines is introduced for the real-time accurate extraction of building line segments. Finally, we present a graph search-based perceptual grouping approach to hierarchically group previously detected line segments into candidate rectangular buildings. The recursive process was improved through the efficient examination of geometrical information with line linking and closed contour search, in order to obtain more reasonable omission and commission rate in building contour grouping. Extensive experiments performed on VHR optical QuickBird imageries justify the effectiveness and robustness of the proposed linear-time procedure with an overall accuracy of 80.9% and completeness of 87.3%. This method does not require user intervention and thereby has the potential to be adopted in online applications and industrial use in the near future.

Soil salinization is one of the most devastating land degradation process causing agricultural yields reduction. This paper presents a hyperspectral prediction model of soil salinity using partial least squares regression (PLSR) in Tianjin costal area. Soil spectral reflectance of soil samples varying in salinity was measured using an ASD Field Spec spectrometer. The treated continuum-removed (CR) reflectance and first-order derivative reflectance (FDR) were used and compared to explore the more preferable predicting model of soil salinity, which could detect subtle differences in spectral absorption features compared with original reflectance. The results showed that the soil spectra reflectance got distinct absorption feature with peaks centred at 411 nm, 475 nm, 663 nm, 868 nm, 1100 nm ~ 1250 nm, 1400 nm, 690 nm, 1911 nm, 2206 nm and 2338 nm, representing key bands for soil salt content estimation. Through established Partial Least-Square Regression model based on treated soil spectra, the first derived-continuum-removed reflectance was the optimal spectra indexes, prediction accuracy of the optimal PLSR model was 94.4%.

Road is a kind of very typical artificial object. Road extraction from multi-scale remote sensing images is significant both in military field and in people's daily lives. With the development of remote sensing technology, the scale of remote sensing images that can be obtained becomes various. Therefore, the research of multi-scale remote sensing images is getting more and more attention and it is really a challenging task in the field of image processing. In this paper, a method of road extraction from multi-scale remote sensing images is proposed. Firstly, the textures are extracted and added to the bands of the original image. The filtering, resampling and segmentation operations are then implemented. Next, the spectral characteristics and textures of roads on the remote sensing images are statistically analyzed, and the changes of those on multi-scale remote sensing images are obtained. Then, considering the road characteristics displayed on remote sensing images, some parameters of spectral characteristics and textures are selected to extract roads using the object-oriented method. Finally, the results of road extraction are post-processed based on the opening and closing operation of mathematical morphology. This study has great significance in areas such as features optimization, target recognition, building feature database and improving the utilization of remote sensing data.

The paper realizes façade 3D reconstruction using recently promising oblique photogrammetry data. the point-to-point problems in traffic network. We make full use of the multi-level image features to extract interest regions of façade, and then present a backwards coarse-to-fine matching, which makes the auxiliary data unnecessary. The experiment shows the efficiency and robustness of the proposed method, and the vectors describing façade 3D information are also verify the high precision.

In this paper, the feasibility of building seismic damage assessment using oblique images is discussed. Firstly, some geometric features of buildings are selected and analyzed according to the universal geometric characteristics of most buildings. Secondly, the parameters of each geometric features are calculated by the method of photogrammetry. Finally, the results of parameters calculation are analyzed and discussed. The results show that more information of building can be obtained from oblique airborne images than from vertical images. Furthermore, these information can reflect the conditions of buildings correctly and effectively in three dimensional (3D) space. The conclusion is drawn that it is feasible and potential to conduct building seismic damage assessment based on oblique airborne images.

Soil Salinization is one of the most devastating land degradation in agriculture and environment all around the world. The salinization classification map composed of four stages of salinity can be generated using remote sensing technology. In this paper we propose a semi-supervised procedure for dynamic evolution monitoring of soil salinization in Tianjin costal area based on the use of Landsat data. The trends and extents of soil salinization for a period from 1989 to 2009 are investigated every 10 years. Compared with traditional 2-D spectral space of NIR-R bands, the results reveals that highly saline soils can be easily differentiated in NDVI-Brightness 2-D spectral space, which is a powerful tool for exploring salinization information in Landsat data. Mapping and monitoring soil salinization changes in arid and semi-arid regions by means of remote sensing is effective for producing detailed and accurate land degradation information.

This paper presents an automatic procedure for rapid building extraction from optical very high resolution (VHR) satellite imagery. Classical extraction models are always complex and time-consuming. The optimized process of building extraction consists of three main rapid stages: edge-preserving and smoothing bilateral filter, line segment detection, perceptual grouping polygonal building boundary. Firstly, we use bilateral filter to smooth original image with edge-preserving. Secondly, a state-of-the-art line segment detector (LSD) algorithm gives highly accurate building contour segments. Finally, we apply the perceptual grouping approach based on graph search to organize detected contour line segments of interested buildings. We test our method on optical VHR QuickBird satellite imagery and obtain promising experimental results with overall accuracy of 79.1%, which confirm the effectiveness and robustness of this linear-time procedure.