Abstract-High-precision localization of underwater robots in enclosed tunnel environments is challenging due to the absence of satellite signals and the practical difficulty of deploying large-scale acoustic infrastructure. Existing solutions typically fuse inertial navigation with vision, laser, or acoustic sensing, yet their performance is often limited by rapid inertial drift and degraded sensing under turbidity, specular surfaces, and constrained geometries. This paper presents SPLoc, a structural-prior-assisted localization and measurement framework, together with an underwater tunnel pose measurement platform that tightly integrates a blue–green structured-light triangulation sensor and an IMU. The structured-light subsystem acquires dense cross-sectional point clouds with centimeter-scale sampling along both axial and radial directions. From these measurements, tunnel cross-sectional primitives are extracted through feature enhancement, redundancy suppression, and RANSAC-based circle fitting with region-growing verification, and are then parameterized as structural priors. To achieve metrologically consistent fusion, we formulate a least-squares adjustment that explicitly models heading and position states, laser observation residuals, and dominant pose-error terms (including heading/position biases and observation perturbations), and solve it via iterative refinement to provide dynamic pose compensation. Simulation and physical-equivalent experiments on a purpose-built testbed demonstrate that SPLoc achieves < 2 cm localization error and < 0.5° heading error under diverse initial disturbances, while improving robustness against sensing outliers and geometric ambiguities. The proposed platform and estimation model provide a practical measurement route for accurate underwater tunnel robot localization and support engineering deployment where external beacons are unavailable.

Robust pavement crack detection in complex scenes remains a significant challenge. This stems not merely from the scarcity of annotated data, but more critically, from the severe lack of pattern diversity within existing datasets. Key variations in morphology, scale, background texture, and imaging conditions are often underrepresented, which fundamentally impedes the generalization capability of recognition models. While generative approaches (e.g., GANs and diffusion models) offer a potential path to augment this diversity synthetically, they commonly suffer from poor background realism, entangled structural-appearance representations, and a lack of precise control over generated defects. This paper presents the Crack Diffusion Generator (DiffCrack), a diffusion-based framework designed for semantic-structural controllability in crack image synthesis. DiffCrack decouples crack geometry and visual appearance through two conditioning inputs: a binary mask to anchor spatial layout, and a Hierarchical Prompt Attention (HPA) module to independently modulate attributes such as width, depth, color, and texture. This design enables the controllable and targeted generation of diverse, photorealistic crack patterns that are often missing from real-world datasets. Extensive experiments on real datasets demonstrate that training with DiffCrack-generated images enhances the F1-score of segmentation models by up to 23% on average under complex scene conditions. This result validates that DiffCrack is a scalable, pattern-aware data augmentation tool. By addressing the critical bottleneck of data diversity, our framework offers a principled pathway to improving the robustness and generalization of infrastructure inspection models.

Most publicly accessible remote sensing data suffer from low resolution, limiting their practical applications. To address this, we propose a diffusion model guided by neural operators for continuous remote sensing image super-resolution (NeurOp-Diff). Neural operators are used to learn resolution representations at arbitrary scales, encoding low-resolution (LR) images into high-dimensional features, which are then used as prior conditions to guide the diffusion model for denoising. This effectively addresses the artifacts and excessive smoothing issues present in existing super-resolution (SR) methods, enabling the generation of high-quality, continuous super-resolution images. Specifically, we adjust the super-resolution scale by a scaling factor s, allowing the model to adapt to different super-resolution magnifications. Furthermore, experiments on multiple datasets demonstrate the effectiveness of NeurOp-Diff. Our code is available at https://github.com/zerono000/NeurOp-Diff.

Multi-agent cooperative SLAM often encounters challenges in similar indoor environments characterized by repetitive structures, such as corridors and rooms. These challenges can lead to significant inaccuracies in shared location identification when employing point cloud-based techniques. To mitigate these issues, we introduce TWC-SLAM, a multi-agent cooperative SLAM framework that integrates text semantics and WiFi signal features to enhance location identification and loop closure detection. TWC-SLAM comprises a single-agent front-end odometry module based on FAST-LIO2, a location identification and loop closure detection module that leverages text semantics and WiFi features, and a global mapping module. The agents are equipped with sensors capable of capturing textual information and detecting WiFi signals. By correlating these data sources, TWC-SLAM establishes a common location, facilitating point cloud alignment across different agents’ maps. Furthermore, the system employs loop closure detection and optimization modules to achieve global optimization and cohesive mapping. We evaluated our approach using an indoor dataset featuring similar corridors, rooms, and text sign. The results demonstrate that TWC-SLAM significantly improves the performance of cooperative SLAM systems in complex environments with repetitive architectural features.

Land-air bimodal robots (LABR) are gaining attention for autonomous navigation, combining high mobility from aerial vehicles with long endurance from ground vehicles. However, existing LABR navigation methods are limited by suboptimal trajectories from mapping-based approaches and the excessive computational demands of learning-based methods. To address this, we propose a two-stage lightweight framework that integrates global key points prediction with local trajectory refinement to generate efficient and reachable trajectories. In the first stage, the Global Key points Prediction Network (GKPN) was used to generate a hybrid land-air keypoint path. The GKPN includes a Sobel Perception Network (SPN) for improved obstacle detection and a Lightweight Attention Planning Network (LAPN) to improves predictive ability by capturing contextual information. In the second stage, the global path is segmented based on predicted key points and refined using a mapping-based planner to create smooth, collision-free trajectories. Experiments conducted on our LABR platform show that our framework reduces network parameters by 14% and energy consumption during land-air transitions by 35% compared to existing approaches. The framework achieves real-time navigation without GPU acceleration and enables zero-shot transfer from simulation to reality during deployment.

This paper presents a lightweight bidirectional cable-driven ankle exoskeleton system (total mass: 2.6 kg) based on a series elastic actuation architecture (actuator module mass: 1.05 kg). The system utilizes a waist-mounted drive unit and Bowden cables to deliver bidirectional assistance to the ankle joint (nominal force: 460 N, peak force: 680 N). By integrating a dynamically coupled adaptive oscillator (AO), the system achieves robust gait synchronization across a range of walking speeds (0.6 – 1.8 m/s, phase estimation RMSE <2.48%, stride frequency estimation RMSE <0.1 Hz). This is complemented by a Gaussian Process (GP)-based torque planner and a cascaded torque control framework, ensuring seamless coordination with natural gait. Experimental characterization of the actuator demonstrates its high dynamic performance (torque bandwidth: 12.5 Hz) and low-impedance characteristics (peak passive backdrive torque: 0.97 N · m). Human trials involving five participants show that the system significantly expands the ankle joint range of motion (up to [-15.64 °, 20.67 ° ] at high speeds) while reducing peak muscle activation levels in the tibialis anterior (18.54%–30.21%) and gastrocnemius (19.34%– 25.45%). This design, combining lightweight construction with adaptive control strategies, provides a highly effective solution for daily mobility assistance and rehabilitation applications.

Time geography is an elegant framework for analyzing human activity–travel behaviors across temporal and spatial dimensions. At the core of this framework are the concepts of space–time paths, which represent historical trajectories, and space–time prisms, which define potential future activity spaces. However, most studies examine space–time paths and prisms in isolation, overlooking the integrative nature of time geography as a theoretical framework that incorporates the past, present, and future within a continuous temporal dimension. This study addresses this gap by developing novel methods for measuring and querying the spatiotemporal proximity of these heterogeneous and heterochronous time-geographic entities. To operationalize these methods, a GIS tool was implemented to support the local density analysis of time-geographic entities. Comprehensive computational experiments were conducted to validate the developed methods using large-scale, network-constrained, and time-geographic datasets. The developed methods exhibited high computational efficiency (approximately 2 s) in computing the local density for each path within extensive prism collections. The experimental results demonstrate the effectiveness of the developed methods in analyzing and visualizing the local density of historical paths in the near future. The methods offer new insights into space–time interactions, advancing research in human mobility forecasting and spatial decision-making.

Data mining of network-constrained trajectories has broad applications in the GIScience field. The calculation of a complete trajectory similarity matrix is a key step in various data mining algorithms. However, computing this matrix is computationally intensive for large datasets, as it involves numerous point-to-point shortest-path (PPSP) queries. To tackle this issue, we propose a new spatiotemporal construct called the space-time tree, which directly delineates the network distance from a query trajectory to any network space-time point. By constructing the space-time tree, we can efficiently compute the trajectory similarity matrix without additional PPSP queries. The space-time tree supports several similarity metrics, including closest pair distance, furthest pair distance, longest common subsequence (LCSS), and distance-weighted LCSS. It can further integrate with advanced spatiotemporal query techniques for scalable partial trajectory similarity matrix calculations. A case study using real datasets was conducted to apply the space-time tree in the trajectory clustering application. The results show that the space-time tree completed the clustering task on 0.5 million trajectories within 49 minutes, achieving a nearly 147-fold speedup compared to state-of-the-art methods.

A robust back-end module with loop closure detection is crucial for accurate positioning and mapping in LiDAR-based simultaneous localization and mapping (SLAM) systems, particularly in Internet of Things (IoT) environments where multiple devices collaborate. Traditional methods that rely on images or point clouds often fail in environments with similar structures or textures, leading to incorrect loop closures. To address this, we propose a novel LiDAR SLAM system that integrates a front-end odometry module, a loop closure detection module using text semantics and geometric constraints, and a global optimization module. By using cameras on an unmanned ground vehicle (UGV), the system captures text information from the environment, enabling semantic matching to identify potential loops. Geometric constraints help eliminate erroneous loops caused by identical text in different locations. Evaluations on datasets with similarly structured environments, such as indoor parking lots, outdoor campus areas, and mixed indoor–outdoor scenes, show that our method significantly improves loop closure detection accuracy and global precision compared to existing state-of-the-art approaches. Our research can support autonomous IoT systems and multiagent systems that rely on accurate positioning and mapping, with potential applications in embodied intelligence, self-driving cars, and smart cities.

Twisted light, characterized by its Orbital Angular Momentum (OAM) and helical phase front, holds great promise for advancing Free-Space Optical (FSO) communication systems by significantly increasing both capacity and spectral efficiency. However, Atmospheric Turbulence induces power fluctuations and wavefront distortions, degrading system performance. To address these challenges, we propose a Multi-Layered Irregular-Reconfigurable Intelligent Surface (MLI-RIS) architecture designed to enhance data rates and mitigate the detrimental effects of turbulence. To optimize system performance while minimizing computational complexity, a two-stage optimization framework is introduced: (1) MLI-RIS topology design, and (2) joint phase-shift and precoding optimization. This framework leverages a Diversification Tabu–Advanced Neighborhood Search (DT-ANS) algorithm, along with a Neighborhood Exploration Strategy integrated with Simulated Annealing (NES-SA). Furthermore, a compressive sensing-based correction method using regular RIS is implemented to compensate for wavefront distortions and enhance the purity of OAM modes. Numerical simulations validate the effectiveness of the proposed approach in reducing error rates, mitigating wavefront distortions and crosstalk, and enhancing overall system performance. The results show that the proposed system achieves a Forward Error Correction level with a 1.4 dB loss compared to the ideal case, whereas the best of the other approaches considered in this paper achieve this level with an 18 dB loss compared to the ideal case.

Reliable indoor localization is crucial for location-based services.Unlike outdoor environments where the Global Navigation Satellite System (GNSS) is prevalent, indoor localization systems employ diverse methods to enhance the accuracy of individual devices. However, these methods face limitations, such as the dependence on pre-existing map data and the necessity of installing anchors. The advancement of the Internet of Things (IoT) and the increasing availability of smart devices have enabled the development of more flexible and dynamic indoor localization solutions. In this paper, we propose a novel method to enhance indoor localization through cooperative localization framework. The core concept involves utilizing existing robots as mobile robot anchors to enhance pedestrian localization accuracy through interaction with pedestrians, particularly in environments lacking fixed anchors. We employed a factor graph optimization approach to tightly couple intra-device and inter-device data. This integration dynamically adjusts the inclusion of anchor data based on its quality, thereby minimizing error propagation. The experimental results demonstrate that the localization accuracy of our proposed method better than extend Kalman filter algorithms, emphasizing the potential of mobile IoT devices in indoor localization systems.

Challenges in remote sensing object detection (RSOD), such as high inter-class similarity, imbalanced foreground-background distribution, and the small size of objects in remote sensing images significantly hinder detection accuracy. Moreover, the trade-off between model accuracy and computational complexity poses additional constraints on the application of RSOD algorithms. To address these issues, this study proposes an efficient and lightweight RSOD algorithm integrating multi-scale receptive fields and foreground focus mechanism, named Robust Foreground Weighted Network (RFWNet). Specifically, we proposed a lightweight backbone network Receptive Field Adaptive Selection Network (RFASNet), leveraging the rich context information of remote sensing images to enhance class separability. Additionally, we developed a Foreground Background Separation Module (FBSM) consisting of a Background Redundant Information Filtering Module (BRIFM) and a Foreground Information Enhancement Module (FIEM) to emphasize critical regions within images while filtering redundant background information. Finally, we designed a loss function, the Weighted CIoU-Wasserstein loss (LWCW), which weights the IoU-based loss by using the Normalized Wasserstein Distance to mitigate model sensitivity to small object position deviations. The comprehensive experimental results demonstrate that RFWNet achieved 95.3% and 73.2% mAP with 6.0M parameters on the DOTA V1.0 and NWPU VHR-10 datasets, respectively, with an inference speed of 52 FPS.

High inter-class similarity and diverse geometric configurations in high spatial resolution remote sensing images pose significant challenges for remote sensing object detection. To address these challenges, this study proposed a novel remote sensing oriented object detector MGNet based on multi-cascaded feature selection and geometric constraints, designed to enhance discriminative feature representation and optimize oriented bounding box regression. First, we designed a novel backbone Multi-Cascaded Feature Selection Network (MCFSNet) to improve inter-class separability by jointly leveraging and fusing contextual information and multi-scale features. Second, we designed a Geometric-Constrained Probabilistic Loss, which introduced an angular offset factor and a geometric perception factor to refine angular prediction accuracy of square targets and shape sensitivity. To systematically evaluate angular localization performance, we constructed a Multi-scale Plane Remote Sensing Object Detection dataset (MPRSOD) with arbitrary-orientated annotations and proposed the Angle Error Rate (AER), the first dedicated evaluation metric for quantifying angular prediction errors. Extensive experiments on DIOR-R, DOTA V1.5, MAR20 and MPRSOD datasets demonstrate that MGNet obtains competitive mAP results while maintaining computational efficiency.

Unsupervised domain adaptation semantic segmentation for high spatial resolution remote sensing imagery aims to utilize a large amount of labeled data from the source domain to train a supervised model. This model is then adapted for pixel-dense prediction in the unlabeled target domain. Current methods in remote sensing unsupervised domain adaptation mainly focus on mitigating pseudo-label noise. However, they overlook the commonalities between pixel representations across domains. This oversight results in scattered semantic feature representations, thereby limiting segmentation accuracy. To address this issue, this study proposes a novel method prioritizes the extraction of learning universal features, domain-invariant representations spanning low-dimensional discriminative attributes and globally consistent spatial patterns. Specifically, a low-dimensional feature enhancement module is designed to increase feature diversity in shallow layers by employing two learnable scaling and shifting parameters. Additionally, a high-dimensional feature smoothing module uses Gaussian filtering to balance the distribution of global spatial features, enhancing feature learning in deep layers. Comprehensive evaluations on two remote sensing domain adaptation semantic segmentation datasets demonstrate that the superiority of our method, achieving state-of-the-art performance against contemporary baselines in terms of segmentation accuracy and feature alignment.

Accurate beach topography mapping is crucial for understanding coastal dynamics and mitigating climate change impacts. However, traditional methods such as airborne LiDAR have limitations, leading to substantial gaps in national-scale elevation data. This study presents an innovative framework to reconstruct missing elevation data along New Zealand’s coastline by integrating airborne LiDAR, Sentinel-2 optical imagery, and geometric features (distance) using machine learning methods. Our results show that Artificial Neural Network (ANN) emerged as the best model (test set: R²=0.79, RMSE=0.91 m; validation set: 0.79, RMSE=0.93 m), outperforming other models in accuracy. The produced 10-m DEM for national-scale sandy beaches expands area coverage by 286.6% (114.15 km²), filling gaps in 1249 beaches, including remote areas such as Stewart Island. This novel framework offers a scalable solution for improving the comprehensiveness and accuracy of beach topography. It provides essential support for inundation prediction, habitat management, and the development of climate adaptation strategies, thereby facilitating more informed decision-making in coastal zone management and climate change mitigation efforts.

Accurate and high spatiotemporal resolution water quality data are critical for the effective management of marine and coastal ecosystems. However, accurate atmospheric correction under high solar zenith angles (SZA) remains a challenge, introducing substantial uncertainties in satellite-derived water quality indicators (WQI) under high SZA. With an attempt to fill the gap, this study evaluated three types of strategies for satellite retrieval of suspended particulate matter (SPM) and chlorophyll-a (Chl-a) concentrations from top-of-atmosphere reflectance (ρt), Rayleigh-corrected reflectance (ρrc) and remote sensing reflectance (Rrs), respectively. The models, named XGBWQI, based on three types of remote sensing data were tested with in-situ data and compared with the Geostationary Ocean Color Imager (GOCI) standard algorithms. Results showed that: (i) ρt-based XGBWQI had the best accuracy (R2= 0.90 and MAPD = 14.65% for SPM, R2= 0.85 and MAPD = 5.34% for Chl-a); (ii) model testing results with in-situ data also confirmed the advantage of ρt-based XGBWQI over other models (R2= 0.88, MAPD = 26.9% and MRPD =11.8% for SPM, R2= 0.78, MAPD = 43.3% and MRPD = -15.5% for Chl-a); and (iii) the XGBWQI models obtained more valid WQI values for GOCI images under high SZA and successfully revealed the diurnal variations of a red tide event in the Yellow Sea and the SPM dynamics in the East China Sea. Therefore, ρt-based XGBWQI models were recommended as the best strategy for satellite retrievals of WQI under high SZA. The methods can serve as an effective tool in retrieving WQI in coastal waters under high SZA, and thus contribute to better and high-frequency water quality monitoring.

The lunar surface presents an exceptionally harsh environment, while shallow subsurface cavities offer critical protection against harmful radiation, extreme thermal fluctuations, and meteorite impacts. This study integrates Diviner temperature data, LOLA albedo data, and radar-derived density profiles to perform thermal simulations of the subsurface cavity detected by ground-penetrating radar within five meters beneath the lunar near-side surface at the Chang’E-3 landing site. The primary objective is to assess its feasibility and thermal stability as a natural shelter for lunar base site selection. This paper employs the finite element method to simulate diurnal temperature variations in both intact cavities and excavated cavities with varying opening diameters. The results show that temperature fluctuations within the cavity decrease as the skylight diameter narrows. The internal thermal environment demonstrates relatively high stability, attributable to the cavity’s location in a mid-latitude region with limited solar radiation exposure and to the excellent thermal insulation properties of lunar regolith. Even in cavities with a 2-meter skylight diameter, the internal diurnal temperature variation remains within 20 K. These findings confirm the thermal feasibility of utilizing such cavities in the Chang’E-3 landing area as natural shelters for human habitation and provide valuable insights and recommendations for future lunar base siting.

The high risks in tunnel construction underscore the critical necessity for intelligent tunnel construction. Unmanned tunnel data collection is vital for intelligent construction; additionally, semantic segmentation aids in understanding the environment. However, complex tunnel terrains are challenging for three-dimensional (3D) laser scanning, and diverse interior structures and nontunnel elements complicate accurate segmentation by subsequent networks. Therefore, this paper proposed a tunnel mobile 3D mapping system (TMMS) for complex terrain in construction tunnels using a quadruped robot and simultaneous localization and mapping (SLAM). Additionally, a deep learning-based semantic segmentation network (TUC-Net) is proposed for analysing 3D point clouds in tunnels under construction. The research presented the neighbourhood feature perception enhancement (NFPE) module to enhance the representation of local features, introduce a self-attention (SA) module and improve the loss function to improve network accuracy. The NFPE module enhances feature aggregation for unstructured objects, and the SA module improves the learning of global features, critical for tunnel point cloud segmentation. The TMMS is used to collect point cloud data from tunnels under construction, leading to the creation of the tunnels under construction point clouds (TUCPC) dataset for training and evaluating the TUC-Net network. Compared to other 3D point cloud semantic segmentation methods, the proposed method demonstrated superior performance, achieving an overall accuracy (OA) of 99.45% and a mean intersection over union (mIoU) of 94.06%, surpassing that of other methods by at least 4.41%. In addition, ablation studies were also performed on the NFPE and SA modules to validate their efficacy.

Interferometric Synthetic Aperture Radar (InSAR) has become a widely used and efficient tool for monitoring large-scale, long-term land subsidence. Most applications, especially those related to risky assessment, utilize only the mean deformation rate derived through a linear fit of the InSAR-derived time series deformations without considering the full-time sequence. In this way, the embedded information, such as the onset, duration, and patterns of abnormal deformations, is ignored. This information, however, should not be compromised as it is critical for assessing the risky deformations and recognizing the causal factors. Due to the large data volume, processing the full-time-series information derived by InSAR analysis can be a challenge. In this study, we propose to use the Time-series Anomaly Detection Generative Adversarial Network (TadGAN) to deal with the time series InSAR deformation and recognize the onset and duration of abnormal epochs. The proposed method has been tested with InSAR-derived results over the Hong Kong airport region. It performs better than conventional risk assessment methods, such as the one based on Mean Absolute Deviation (MAD) outlier detection.

This study, conducted in a Shenzhen district, assesses the necessity and cost-effectiveness of Ground Penetrating Radar (GPR) road inspections. The evaluation involves automatic identification of road defects, such as voids, cavities, and loose soil, followed by an analysis of their spatial distribution patterns, clustering effects, density, and correlation with the metro system. The study proposes an optimized strategy for large-scale road inspections based on the spatial distribution characteristics of identified defects, emphasizing the importance of more frequent inspections on vulnerable roads. The findings contribute to the efficient allocation of public funds, minimizing wasteful expenditures, and enhancing road operational safety.

Interferometric Synthetic Aperture Radar (InSAR) has been recognized as an effective remote sensing tool for Earth Observation (EO) thanks to its capacity to penetrate clouds and forests. In the field of InSAR deformation monitoring, coherence is used as an indicator for the stability of the interferometric phase. Actually, interferometric coherence itself contains valuable information about land surface and can be used to classify land cover types. Currently, there are few studies about the possibility of retrieving forest canopy height from interferometric coherence, especially its multitemporal variation. In this study, we emphasize the relationship between multi-temporal InSAR coherence and forest canopy height by investigating the InSAR coherence variation of multi-temporal Sentinel-1 data regarding the tree height measured by the NASA Global Ecosystem Dynamics Investigation (GEDI). Results show that forest canopy height can be estimated by using the Random Forest (RF) regression model trained by samples of GEDI height and InSAR coherence variation. InSAR coherence performs better in estimating the forest canopy height than using other remote sensing data, such as multispectral reflectance from Sentinel2 and SAR backscatter intensity.

Analyzing road network subsidence and sinkholes is crucial for ensuring urban traffic and people's safety. InSAR technique is a non-contact measurement technique with high spatiotemporal resolution, wide monitoring range, and unaffected by road conditions, which is of great significance for the analysis of deformation hazards of man-made linear infrastructures, such as road networks and high-speed railways. In this study, we adopt 44 Sentinel-1A images from January 2022 to July 2023 to study the deformation of the road network in Longgang District, Shenzhen, China based on PS-InSAR. The road deformation is further analyzed associated with sinkhole information from field investigation. The results show that the deformation rate of the road network is between -38.3 mm/year and 16.9 mm/year, and the correlation coefficient between the top burial depth of the sinkholes and the maximum deformation rate is 0.58.

There is a growing interest in the optimization of vehicle fleets management in urban environments. However, limited attention has been paid to the integrated optimization of electric taxi fleets accounting for different operations as well as complex spatiotemporal demand dynamics. To this end, this study develops a real-time recommendation framework based on deep reinforcement learning (DRL) for electric taxis (E-taxis) to improve their system performance with explicit modeling of multiple vehicle actions and varying travel demand across space and over time. Spatiotemporal patterns of urban taxi travels are extracted from large-scale taxi trajectories. Spatiotemporal strategies are proposed to coordinate E-taxis’ repositioning and recharging with optimized recommendation for next destinations and charging stations. A spatiotemporal double deep Q-network (ST-DDQN) is embedded in the DRL framework to maximize the daily profit. A prototype real-time recommendation system for E-taxis is implemented for the decision-making of E-taxi drivers and sensitivity analyses are carried out. The experimental results in Shenzhen, China suggest that the proposed framework could improve the overall performance. This study will benefit the promotion of connected E-taxis and the development of clean and smart transportation.

Social distancing and contact tracing are effective nonpharmaceutical means to ensure public safety and control the rapid spread of infectious diseases. Internet of Things (IoT) sensors can provide reliable data sources for contact tracing, especially in urban public areas. However, existing contact tracing studies mainly use 2-D coordinates or distances to detect direct contact between people and lack indirect contact behavior modeling and sensing in urban 3-D environments. Additionally, an efficient storage and spatiotemporal search method is required to find unsafe contact cases from the large amount of data collected by IoT sensors. This article proposes an innovative spatiotemporal detection and tracing framework for both direct and indirect contact behavior using multicamera sensors. A multicamera coordinate conversion model (M-CCM) is designed to achieve camera calibration and 3-D trajectory aggregation based on a spatiotemporal constraint strategy. Using the aggregated 3-D trajectories, this method further defines several fine-grained characteristics of both direct and indirect contact behavior. A contact graph is designed to model and represent contact activities, which supports efficient spatiotemporal searching and tracing of unsafe contact activities. We have verified the performance of the proposed framework using both a public data set and our data set. Experiments demonstrate that the 3-D trajectory coordinate conversation accuracy was 0.2 m using the proposed M-CCM. The social distance and contact time detection precision and recall are 80% and 93%, respectively, for close contact (<1.5 m). Furthermore, the contact graph can reduce the memory space for monitoring data storage and support efficient spatiotemporal contact tracing.

Localization information is increasingly crucial for incorporating location context into Internet of Things (IoT) data. As an important task in visual localization, keyframe selection helps effective augmentation of visual odometry. Although considerable progress has been made in the research field of keyframe selection, they have rarely focused on dealing with degraded input sensory data in the real world. To this extent, this work proposes a novel concept by incorporating image quality evaluation into the visual localization so that the keyframe selection module can identify images that may cause undesirable effects and take measures to avoid the catastrophic impact of degraded images. The quality for each image is estimated online using deep classifier trained with the image-itself, image-differential, and external information. Since no model-specific knowledge is needed, our method is applicable to any visual localization system. By creating a challenging dataset based on current public datasets under autonomous driving and unmanned aerial vehicles (UAV) scenarios and using it to evaluate our method, we obtain estimated trajectories that are closer to the original situation while validating its robustness to challenging degraded environments.

Visual perception technology is an important means to facilitate safe navigation for visually impaired people based on Internet of Things (IoT)-enabled camera sensors. However, due to the rapid development of urban traffic systems, traveling outdoors is becoming increasingly complicated. Visually impaired individuals must implement different types of tasks simultaneously, such as finding roads, avoiding obstacles, and viewing traffic lights, which is challenging for both them and navigation assistance methods. To solve these problems, we propose a multitask visual navigation method for visually impaired individuals using an IoT-based camera. A lightweight neural network is designed, which adopts a multitask learning architecture to perform scene classification and path detection tasks simultaneously. We propose two modules, i.e., an enhanced inverted residuals (EIR) block and a lightweight Vision Transformer (ViT) block (LWVIT block), to effectively combine the properties of convolutional neural networks (CNNs) and ViT networks. The two modules allow the network to better learn local features and global representations of images while remaining lightweight. The experimental results show that the proposed method can achieve these tasks simultaneously in a lightweight manner, which is important for IoT-based navigation applications. The accuracy of our method in scene classification reaches 91.7%. The path direction and endpoint detection errors are 6.59∘ and 0.09, respectively, for blind road and 6.81∘ and 0.06, respectively, for crosswalk. The number of parameters of our method is 0.993 M, which is smaller than that of the comparison methods. An ablation study further demonstrates the effectiveness of the proposed method.

To improve the accuracy and efficiency of LiDAR mapping, cooperative simultaneous localization and mapping (SLAM) has been considered for complex large scenes. Recognizing the same positions and detecting global loop closures are important for achieving cooperative SLAM. However, most of the current position recognition and loop closure detection methods are based on images or point clouds. These methods may make mistakes if structures or textures are similar. To overcome this problem, we propose SemanticCSLAM, which is a Cooperative SLAM system that uses environment semantic landmarks for position recognition and loop closure detection. The proposed SemanticCSLAM consistes of a single SLAM module based on A-LOAM, a trajectory alignment module and a global optimization module based on environment landmarks. Through the inertial measurement unit (IMU) carried by the agent, such as an unmanned ground vehicle (UGV), the environment landmarks can be detected. Based on these environment landmarks, the alignment module aligns trajectories from different agents. Finally, the loop closure detection and optimization module performs loop closure detection and global optimization based on these environment landmarks. We collected a dataset, which contains indoor and outdoor data, for testing. These experimental results in different scenes show that the environment landmarks can effectively improve the performance of cooperative SLAM systems.

With the coming of the era of Internet of Everything and 5G, streaming data access via a uniform approach is a great challenge with metadata heterogeneity, streaming characteristics, and data inconsistency. Therefore, streaming data access via a uniform approach is a great challenge. In our study, we propose a unified and efficient streaming vehicle data access model (SVASM) to uniform the heterogeneity of streaming vehicle spatiotemporal data. The proposed meta-model has been divided into a nonstreaming meta-model and a streaming meta-model according to the static or dynamic characteristics. The non-streaming metamodel is composed of basic data types, which are represented by seven tuples. Dynamic elements contain observation parts, and dynamic elements of streaming vehicle metadata consist of a streaming meta-model. By extending the Observations & Measurements (O&M) encoding profile, the streaming characteristics are supplemented, and the mandatory elements and extending elements have been defined. To achieve its practical application, the corresponding massive vehicle data access algorithm based on Apache Storm has been proposed. In experiments, data access via MongoDB is an ideal data access method with good performance, as applied in our tests. As the experiments demonstrates, the efficiency of SVASM is approximately 2.5 times that of MongoDB in entire or single bolts. In addition, the data access via different bolts can achieve an adaptive, customized, flexible access mechanism. Consequently, the proposed model is a unified, adaptive, efficient data access model for massive streaming vehicle data under an Internet of Things sensor network.

Precise deformation surveying is essential for ensuring safety and sustainable development in densely populated cities. Synthetic aperture radar interferometry (InSAR) is a technique well suited for large-scale deformation measurement. It is widely applied in monitoring geological disasters such as earthquakes, landslides, and volcanoes. However, it faces challenges when used to monitor urban infrastructure, which requires higher precision and more details. To overcome these challenges, we introduce an innovative, object-oriented multitemporal InSAR (Obj-InSAR) framework. Compared with state-of-the-art InSAR methods, Obj-InSAR shifts the basic processing unit from pixel to object. This allows the use of additional information provided by the object/segment to enable detailed and precise results, as required in urban scenarios. In this framework, a novel object-based plug-in network is designed to establish an impressively dense network for parameters solving. Additionally, a new object-based adjustment procedure through 3-D point cloud alignment is proposed to accurately determine the location of InSAR scatterers and guide network construction. As a result, our approach produces a more detailed, accurate and precisely geolocated 4-D infrastructure point cloud than conventional pixel-oriented frameworks, enabling effective monitoring of urban deformation. A case study confirms the usefulness and adaptability of this approach in urban SAR interferometry. These advancements will contribute to the development of time-series InSAR technology in the new era, enhance the capability of monitoring complex urban deformation in high-density cities, and serve urban safety operations and sustainable economic development.

Accurate satellite retrieval of oceanic particulate organic nitrogen (PON) concentrations could contribute to a better and more comprehensive understanding of global marine biogeochemical processes. However, no satisfactory satellite PON retrieval model could be found in the literature. In an attempt to develop applicable PON models, large diverse matchups of synchronous global oceanic in situ PON measurements and ocean color satellite data were used to develop PON retrieval models using the Gaussian process regression (GPR) method for SeaWiFS, Terra-Moderate Resolution Imaging Spectroradiometer (MODIS), MERIS, Aqua-MODIS, and SNPP-Visible Infrared Imaging Radiometer Suite (VIIRS), respectively. The GPR PON models were compared with polynomial PON models based on single bio-optical properties or band index, and further used to retrieve spatiotemporal variations of global oceanic PON concentrations. Combined with the satellite-derived particulate organic carbon (POC) products, the possibility of deriving the POC to nitrogen ratio (POC:PON) was further explored. Results showed that GPR PON models, with$R^{2}$, root mean square error (RMSE), and mean absolute percentage error (MAPE) ranging from 0.76 to 0.87, 0.15 to 0.18, and 9.19% to 12.59%, respectively, had a comparable performance for both Case-1 and Case-2 waters and outperformed the polynomial PON models. The global PON concentrations derived from different satellite missions were generally consistent, with their mean relative differences all less than 14.70% between Aqua-MODIS and the other four missions. The GPR-derived POC:PON was acceptable, with most relative errors within ±40% compared to in situ POC:PON. The applicable satellite PON retrieval models and the readily available satellite PON products should be helpful for studying oceanic PON dynamics and ecological processes in marine biogeochemical cycles.

Thermal images capture temperature information of the environments instead of texture, making it well suitable for obtaining position in dark environments. Many methods have been proposed to handle RGB images, while thermal image-based localization methods are not well studied. To address it, we propose DarkLoc+, a thermal image-based indoor localization method based on the attention model and relative constraints between images under a learning-based localization framework. To be specific, we utilize self-attention to extract reprehensive features from thermal images and exploit relative constraints to enforce the convolutional neural networks (CNNs) to predict global poses. Relative pose loss(RelLoss)and relative regression loss are designed to work with global poses to constrain the network in feature and pose space simultaneously. We evaluate the proposed method on the public thermal images indoor dataset and our own dataset. The experimental results demonstrate that our method can obtain accurate position information.

Ground-based interferometric radar (GBIR) is an innovative tool for monitoring land surface subsidence and urban infrastructure deformation caused by rapid urbanization. However, the interferograms of GBIR are often contaminated by severe atmospheric effects, especially in coastal areas. In this study, we use independent component analysis (ICA) to extract atmospheric effects for the interferograms of GBIR. Analysis of the performance of ICA and traditional surface fitting methods have been carried out. The results suggest that the average improved rate of ICA is 93.05%, which is 69.33% higher than that of surface fitting.

Clustering the trajectories of vehicles moving on road networks is a key data mining technique for understanding human mobility patterns, as well as their interactions with urban environments. The development of efficient and scalable trajectory clustering algorithms, however, still faces challenges because of the computational costs when measuring similarities among a large number of network-constrained trajectories. To address this problem, a novel trajectory clustering framework based on the well-developed Density-Based Spatial Clustering of Applications with Noise (DBSCAN) approach is proposed. This proposed framework accurately quantifies similarities using a trajectory representation of continuous polylines in the space and time dimensions, and does not require trajectory discretization. Further, the proposed framework utilizes the space-time buffering concept to formulate ε-neighborhood queries that directly retrieve the ε-neighbors of trajectories and thus avoids computing a trajectory similarity matrix. State-of-the-art trajectory databases and index structures are incorporated to further improve trajectory clustering performance. A comprehensive case study was carried out using an open dataset of 20,161 trajectories. Results show that the proposed framework efficiently executed trajectory clustering on the large test dataset within 3 min. This was approximately 2,700 times faster than existing DBSCAN algorithms.

The availability of Spatiotemporal Big Data has provided a golden opportunity for time geographical studies that have long been constrained by the lack of individual-level data. However, how to store, manage, and query a huge number of time geographic entities effectively and efficiently with complex spatiotemporal characteristics and relationships poses a significant challenge to contemporary GIS platforms. In this article, a hierarchical compressed linear reference (CLR) model is proposed to transform network-constrained time geographic entities from three-dimensional (3D) (x, y, t) space into two-dimensional (2D) space. Accordingly, time geographic entities can be represented as 2D spatial entities and stored in a classical spatial database. The proposed CLR model supports a hierarchical linear reference system (LRS) including not only underlying a link-based LRS but also multiple higher-level route-based LRSs. In addition, an LRS-based spatiotemporal index structure is developed to index both time geographic entities and the corresponding hierarchical network. The results of computational experiments on large datasets of space–time paths and prisms show that the proposed hierarchical CLR model is effective at storing and managing time geographic entities in road networks. The developed index structure achieves satisfactory query performance in milliseconds on large datasets of time geographic entities.

Understanding the relationship between mixed land use and urban vibrancy is vital in advanced urban planning applications. This study presents a Bayesian spatially varying coefficient (SVC) model to explore the spatially nonstationary relationship between mixed land use and urban vibrancy after controlling for other factors. We first use the convolutional conditional autoregressive prior to accommodate the ecological bias resulting from unobserved confounders. Then we develop our approach in the case of a single predictor to allow the spatially varying coefficient process. We further introduce a type of the Bayesian SVC model that considers the stratified heterogeneity of the outcome, allowing the coefficients to simultaneously vary at the local and subregion level. We illustrate the proposed model by conducting a case study in Shenzhen using mobile phone data, an officially registered point-of-interest (POI) dataset, and several supplementary datasets. The model evaluation results show that including spatially unstructured and structured component combinations can improve the model's fitness and predictive ability; additionally, considering spatial stratified heterogeneity can further enhance the model's performance. Our findings provide an alternative for measuring the variable local-scale association between mixed-use and urban vibrancy and offer new insights that broaden the fields of environmental science and spatial statistics.

Existing traffic monitoring approaches cannot completely cover all road segments in real-time, leading to massive amounts of missing traffic data, which limits the implementation of intelligent transportation systems. Most existing methods lack deep mining of the unique spatiotemporal characteristics of traffic flows, resulting in difficulty in application to urban traffic with complex topologies and variable states. In this paper, we propose a novel Spatio-Temporal constrained Low-Rank Tensor Completion (ST-LRTC) method, which adopts a manifold embedding approach to depict the local geometric structure of spatiotemporal domains. Specifically, under the low-rank assumption, the method introduces temporal constraints based on the continuity and periodicity of traffic flow and a spatial constraint matrix reflecting the traffic flow transmission mechanism. We embed low-dimensional spatiotemporal constraint matrices into the low-rank tensor completion solving process to fully utilize the global features and local spatiotemporal characteristics of the traffic tensor. Experiments were performed using traffic data from Xi’an, China, and the results indicated that ST-LRTC outperformed state-of-the-art methods under various missing rates and patterns. Thorough experiments have demonstrated that the incorporation of spatiotemporal analysis can enhance the adaptability of the tensor completion model to complex urban scenarios, which guarantees better monitoring, diagnosis, and optimization of urban traffic states.

Waypoint estimation (WE) has a wide range of applications for indoor walkers, such as fire rescue and navigation to find exit doors, lifts, or stairs as examples of waypoints, etc. Data-driven WE has been on the rise with advancements in deep learning algorithms. The current WE methods, however, face two challenges. On the one hand, most waypoint detection approaches rely on visual sensors, hence, their estimation performance is limited by light when collecting visual data. On the other hand, data-driven methods necessitate a large number of labeled data to train a WE model, which significantly increases the time spent manually marking labels. Targeting the above two challenges, our work first proposes a novel deep Bayesian active learning waypoint estimator for indoor walkers (DeepWE) based on human activity recognition (HAR). This estimates six indoor waypoints through walkers’ daily activities due to the strong correlation between human activities and waypoints. First, an initial DeepWE model is developed using a Bayesian ensembled convolutional neural network (B-CNN) using the accelerometer and gyroscope data. Then, active learning is employed to query the most formative samples from pool points with four acquisition functions, and only these queried samples are labeled manually. Finally, the initial DeepWE model is updated from this labeled data using an incremental learning algorithm. Empirical results on two publicly available USC-HAD and OPPORTUNITY data sets show DeepWE performs a considerable accuracy boost for WE, with a substantial amount of acquired pool points reduction (more than 40%).

In recent years, several novel satellite platforms and sensors have been proposed for the Earth Radiation Budget (ERB). Simulating the sensor-measured signals could be helpful for optimizing the settings of sensors and exploring their potential in ERB. The anisotropic factor, depicting the anisotropy of Earth’s radiation, is essential in the simulation. However, developing angular distribution models involves complex procedures of data preparation, processing, and modeling. This study, targeting at simplifying the procedure of simulating the signals of ERB sensors, proposed a suit of models for estimating the longwave anisotropic factors directly from the Earth’s radiative fluxes. The models were developed with CERES/Terra data sensed in rotating azimuth plane (RAP) mode during 2000-2005 and the artificial neural network (ANN) algorithm, and tested with 12 monthly of CERES/Terra data collected in RAP and cross-track mode during 2021-2022, respectively. Models were developed for 10 scene types based on Earth’s surface types, and compared with the operational ANN ADMs. Results showed that the longwave anisotropic factors were accurately estimated with the correlation coefficient (r) varying between 0.84 and 0.98 and MAPE within 1.20% for the test dataset, and the approach proposed in this study had comparable performance with the ANN ADMs. With the estimated anisotropic factors, the sensor-measured radiances were accurately retrieved with r=1.00 and MAPE=0.53%. Therefore, the proposed approach is promising in accurate and efficient simulations of novel ERB platforms and sensors like the Moon-based Earth Radiation.

High-quality 3D point cloud maps are essential for precise indoor environments modeling. However, constructing such maps in multi-storey indoor environments is challenging due to the presence of narrow non-structural spaces, such as staircases, corners, and corridors with similar textures. Simultaneous localization and mapping (SLAM) in these scenes is particularly difficult, as cumulative errors can lead to incorrect loop closures and drastic degradation in map quality. To address these challenges. This paper proposed a SLAM method base on multiple ground constraints pose optimization (MGCPO) which uses a backpack LiDAR system. The proposed method includes two novel modules. The first, a regression analysis-based scenarios recognition (RASR) module provides a reference for the construction of ground constraints. The second, based on different scene detection results, the MGCPO module constrains the sensor pose using the floor plane to reduce localization errors and effectively decrease loop closure detection errors. Qualitative experiments demonstrate that our proposed method outperforms state-of-the-art methods in challenging scenarios. Quantitative experiments show that our method achieves an error rate of just 1.06% using only LiDAR sensors.

Image-based localization is an essential component for many applications such as autonomous driving, virtual reality. Many researchers focus on developing methods for daytime via RGB images. Few research study the methods for night condition. The main reason is that the RGB-based image localization methods fail to work in dark scenes due to the low illumination. Thermal images capture temperature information instead of texture, making it well suitable for dark environments. However, thermal image-based localization methods are not well studied. To address it, we propose an attention-based localization method for night condition (DarkLoc) using thermal images. The proposed method introduce the attention mechanism in a deep learning-based framework to extract key information from low quality thermal images. The attention model can enforce the whole network focus on geometry meaningful feature in thermal images and thus improve the localization accuracy. We perform extensive experiment on the thermal image dataset. The results show that the attention model can enforce the whole network to learn geometry meaningful feature from thermal images and effectively locate the thermal image in real-time.

With the development of quadrotor-based location services, accurate indoor quadrotor localization plays an important role in various applications. Tight fusion refers to the process of integrating multisensor data into state estimation for optimization, and finally obtaining pose information. In this article, we propose a novel tight fusion method for quadrotor localization by fusing the WiFi round-trip time (RTT) and built-in smartphone microelectromechanical sensors. Unlike existing 3-D localization frameworks, the key contribution of the proposed method is to integrate 3-D outlier detection, state estimation, coordinate frame alignment, and data fusion into a nonlinear filtering framework. Specifically, this method is divided into four main steps: 1) the coordinates of the mobile phone and the quadrotor are converted to the same coordinate system through the coordinate alignment method we propose; 2) the proposed outlier detection method is used to obtain the 3-D coordinates of the quadrotor based on WiFi RTT; 3) the WiFi RTT localization results are integrated into an error-state Kalman filter (ESKF) to perform the integrated localization of the quadrotors; and 4) a Rauch–Tung–Striebel (RTS) smoothing method is used to optimize the localization results. The experimental results demonstrate that the proposed method outperforms the classic localization method in terms of both accuracy and robustness.

Standard remote sensing observation (RSO) access and formulization is essential to Internet of Things (IoT) data management, such as in precision agriculture (PA). Because of the heterogeneous characteristics and the petabyte data size of RSO, massive remote sensing processing in RSO management has been hampered. Here, we present a heterogeneous access metamodel for efficient RSO management (HAMERM) and verify it in PA. The structure of basic metadata components is defined. A five-tuple metadata structure based on the metaobject facility is designed. HAMERM consists of identification, platform, observation, product, and access, which represent the five aspects of RSO metadata information. In addition, the flatMap/reduceByKey algorithms and the table structure have been proposed under Sensor Web and Geographic Information Science (GIS) techniques. Intensive experiments in Guangdong Province, China are conducted to test the proposed method. Two RSO metadata formulization instances were conducted to examine the ability of sheltering the differences of multisource and heterogeneous RSO. Experiments containing data storage and data soil moisture (SM) mapping were performed. The results suggest that the HAMERM method achieved a performance 30.1 times higher than that of Hadoop and three times higher than that of Spark (stand-alone). Consequently, the proposed HAMERM can be applied to achieve efficient SM mapping within PA, which is helpful for efficient RSO management for the IoT.

Fusing a Hyperspectral image (HSI) and a multispectral image (MSI) from different sensors is an economic and effective approach to get an image with both high spatial and spectral resolution, but localized changes between the multiplatform images can have negative impacts on the fusion. In this letter, we propose a novel sparsity constrained fusion method (SCFus) to fuse multiplatform HSIs and MSIs based on matrix factorization. Specifically, we imposed$\ell _{1}$norm on the residual term of the MSI to account for the localized changes between the hyperspectral and MSIs. Furthermore, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients by exploiting the plug-and-play framework. We refer to the proposed method as SCFus for hyperspectral and MSIs. Experimental results suggest that the proposed fusion method is more effective in fusing hyperspectral and MSIs than the competitors.

Nighttime light (NTL) radiance can reflect human settlements and activities. A lot of studies indicate that NTL brightness can be a sound proxy of socioeconomic factors on large scales, such as the population size, gross domestic product, electric power consumption, etc. However, few studies have been dedicated to these topics on fine scales. In this study, we examined the correlation between the Suomi National Polar-orbiting Partnership-Visible Infrared Imaging Radiometer Suite (S-NPP/VIIRS) NTL intensity (NLI) and population density, as well that between NLI and per capita income at two census units’ levels, census tract and block group, in two research areas. The result shows that the NLI has a moderate or weak positive correlation with the population density at both scales. However, when land use type is integrated with population density, the correlation becomes very strong. The NLI and per capita income have a weak or very weak negative correlation at both scales. Moreover, we find that the correlation coefficient is positively correlated with the unit’s size. The larger scale also has a higher correlation coefficient. The research conducted here could be beneficial for the application of S-NPP/VIIRS NTL data in studying socioeconomic activities in human settlement areas.

Interferometric synthetic aperture radar (InSAR) has been more and more applied in acquiring long-term deformation of land surface in a large coverage and is becoming a routine investigation technique. Validation of the InSAR results depends largely on thein situmeasurements. These measurements are usually point-wise and of high cost, as many sensors need to be set up for the long-term monitoring. In applications associated with a large area, qualitative and low-cost validation may be more necessary at the first place, which is still a challenge. In the recent decade, the crowdsourcing and volunteered geographic information (VGI) have been more and more accepted in the field of geoinformatics. Inspired by these, this letter proposes an InSAR crowdsourcing annotation system to integrate the InSAR displacements and the photographs uploaded by volunteers. We processed 119 Sentinel-1A data ranging from 2017 to 2021 to derive long-term displacements. Then, based on the InSAR displacements, task areas were selected and published to public via the system. Volunteers online accepted the task and uploaded photographs, indicating land displacements. In a coastal city, Shenzhen of China, 135 task areas were selected and published in total, and 1742 useful photographs were uploaded. The uploaded photographs were then inspected to validate and analyze the InSAR results. Post-analysis found some high correlation between the uploaded photographs with the InSAR alerting displacements. The proposed system is a prototype, and its interface and functions can be further extended in the future toward an effective and efficient alerting system for risking land deformations.

Landsat images have played an important role in the field of Earth observation and geoinformatics. However, optical Landsat images are frequently contaminated by cloud cover, especially in tropical and subtropical regions, which limits the utilization of these images. To improve the utilization of Landsat images, in this study, we propose a novel spatiotemporal neural network with four modules: a cloud detection module, a spatial–temporal learning module, a spatial–temporal feature fusion module, and a reconstruction module. The results of the experiments demonstrate that the proposed method is quantitatively effective (root mean square error < 0.0179) and can achieve a better result for reconstructing Landsat images than some of the widely used existing deep learning methods and multitemporal methods. The proposed neural network method provides an effective tool for the removal of contiguous, thick clouds from satellite images, so as to improve the quality of subsequent remote sensing mapping and geoinformation extraction.

The high spectral resolution of hyperspectral images (HSIs) generally comes at the expense of low spatial resolution, which hinders the application of HSIs. Fusing an HSI and a multispectral image (MSI) from different sensors to get an image with the high spatial and spectral resolution is an economic and effective approach, but localized spatial and spectral changes between images acquired at different time instants can have negative impacts on the fusion results, which has rarely been considered in many fusion methods. In this article, we propose a novel group sparsity constrained fusion (GSFus) method to fuse hyperspectral and MSIs based on matrix factorization. Specifically, we imposed$\ell _{2,1}$norm on the residual term of the MSI to account for the localized interimage changes occurring during the acquisition of the hyperspectral and MSIs. Furthermore, by exploiting the plug-and-play framework, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients. We refer to the proposed fusion method as GSFus method. We performed fusion experiments on two kinds of datasets, i.e., with and without obvious localized changes between the HSIs and MSIs, and a full resolution dataset. Extensive experiments in comparison with seven state-of-the-art fusion methods suggest that the proposed fusion method is more effective on fusing hyperspectral and MSIs than the competitors.

Airborne laser bathymetry (ALB) systems with digital full-waveform signal collection can obtain corresponding temporal positions from several backscattering surfaces by laser beam irradiation. This information can help describe the multi-elevation structures of the target and explore the echo signal attenuation response in different nonuniform mediums during laser propagation. Therefore, a full-waveform echo signal is quite practical for integrated water-land detection. However, the wavelength used in the ALB system is generally in the visible band range of 470~580 nm, and the received signal is constantly interfered with by many nontarget factors, such as imperfections in the receiving channel or the strong scattering from the transmission medium. The conventional processing method transforms nontarget interference into noise point cloud filtering or classification extraction, enabling the detection of a single surface or regular geometry. The accuracy of the identification and extraction for multi-elevation target surfaces echo signal is always reduced due to the significant noise signal intensity. We proposed a signal component detection method by constructing the echo signal feature functions and the conditional random field (CRF) model based on the full-waveform decomposition. The processing result for actual measurement data verified that the CRF strategy can effectively reduce the uncertainty of target surface detection. Compared with the single-beam echo sounder, the root mean square errors of the elevation deviation underwater were reduced by 3.2 cm and 4.9 cm respectively in the two different experimental areas.

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available athttp://jiasen.tech/papers/.

Domain adaptation, which cleverly applies the classifier learned from the source domain with sufficient labeled samples to the target domain with limited labeled samples, provides a feasible alternative to handle the small training sample problem of hyperspectral image (HSI) classification and has attracted much attention in the research field recently. Apparently, feature discriminative ability is vital for domain adaptation, which plays a crucial role during the migration process of transfer learning. In this article, a gradient feature-oriented 3-D domain adaptation (GF-3DDA) approach is proposed for HSI classification. First, 3-D Gabor is employed to remove noise from the original data, and two 2-D gradient-based features, 2-D Sobel gradient (SG) and 2-D derivative-of-Gaussian (DtG), are extended to the 3-D domain to coincide with the integrated spatial–spectral organization of HSI. Thus, the 3-D Sobel–Gabor gradient (3DSGG) and 3-D derivative-of-Gaussian-Gabor (3DDGG) features are achieved. Second, a 3-D domain adaptation method is implemented to jointly exploit the second- and fourth-order statistical descriptors in the spatial–spectral dimensions, which could effectively reduce domain shifts and thus achieve improved domain adaptation. Third, all the extracted domain-adapted feature modules are collaboratively classified by extreme learning machine (ELM), and the probability-like outputs of every ELM classifier are combined together to accomplish the classification task. Four hyperspectral data sets that each contains two scenes, i.e., Pavia, Shanghai–Hangzhou, Indiana, and Houston, are tested in the experiments. When only ten labeled samples per class are used in the target domain, the classification accuracies on four hyperspectral data sets achieved by our GF-3DDA approach are 93.31%, 84.35%, 69.32%, and 80.06%, respectively.

Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost of acquisition equipment, thereby providing a feasible way to improve the quality of remote sensing images. Clearly, image SR is a severe ill-posed problem. With the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. Since the texture information of various remote sensing images are totally different from each other, in this paper, we proposed a network based on generative adversarial network (GAN) to achieve high resolution remote sensing images, named multi-attention generative adversarial network (MA-GAN). The main body of the generator in MA-GAN contains three blocks: pyramid-convolutional residualdense (PCRD) block, attention-based upsampling (AUP) block and attention-based fusion (AF) block. Specifically, the developed attention-pyramid convolutional (AttPConv) operator in PCRD block combines multi-scale convolution and channel attention (CA) to automatically learn and adjust the scale of residuals for better representation. The established AUP block utilizes pixel attention (PA) to perform arbitrary scales of upsampling. And the AF block employs branch attention (BA) to integrate upsampled low-resolution images with high-level features. Besides, the loss function takes both adversarial loss and feature loss into consideration to guide the learning procedure of generator. We have compared our MA-GAN approach with several state-of-the-art methods on a number of remote sensing scenes, and experimental results consistently demonstrate the effectiveness of the proposed MA-GAN. For study replication, the source code will be released at: https://github.com/ZhihaoWang1997/MA-GAN.

Incremental Structure from Motion (ISfM) has been widely used for UAV image orientation. Its efficiency, however, decreases dramatically due to iterative BA (bundle adjustment). Although the divide-and-conquer strategy has been utilized for efficiency improvement, cluster merging becomes difficult or depends on seriously designed common image poses or 3D points. This paper proposes an algorithm to extract the global model for cluster merging and designs a parallel ISfM solution to achieve efficient and accurate image orientation. First, based on vocabulary tree retrieval, match pairs are selected to construct an undirected weighted match graph, whose edge weights are calculated by considering both the number and distribution of feature matches. Second, an algorithm termed weighted connected dominating set (WCDS), is designed to achieve the simplification of the match graph and build the global model, which incorporates the edge weight in the graph vertex selection and enables the successful reconstruction of the global model. Third, the match graph is simultaneously divided into compact and non-overlapped clusters. After the parallel reconstruction, cluster merging is conducted with the aid of the global model. Finally, by using three UAV datasets that are captured by classical oblique and recent optimized views photogrammetry, the validation of the proposed solution is verified through comprehensive analysis and comparison. The experimental results demonstrate that the proposed parallel ISfM can achieve 17.4 times efficiency improvement and comparative orientation accuracy. In absolute BA, the geo-referencing accuracy is approximately 2.0 and 3.0 times the GSD (Ground Sampling Distance) value in the horizontal and vertical directions, respectively. For parallel ISfM, the proposed solution is a more reliable alternative.

The Ocean and Land Color Instrument (OLCI) on Sentinel-3 is one of the most advanced ocean color satellite sensors for aquatic environment monitoring. However, limited studies have been focused on a comprehensive assessment of atmospheric correction (AC) methods for OLCI. In an attempt to fill the gap, this study evaluated seven different AC methods for OLCI using global automaticin situobservations from Aerosol Robotic Network-Ocean Color (AERONET-OC). Results showed that the POLYnomial-based algorithm applied to MERIS (POLYMER) had the best performance for bands with wavelength ≤ 443 nm, and the SeaDAS method based on 779 and 865 nm was the best for longer spectral bands; however, SeaDAS (SeaWiFS Data Analysis System) processing algorithm based on 779 and 1020 nm, as well as 865 and 1020 nm, obtained degraded AC performance; Case 2 Regional CoastColor (C2RCC) also produced large uncertainties; Baseline AC (BAC) method might be better than SeaDAS method; and simple subtraction method was the worst except for turbid waters. POLYMER and C2RCC underestimated high remote sensing reflectance (Rrs) at red and green bands; SeaDAS method based on 779 and 865 nm held an advantage for clear waters over the other two band combinations, while their difference turned small for turbid waters. AC uncertainties generally impacted the performance of chlorophyll retrievals. POLYMER outperformed other methods for chlorophyll retrieval. This study provides a good reference for selecting a suitable AC method for aquatic environment monitoring with Sentinel-3 OLCI.

In this paper we propose a new line segment detection framework based on a novel Pixel Orientation Estimation (POE) method, which detects line segments from binary edge maps and can be combined with any edge detectors. We show its efficiency by testing it in 1-look SAR images. The novel Pixel Orientation Estimation method estimates the orientation of each edge pixel by counting the number of edge pixels along a set of orientations. As the most edge pixels exist along the orientation of the line segment, the orientation of the edge pixel is given by the orientation that gives the maximum number of counts. Counting the number of edge pixels along different orientations is equivalent to convolving the local neighbourhood of an edge pixel with a set of carefully designed window functions with each window function corresponding to a fixed orientation. With the estimated orientations of pixels in the edge map, pixels can be grouped with a region growing step to form line support regions. Regions with their size larger than a size threshold will be accepted. Finally, rectangles are used to approximate the accepted regions and those rectangles are detected line segments. Experiments in both simulated SAR dataset and three 1-look Sentinel-1 images demonstrate the efficiency of the proposed method. In particular, we advance the state-of-the-art performances by 18 percent (F1-score) on the 1-look dataset simulated from YorkUrban-LineSegment Dataset.

As the only natural satellite of the Earth, the Moon provides vital location resources and supportive environment for Earth observations, and the Moon-based Earth observation (MEO) has unparalleled advantages in global climate change and large-scale phenomena. The ocean plays an important role in regulating climate and global water and carbon cycle. With an attempt to explore the feasibility of MEO-based marine environment monitoring, this study aimed to investigate the observing geometry and revisiting frequency of the MEO-based ocean color remote sensing and further to explore its quantitative application potentials. Results showed that MEO-based ocean color remote sensing, capturing the Earth on an hourly basis, could observe most part of the ocean for over five times per day; however, both solar zenith angle and view zenith angle were high at high-latitude regions; atmospheric reflectance accounted for most of sensor-measured signal, especially at high solar and view zenith angle, while surface-reflected glint reflectance was also notable at low solar zenith angle; and the remote sensing reflectance retrieved from MEO-based ocean color remote sensing could be used for chlorophyll retrieval. In further studies, more efforts should be paid on how to accurately retrieve remote sensing reflectance at high solar and view zenith angle, which would improve the application capability of MEO for polar regions. Overall, this study demonstrated the great potentials of MEO-based ocean color remote sensing, and MEO would be a new observing perspective and long-term consistent data source for marine environment monitoring.

Remotely sensed nighttime light (NTL) data provides an opportunity to observe human economic activities at night. However, existing NTL remote sensing data does not sufficiently reflect human activities at a fine resolution. In this study, we obtain a set of NTL data comprising volunteered passenger aircraft nighttime remote sensing (VPAN-RS) imagery by adopting a passenger plane as the remote sensing platform and using low-cost portable photography equipment as sensors. This method demonstrates the advantages of VPAN-RS data in high spatial resolution, frequent revisits, and low cost, which can supplement existing NTL data. The dataset acquired in this study covers 16 cities in China and one city in Japan, with a spatial resolution up to meter level. The data acquisition method and processing workflow are introduced in this article. Quality assessment shows that the reprojection error of the dataset falls within the range from 5–10 pixels, and the geometric error can be within 15 m. Via a comparison to reference data derived from the Visible Infrared Imaging Radiometer Suite (VIIRS) sensor of the Suomi-National Polar-Orbiting Partnership (NPP) polar-orbiting satellite, we found that the VPAN-RS data provides more details of city lighting. Additionally, the radiance of the VPAN-RS data suitably fits that of monthly composite NPP/VIIRS data. We further outline an application example of VPAN-RS nighttime imagery, examine the unique challenges associated with VPAN-RS data with a focus on the influence of the image acquisition method on the data quality, and provide an outlook for the future of VPAN-RS data. We conclude that VPAN-RS NTL images are an effective data source for nighttime earth observations and have the potential for various applications.

The proper use of distributed scatterer (DS) can improve both the density and quality of synthetic aperture radar (SAR) interferometry (InSAR) measurements. A critical step in DS interferometry (DSI) is the restoration of a consistent phase series from SAR interferogram stacks. Most state-of-the-art algorithms adopt an approximate likelihood function to calculate the likelihood by replacing the true coherence matrix with its estimation, more specifically, the sample coherence matrix (SCM). However, this approximation has a drawback in that the coherence estimates are greatly biased when the coherence is low. In this study, we derive a new likelihood function without such an approximation. Accordingly, a DSI framework using this function for phase estimation and point selection is provided. In this framework, the new likelihood function serves as a cost function for phase estimation and a quality measure for DS selection. Its performance is investigated by experiments in a simulation study and a real-world case study using Sentinel-1 data over Shenzhen airport in China. The results reveal that the proposed DSI framework outperforms the existing state-of-the-art approaches in different scenarios, in terms of providing a more accurate estimation and improving DS density and coverage.

Simultaneous localization and mapping (SLAM) is a fundamental technique block in the indoor-navigation system for most autonomous vehicles and robots. SLAM aims at building a global consistent map of the environment while simultaneously determining the position and orientation of the robot in this map. Significant advances have been made in visual SLAM techniques in the past several years. However, due to the fragile performance in tracking feature points in environments that lack texture, e.g., a warehouse with blank white walls, visual SLAM can hardly provide a reliable localization. Compared with visual SLAM, LiDAR SLAM can often provide more robust localization in indoor environments by using 3D spatial information directly captured by LiDAR point clouds. Thus, LiDAR SLAM techniques are often employed in industrial applications such as automated guided vehicles (AGVs). In the past decades, a number of LiDAR SLAM methods have been proposed. However, the strength and weakness points of various LiDAR SLAMs are not clear, which may perplex the researchers and engineers. In this article, analysis and comparisons are made on different LiDAR SLAM-based indoor navigation methods, and extensive experiments are conducted to evaluate their performances in real environments. The comparative analysis and results can help researchers in academia and industry in constructing a suitable LiDAR SLAM system for indoor navigation for their own usage scenarios.

Short-term traffic prediction is of great importance to the management of traffic congestion, a pervasive and difficult-to-solve problem in many metropolises all over the world. However, existing studies on traffic prediction contain rough traffic information at the carriageway level that ignore the distinction between different turns in one intersection. With the aim of predicting traffic at road intersections from big trace data on a finer scale, this study proposes a novel method, the fine-grained traffic prediction method (FTPG) with a graph attention network (GAT), which predicts traffic information, including traffic flow speeds, traffic states, and average queue lengths, at the turn level. In the FTPG, a method for estimation of the queue starting point is proposed to improve the accuracy of traffic information detection. Furthermore, the topology is constructed under turn-level conditions, and a GAT-based method, the spatio-temporal residual graph attention network (ST-RGAN), is proposed to improve the prediction accuracy. Experiments are performed using taxi GPS trace data collected in the city of Wuhan and show that the proposed FTPG method can make predictions with fine-grained traffic information for road intersections accurately and robustly.

Cracks in tunnel linings are the most common tunnel defects. As early indicators of structural deterioration, cracks represent critical problems for the safety of tunnels. Several mobile tunnel inspection systems (MTISs) have been developed for tunnel crack inspection. However, due to the weak signals of cracks, these MTISs require considerable exposure time to capture high-quality tunnel images, necessitating a low travel speed. Meanwhile, traditional crack detection methods encounter difficulties in processing tunnel crack images because of their low contrast and poor continuity. To overcome these challenges, this study presents a new MTIS for fast tunnel crack inspection that consists of a novel mobile imaging module and an automatic crack detection module. The imaging module is composed of an array of high-resolution charge-coupled device (CCD) cameras, a mobile laser scanner, and a lighting array. The core of the crack detection module is a novel lightweight convolutional neural network (CNN) designed for efficient tunnel crack detection, with an effective spatial constraint strategy to guarantee crack continuity. We collected a new tunnel crack dataset consisting of 1,218 images using our mobile imaging module at a driving speed of 80 km/h. Comprehensive experiments were conducted on this dataset to evaluate the performance of our proposed network. The results demonstrate that the presented CNN can effectively detect tunnel cracks with state-of-the-art performance, achieving an F1-score greater than 0.88 and an inference speed of 17 FPS with only 3.4M model parameters. The code and data are available athttps://github.com/urban-informatics/LinkCrack.

To improve the accuracy and efficiency of 3D LiDAR mapping, real-time cooperative SLAM has been considered to explore large and complex areas. To merge the individual maps from multiple robots, it is crucial to identify the common areas and obtain alternative matches between them. However, data transmission, especially in sparse networks with narrow bandwidth and limited range, is a challenging issue for the above problem. Since the distribution manner is suitable for limited communication, we proposed a common framework of 3D real-time distributed cooperative SLAM to fill the community gap. Assuming that each robot can communicate with others, the presented framework consists of four key modules: place recognition, relative pose estimation, distributed graph optimization, and communication. Meanwhile, we developed a complete real-time distributed cooperative SLAM system, called RDC-SLAM, by integrating state-of-the-art components into the framework. For computation and data transmission efficiency, descriptor-based registration is used instead of the conventional point cloud matching. An intensity-based descriptor is developed to perform the place recognition and obtain the alternative matches, while an eigenvalue-based segment descriptor is applied to further refine the relative pose estimations between these alternative matches. A distributed graph optimization method is utilized to obtain the maximum likelihood of multi-trajectory estimation. A communication protocol is also designed to associate data among robots that are easy to deploy and have low network requirements. The RDC-SLAM is validated by real-world experiments and exhibits superior performance concerning accuracy, computation efficiency, and data efficiency.

Sewer pipe is an essential infrastructure in the city as it undertakes the transportation and circulation of wastewater resources. But sewer pipe it is easy to have faults and cause serious secondary urban accidents, such as road holes and road collapse. Because of the complex underground circumstance, inspecting large-area sewer pipes using closed-circuit television or periscope television is difficult. In this study, we proposed a collaborative sewer pipe inspection approach by using novel low-cost pipe robotic capsules, which capture the images of the pipeline inner walls when floating with the water flow. A set of workers collaboratively drop and salvage capsules to cover a large-area pipe network. The routes of workers and pipe capsules are optimized by a meta-heuristic algorithm integrating local search and simulated annealing. The deep neural network is used to recognize faults from raw captured images. A field experiment in Shenzhen was conducted to evaluate the performance of the proposed approach. The results demonstrate that it outperforms the naive inspection method with a shorter travel distance and less waiting time. It is also effective for inspecting the large-area sewer pipe networks with an overall precision of 0.92. It will help us to eliminate the potential safety risk of the public and promote the level of urban governance.

Understanding population distribution has excellent applications for planning and provision of municipal services. This study aims to explore the space-time structure of population distribution with area-level mobile phone data. We discuss a kind of Bayesian hierarchical models, fitted by Markov chain Monte Carlo simulation, that combines the overall spatial pattern and temporal trends as well as the departures from these stable components. We carry out an empirical study in Shenzhen, China, using the area-level mobile phone users in 24 hours. The results indicate that the estimation of the overall spatial pattern is not deteriorated when using a sophisticated spatio-temporal model. The temporal trend exhibits a reasonable fluctuation during the study period. Then we apply two rules to detect areas showing unstable trends of population fluctuation based on the posterior probabilities of the space-time interactions. We also include the population statistics and indices for mixed-use to explore the spatial pattern of population fluctuation. Our findings confirm that the Bayesian spatio-temporal model can enhance the understanding of the space-time variability of population distribution using mobile phone data. Further research should examine the spatial nonstationary effects of explanatory factors on mobile phone-based population fluctuation.

With the promotion of low-carbon travel, pedestrian network plays an important role in many location-based applications, such as pedestrian navigation and refined traffic management. Due to the lack of systematic data acquisition mechanics, the accuracies and detail levels of pedestrian network data are hardly capable of satisfying the demands of such transportation applications. Presently, various mobile phone apps recorded and stored users' movement trajectories, which provide a valuable data source for pedestrian network construction. Hence, this article proposes a crowdsourcing-based system for generating pedestrian network that encompasses three key components of crowdsourced walking trajectory data filtering, pedestrian network construction and evaluation of pedestrian network. Self-collected data and open platform data were used to evaluate the proposed system. Experimental results demonstrate that the proposed method can accurately and completely extract pedestrian network. Moreover, the pedestrian network can be updated in a timely manner by the proposed method. The data collection application and the collected data are available to the public.

The development of information and communication technologies has produced massive human sensing data sets, such as point of interest, mobile phone data, and social media data sets. These data sets provide alternative human perceptions of urban spaces; therefore, they have become effective supplements for remote sensing tasks. This letter presents an exploratory framework to examine the scale effect of fusing remote sensing and human sensing. The physical and social semantics are extracted from raw remote sensing images and human sensing data, respectively. A dynamic weighting strategy is developed to explore the fusion of remote sensing and human sensing. Taking urban function inference as an example, the scale effect is evaluated by weighting remote sensing and human sensing. The experiment demonstrates that fusing remote sensing and human sensing enables us to recognize multiple types of urban functions. Meanwhile, the results are significantly affected by the scale.

Due to the importance in many military and civilian applications, hyperspectral anomaly detection has attracted remarkable interest. Low-rank representation (LRR)-based anomaly detectors use the low-rank property to represent background pixels, and pixels that cannot be well represented are detected as anomalies. The ability of an LRR-based detector to separate background pixels and anomalous pixels depends on the dictionary representation ability, which usually can be enhanced by designing a proper prior for dictionary representation coefficients and constructing a better dictionary. However, it is not easy to handcraft effective and meaningful regularizers for dictionary coefficients. In this article, we propose a novel anomaly detection algorithm that uses a plug-and-play prior for representation coefficients and constructs a new dictionary based on clustering. Instead of cumbersomely handcrafting a regularizer for representation coefficients, we propose solving the anomaly detection problem using the plug-and-play framework, which enables us to plug state-of-the-art priors for representation coefficients. An effective convolutional neural network (CNN) denoiser is plugged into our framework to fully exploit the spatial correlation of representation coefficients. We also propose a modified background dictionary construction method, which carefully includes background pixels and excludes anomalous pixels from clustering results. We refer to the proposed anomaly detection method as plug-and-play denoising CNN regularized anomaly detection (DeCNN-AD) method. Extensive experiments were performed on five data sets in a comparison with eight state-of-the-art anomaly detection methods. The experimental results suggest that the proposed method is effective in anomaly detection and can produce better anomaly detection results than that of the comparison methods. The codes of this work will be available athttps://github.com/FxyPdfor the sake of reproducibility.

In the hyperspectral image, each pixel corresponds to a small area on the Earth's surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial-spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial-spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples.

Hyperspectral images encompass abundant information and provide unique characteristics for material classification. However, the labeling of training samples can be challenging in hyperspectral image classification. To address this problem, this study proposes a framework named flexible Gabor-based superpixel-level unsupervised linear discriminant analysis (FG-SuULDA) to extract the most informative and discriminating features for classification. First, a number of 3-D flexible Gabor filters are rigorously designed using an asymmetric sinusoidal wave to sufficiently characterize the spatial–spectral structure in hyperspectral images. Then, an unsupervised linear discriminant analysis strategy guided by the entropy rate superpixel (ERS) segmentation algorithm, calledSuULDA, is skillfully introduced to reduce the extracted large amount of FG features. TheSuULDA method not only boosts the classification capability but also increases the peculiarity of features, with the aid of superpixel information. Finally, the achieved features are imported to the popular support vector machine classifier. The proposed FG-SuULDA framework is applied to four real hyperspectral image data sets, and the experiments constantly prove that our FG-SuULDA is superior to several state-of-the-art methods in both classification performance and computational efficiency, especially with scarce training samples. The codes of this work are available athttp://jiasen.tech/papers/for the sake of reproducibility.

Electric vehicles (EVs) currently face formidable challenges in promotion, i.e., short driving ranges, long charging times, and few charging stations, thereby limiting their acceptability to taxi drivers. Leveraging massive-scale taxi GPS trajectory data, we present a novel real-time route recommendation system for electric taxi (ET) drivers. Taxi travel knowledge, including the probability of picking up passengers and the distribution of destinations, is learned from the raw GPS trajectories. Considering the cascading effect of route decision making, consecutive ET actions are modeled with an action tree. The corresponding expected net revenue is estimated based on the learned knowledge. A prototype online system is developed for providing route recommendations, e.g., when to go to a charging station or cruise on certain roads. An experiment in Shenzhen demonstrates that the average daily net revenue of ET drivers is better than those of 76.2% of gasoline taxi drivers. The presented approach not only increases the revenue of ET drivers in the short term but also improves the viability of EVs in the long run.

Global positioning system (GPS) data generated from taxi trips is a valuable source of information that offers an insight into travel behaviours of urban populations with high spatio-temporal resolution. However, in its raw form, GPS taxi data does not offer information on the purpose (or intended activity) of travel. In this context, to enhance the utility of taxi GPS data sets, we propose a two-layer framework to identify the related activities of each taxi trip automatically and estimate the return trips and successive activities after the trip, by using geographic point-of-interest (POI) data and a combination of spatio-temporal clustering, Bayesian inference and Monte Carlo simulation. Two million taxi trips in New York, the United States of America, and ten million taxi trips in Shenzhen, China, are used as inputs for the two-layer framework. To validate each layer of the framework, we collect 6,003 trip diaries in New York and 712 questionnaire surveys in Shenzhen. The results show that the first layer of the framework performs better than comparable methods published in the literature, while the second layer has high accuracy when inferring return trips.

Pedestrian networks play an important role in various applications, such as pedestrian navigation services and mobility modeling. This paper presents a novel method to extract pedestrian networks from crowdsourced tracking data based on a two-layer framework. This framework includes a walking pattern classification layer and a pedestrian network generation layer. In the first layer, we propose a multi-scale fractal dimension (MFD) algorithm in order to recognize the two different types of walking patterns: walking with a clear destination (WCD) or walking without a clear destination (WOCD). In the second layer, we generate the pedestrian network by combining the pedestrian regions and pedestrian paths. The pedestrian regions are extracted based on a modified connected component analysis (CCA) algorithm from the WOCD traces. We generate the pedestrian paths using a kernel density estimation (KDE)-based point clustering algorithm from the WCD traces. The pedestrian network generation results using two actual crowdsourced datasets show that the proposed method has good performance in both geometrical correctness and topological correctness.

3-D visualization of location-based services (LBSs) in a geographic information system (GIS) and the Internet-of-Things (IoT) applications could enhance the user experience. Texture is generally the most noticeable visual element in a 3-D indoor model. However, it usually tends to be outdated and becomes inaccurate in complex and dynamic environments. The applications of such 3-D visualizations, especially in indoor environments, are further impeded by a limited capacity to represent frequently changing scenes due to object displacements. To overcome this problem, a novel location and 3-D visual awareness-based method is proposed to automatically register and update a live image onto a 3-D indoor model. In this method, a multisensor fusion-based indoor localization algorithm employs an indoor map to integrate multisource data, including activity detection information, pedestrian dead reckoning (PDR), and 3-D vision-based localization data. Based on the positioning results, we locate the region of the photograph capture within a scene and identify the triangular meshes of the facing wall. The texture is extracted from real-time images through a 3-D indoor understanding technique and automatically fused with the indoor 3-D model. The geometric relationship between texture and the 3-D indoor model is established by calculating the UV coordinates. The experimental results demonstrate that our approach can accurately update the dynamic indoor model textures regardless of the scene. Based on this texture updating technique, users experience an enhanced presentation of LBS information on our 3-D visualization platform.

Online-to-offline (O2O) commerce connecting service providers and individuals to address daily human needs is quickly expanding. In particular, on-demand food, whereby food orders are placed online by customers and delivered by couriers, is becoming popular. This novel urban food application requires highly efficient and scalable real-time delivery services. However, it is difficult to recruit enough couriers and route them to facilitate such food ordering systems. This paper presents an online crowdsourced delivery (OCD) approach for on-demand food. Facilitated by Internet-of-Things and 3G/4G/5G technologies, public riders can be attracted to act as crowdsourced workers delivering food by means of shared bicycles or electric motorbikes. An online dynamic optimization framework comprising order collection, solution generation, and sequential delivery processes is presented. A hybrid metaheuristic solution process integrating the adaptive large neighborhood search and tabu search approaches is developed to assign food delivery tasks and generate high-quality delivery routes in a real-time manner. The crowdsourced riders are dynamically shared among different food providers. Simulated small-scale and real-world large-scale on-demand food delivery instances are used to evaluate the performance of the proposed approach. The results indicate that the presented crowdsourced food delivery approach outperforms traditional urban logistics. The developed hybrid optimization mechanism is able to produce high-quality crowdsourced delivery routes in less than 120 s. The results demonstrate that the presented OCD approach can facilitate city-scale on-demand food delivery.

Compared to other biometrics, gait is difficult to conceal and has the advantage of being unobtrusive. Inertial sensors, such as accelerometers and gyroscopes, are often used to capture gait dynamics. These inertial sensors are commonly integrated into smartphones and are widely used by the average person, which makes gait data convenient and inexpensive to collect. In this paper, we study gait recognition using smartphones in the wild. In contrast to traditional methods, which often require a person to walk along a specified road and/or at a normal walking speed, the proposed method collects inertial gait data under unconstrained conditions without knowing when, where, and how the user walks. To obtain good person identification and authentication performance, deep-learning techniques are presented to learn and model the gait biometrics based on walking data. Specifically, a hybrid deep neural network is proposed for robust gait feature representation, where features in the space and time domains are successively abstracted by a convolutional neural network and a recurrent neural network. In the experiments, two datasets collected by smartphones for a total of 118 subjects are used for evaluations. The experiments show that the proposed method achieves higher than 93.5% and 93.7% accuracies in person identification and authentication, respectively.

A 3-D Gabor wavelet provides an effective way to obtain the spectral-spatial-fused features for hyperspectral image, which has shown advantageous performance for material classification and recognition. In this paper, instead of separately employing the Gabor magnitude and phase features, which, respectively, reflect the intensity and variation of surface materials in local area, a cascade superpixel regularized Gabor feature fusion (CSRGFF) approach has been proposed. First, the Gabor filters with particular orientation are utilized to obtain Gabor features (including magnitude and phase) from the original hyperspectral image. Second, a support vector machine (SVM)-based probability representation strategy is developed to fully exploit the decision information in SVM output, and the achieved confidence score can make the following fusion with Gabor phase more effective. Meanwhile, the quadrant bit coding and Hamming distance metric are applied to encode the Gabor phase features and measure sample similarity in sequence. Third, the carefully defined characteristics of two kinds of features are directly combined together without any weighting operation to describe the weight of samples belonging to each class. Finally, a series of superpixel graphs extracted from the raw hyperspectral image with different numbers of superpixels are employed to successively regularize the weighting cube from over-segmentation to under-segmentation, and the classification performance gradually improves with the decrease in the number of superpixels in the regularization procedure. Four widely used real hyperspectral images have been conducted, and the experimental results constantly demonstrate the superiority of our CSRGFF approach over several state-of-the-art methods.

Hydrological drought is a global issue that many countries face. In this paper, we present a new hydrological index (SGI) based on the Gravity Recovery and Climate Experiment (GRACE) product. For comparison, we compare it with the documented drought in July 2010 and calculate the correlation coefficient between it and the standardized precipitation index (SPI). The result shows the new index can reflect spatiotemporal distribution of dryness and wetness, and it has good agreement with SPI in different time scales, and the new index can be used in the hydrological drought measurement in the global scale.

The rapid development of information and communication technology and the popularization of mobile devices have generated a large number of spatiotemporal trajectory data. Trajectory data can be applied to location prediction, which is significant for urban traffic planning and location-based service. Although various methods for personal location prediction have been proposed, the historical trajectory data of some users is always sparse in practical applications, resulting in poor prediction precision of prediction models based on personal historical data for those sparse users. Targeting on this challenge, we propose an "Individual trajectory-Group trajectory assist Individual trajectory" location prediction model (ITGTAIT) by utilizing the group travel patterns to assist in predicting personal locations. First, the model conducts a spatial clustering algorithm on trajectory points to construct the clustering link. Second, the clustering link and Fano's inequality are used to estimate the predictability of the next location. Third, a Variable Order Markov Model that named Prediction by Partial Match (PPM) was adopted to predict the clustering link based on the individual trajectory for users with sufficient data. For users with sparse samples, the PPM utilizes the pattern of group travels, which using the group trajectory to assist individual trajectory. Finally, our method was evaluated by using 608,712 trajectory points from 5000 volunteers at Shenzhen city, China. The result shows that a) with the increase of training data, the precision of the ITGTAIT is gradually stable, b) for users with over four days of data, the highest precision is 87.11%, stable at about 82%, c) for users with only 1-3 days of data, the prediction precision is 55.15%, 67.04%, and 76.86% respectively, which introduces approximately 10.76%, 10.98% and 6.83% performance gains on location predictions respectively by utilizing the group characters.

Sampling fraction is crucial to sampling-related studies and applications, especially in the big data era when most data are neither originally designed nor controllable in the data collection process. A common concern among researchers is ‘what’s the modelling accuracy when using a sample?’. Taking intra-city human mobility as the study objective, this study utilizes a simple and direct method to analyse the influences of various sampling fractions on modelling accuracy. Five common intra-city human mobility indicators (travel distance, travel time, travel frequency, radius of gyration and movement entropy) are evaluated considering mean value, median and probability distribution. Experimental results demonstrate that the representativeness of each considered indicator converges to 1 in its own unique rate and variances. The minimum required sampling fractions to satisfy specific accuracies differ for various indicators and evaluation measures. To further investigate how related factors influence the modelling accuracy of sampling fractions, additional experiments are conducted considering multiple sampling methods, study scopes, and data sources. Several interesting general findings are observed. This study provides a reference for other sampling-based applications.

Deriving shallow-water depth from multispectral satellite imageries appears very promising in near-shore bathymetric mapping for its high efficiency and low costs. Lyzenga's linear band method, which currently is the most widely used algorithm in the field, performs a linear regression between some known water depths and image-derived input variables. In real-world situations, the linear assumption in Lyzenga's method cannot be exactly satisfied because of complex environmental factors, and its performance is, therefore, often limited. In this letter, we improve Lyzenga's method by introducing generalized additive models to describe the nonlinear relationship between water depths and image-derived regressors. The performance of the new method was evaluated by using Landsat-8 multispectral data over the coastal areas of Oahu Island, Hawaii. The experimental results of two independent validation regions show that the depth-retrieval accuracy of the proposed algorithm was about 20% better than that of Lyzenga's method, and the performance improvement was most pronounced in the depth interval of 5-10 m.

Cracks are typical line structures that are of interest in many computer-vision applications. In practice, many cracks, e.g., pavement cracks, show poor continuity and low contrast, which brings great challenges to image-based crack detection by using low-level features. In this paper, we propose DeepCrack - an end-to-end trainable deep convolutional neural network for automatic crack detection by learning high-level features for crack representation. In this method, multi-scale deep convolutional features learned at hierarchical convolutional stages are fused together to capture the line structures. More detailed representations are made in larger-scale feature maps and more holistic representations are made in smaller-scale feature maps. We build DeepCrack net on the encoder-decoder architecture of SegNet, and pairwisely fuse the convolutional features generated in the encoder network and in the decoder network at the same scale. We train DeepCrack net on one crack dataset and evaluate it on three others. The experimental results demonstrate that DeepCrack achieves F-Measure over 0.87 on the three challenging datasets in average and outperforms the current state-of-the-art methods.

Accurate removal of atmospheric interference and precise retrieval of water-leaving reflectance is decisive for subsequent water color applications. As follow-up satellite of Envisat, Sentinel-3 will provide valuable observations of the earth. This study aims to adapt the shortwave infrared extrapolation (SWIRE) atmospheric correction algorithm for Sentinel-3 to derive remote sensing reflectance of turbid waters, and validation it using our in -situ data in Pearl River Estuary. Results showed that SWIRE algorithm could effectively remove atmospheric perturbations, and produced more accurate remote sensing reflectance over complex waters of PRE than NIR and SWIR algorithms.

Lane-level road network updating is crucial for urban traffic applications that use geographic information systems contributing to, for example, intelligent driving, route planning and traffic control. Researchers have developed various algorithms to update road networks using sensor data, such as high-definition images or GPS data; however, approaches that involve change detection for road networks at lane level using GPS data are less common. This paper presents a novel method for automatic change detection of lane-level road networks based on GPS trajectories of vehicles. The proposed method includes two steps: map matching at lane level and lane-level change recognition. To integrate the most up-to-date GPS data with a lane-level road network, this research uses a fuzzy logic road network matching method. The proposed map-matching method starts with a confirmation of candidate lane-level road segments that use error ellipses derived from the GPS data, and then computes the membership degree between GPS data and candidate lane-level segments. The GPS trajectory data is classified into successful or unsuccessful matches using a set of defuzzification rules. Any topological and geometrical changes to road networks are detected by analysing the two kinds of matching results and comparing their relationships with the original road network. Change detection results for road networks in Wuhan, China using collected GPS trajectories show that these methods can be successfully applied to detect lane-level road changes including added lanes, closed lanes and lane-changing and turning rules, while achieving a robust detection precision of above 80%.

Spatiotemporal proximity analysis to determine spatiotemporal proximal paths is a critical step for many movement analysis methods. However, few effective methods have been developed in the literature for spatiotemporal proximity analysis of movement data. Therefore, this study proposes a space-time-integrated approach for spatiotemporal proximal analysis considering space and time dimensions simultaneously. The proposed approach is based on space-time buffering, which is a natural extension of conventional spatial buffering operation to space and time dimensions. Given a space-time path and spatial tolerance, space-time buffering constructs a space-time region by continuously generating spatial buffers for any location along the space-time path. The constructed space-time region can delimit all space-time locations whose spatial distances to the target trajectory are less than a given tolerance. Five space-time overlapping operations based on this space-time buffering are proposed to retrieve all spatiotemporal proximal trajectories to the target space-time path, in terms of different spatiotemporal proximity metrics of space-time paths, such as Fréchet distance and longest common subsequence. The proposed approach is extended to analyze space-time paths constrained in road networks. The compressed linear reference technique is adopted to implement the proposed approach for spatiotemporal proximity analysis in large movement datasets. A case study using real-world movement data verifies that the proposed approach can efficiently retrieve spatiotemporal proximal paths constrained in road networks from a large movement database, and has significant computational advantage over conventional space-time separated approaches.

This paper first proposes a new clustering algorithm for selection of reference points (RPs) based on virtual positions of access points for indoor localization in area without linear constraints, which can not only cluster automatically but also guarantee the consistency of methods between the offline phase clustering and the online phase positioning. A new weighted algorithm based on physical distance is then presented for position determination. With angle velocity measurement provided such as by gyroscope, the weighted algorithm is particularly suited for scenarios where the mobile moves along a trajectory. The number of clusters in traditional RP clustering algorithms needs to be predefined, which means an unsuitable number of clusters would lead to poor estimation accuracy. Traditional weighted K-nearest neighbor (WKNN) algorithm weights the RPs' coordinates by the inverse of the received signal strength indication (RSSI) difference, which is not accurate enough because of the exponential relationship between RSSI and physical distance. Furthermore, methods based on probabilistic model or data fusion do not consider the uneven spatial resolution of Wi-Fi RSSI. Experimental results show that the proposed weighted algorithm considerably outperforms the K-nearest neighbor (KNN), Euclidean-WKNN, ManhattanWKNN, EWKNN, LiFS, and GPR in terms of positioning accuracy which is defined as the cumulative distribution function of position error. The results also demonstrate that RPs in indoor area without linear constraints can be clustered automatically by the proposed clustering algorithm, and cumulative distribution function of the proposed clustering algorithm outperforms KNN, WKNN, RP location clustered, and signal distance clustered.

As manual labeling is very difficult and time-consuming, the labeled samples used to train a supervised classifier are generally limited, which become one of the biggest challenge for hyperspectral imagery classification. In order to tackle this issue, a recent trend is to exploit the structure information of materials, as which reflects the region homogeneity in the spatial domain and offers an invaluable complement to the spectral information. In this respect, 3-D Gabor wavelets have been introduced to extract joint spectral-spatial features for hyperspectral images. One the one hand, the features extracted by 3-D Gabor wavelets lead to very good performance for classification. On the other hand, its drawbacks, i.e., big number of features and high computational cost, limit its applicability. In this paper, a 3-D Gabor-wavelet-based phase coding and Hamming distance-based matching (3DGPC-HDM) framework is developed for hyperspectral imagery classification. The proposed method, instead of taking into account the large volume of Gabor magnitude features, exploits the Gabor phase features with certain orientations (i.e., the direction parallel to the spectral axis), which are then encoded by a simple quadrant bit coding scheme. After that, a normalized Hamming distance matching (HDM) method is adopted to determine the similarity of two samples, and the nearest neighbor classifier is routinely utilized for pixelwise recognition. Finally, experiments on three real hyperspectral data sets show that the proposed 3DGPC-HDM leads to very good performance. Comparisons with the state-of-the-art methods in the literature, in terms of both classifier complexity and generalization ability from very small training sets, are also included.

Gait has been considered as a promising and unique biometric for person identification. Traditionally, gait data are collected using either color sensors, such as a CCD camera, depth sensors, such as a Microsoft Kinect, or inertial sensors, such as an accelerometer. However, a single type of sensors may only capture part of the dynamic gait features and make the gait recognition sensitive to complex covariate conditions, leading to fragile gait-based person identification systems. In this paper, we propose to combine all three types of sensors for gait data collection and gait recognition, which can be used for important identification applications, such as identity recognition to access a restricted building or area. We propose two new algorithms, namely EigenGait and TrajGait, to extract gait features from the inertial data and the RGBD (color and depth) data, respectively. Specifically, EigenGait extracts general gait dynamics from the accelerometer readings in the eigenspace and TrajGait extracts more detailed subdynamics by analyzing 3-D dense trajectories. Finally, both extracted features are fed into a supervised classifier for gait recognition and person identification. Experiments on 50 subjects, with comparisons to several other state-of-the-art gait-recognition approaches, show that the proposed approach can achieve higher recognition accuracy and robustness.

Since it is usually difficult and time-consuming to obtain sufficient training samples by manually labeling, feature extraction, which investigates the characteristics of hyperspectral images (HSIs), such as spectral continuity and spatial locality of surface objects, to achieve the most discriminative feature representation, is very important for HSI classification. Meanwhile, due to the spatial regularity of surface materials, it is desirable to improve the classification performance of HSIs from the superpixel viewpoint. In this paper, we propose a novel local binary pattern (LBP)-based superpixel-level decision fusion method for HSI classification. The proposed framework employs uniform LBP (ULBP) to extract local image features, and then, a support vector machine is utilized to formulate the probability description of each pixel belonging to every class. The composite image of the first three components extracted by a principal component analysis from the HSI data is oversegmented into many homogeneous regions by using the entropy rate segmentation method. Then, a region merging process is applied to make the superpixels obtained more homogeneous and agree with the spatial structure of materials more precisely. Finally, a probability-oriented classification strategy is applied to classify each pixel based on superpixel-level guidance. The proposed framework “ULBP-based superpixel-level decision fusion framework” is named ULBP-SPG. Experimental results on two real HSI data sets have demonstrated that the proposed ULBP-SPG framework is more effective and powerful than several state-of-the-art methods.

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial-spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial-spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial-spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training.

Travel time uncertainty has significant impacts on individual activity-travel scheduling, but at present these impacts have not been considered in most accessibility studies. In this paper, an accessibility evaluation framework is proposed for urban areas with uncertain travel times. A reliable space-time service region (RSTR) model is introduced to represent the space-time service region of a facility under travel time uncertainty. Based on the RSTR model, four reliable place-based accessibility measures are proposed to evaluate accessibility to urban services by incorporating the effects of travel time reliability. To demonstrate the applicability of the proposed framework, a case study using large-scale taxi tracking data is carried out. The results of the case study indicate that the proposed accessibility measures can evaluate large-scale place-based accessibility well in urban areas with uncertain travel times. Conventional place-based accessibility indicators ignoring travel time reliability can significantly overestimate the accessibility to urban services.

Pedestrian navigation at night should differ from daytime navigation due to the psychological safety needs of pedestrians. For example, pedestrians may prefer better-illuminated walking environments, shorter travel distances, and greater numbers of pedestrian companions. Route selection at night is therefore a multi-objective optimization problem. However, multi-objective optimization problems are commonly solved by combining multiple objectives into a single weighted-sum objective function. This study extends the artificial bee colony (ABC) algorithm by modifying several strategies, including the representation of the solutions, the limited neighborhood search, and the Pareto front approximation method. The extended algorithm can be used to generate an optimal route set for pedestrians at night that considers travel distance, the illumination of the walking environment, and the number of pedestrian companions. We compare the proposed algorithm with the well-known Dijkstra shortest-path algorithm and discuss the stability, diversity, and dynamics of the generated solutions. Experiments within a study area confirm the effectiveness of the improved algorithm. This algorithm can also be applied to solving other multi-objective optimization problems.

Understanding urban functions and their relationships with human activities has great implications for smart and sustainable urban development. In this study, we present a novel approach to uncovering urban functions by aggregating human activities inferred from mobile phone positioning and social media data. First, the homes and workplaces (of travelers) are estimated from mobile phone positioning data to annotate the activities conducted at these locations. The remaining activities (such as shopping, schooling, transportation, recreation and entertainment) are labeled using a hidden Markov model with social knowledge learned from social media check-in data over a lengthy period. By aggregating identified human activities, hourly urban functions are inferred, and the diurnal dynamics of those functions are revealed. An empirical analysis was conducted for the case of Shenzhen, China. The results indicate that the proposed approach can capture citywide dynamics of both human activities and urban functions. It also suggests that although many urban areas have been officially labeled with a single land-use type, they may provide different functions over time depending on the types and range of human activities. The study demonstrates that combining different data on human activities could yield an improved understanding of urban functions, which would benefit short-term urban decision-making and long-term urban policy making.

Although mixed use is an emerging strategy that has been widely accepted in urban planning for promoting neighbourhood vibrancy, there is no consensus on how to quantitatively measure the mix and the effects of mixed use on neighbourhood vibrancy. Shannon entropy, the most commonly used diversity measurement in assessing mixed use, has been found to be inadequate in measuring the multifaceted, multidimensional characteristics of mixed use. And lack of data also makes it difficult to find the relationship between mixed use and neighbourhood vibrancy. However, the recent availability of new sources including mobile phone data and Point of Interest (POI) data have made it possible to develop new indices of mixed use and neighbourhood vibrancy to analyse their relationships. Taking advantage of these emerging new data sources, this study used the numbers of mobile phone users in a 24-hour period as a proxy of neighbourhood vibrancy and used POIs from a navigation database to develop a series of mixed-use indicators that can better reflect the multifaceted, multidimensional characteristics of mixed-use neighbourhoods. The Hill numbers, a unified form of diversity measurement used in ecological literature that includes richness, entropy, and the Simpson index, are used to measure the degrees of mixed use. Using such fine-grained data sets and the Hill numbers allowed us to obtain better insights into the relationship between mixed use and neighbourhood vibrancy. Four models varying in POI measurements that reflect different dimensions of mixed use were presented. The results showed that either POI density or entropy can explain approximately 1% of neighbourhood vibrancy, while POI richness contributes significantly in improving neighbourhood vibrancy. The results also revealed that the entropy has limitations as a measure for representing mixed use and demonstrated the necessity of adopting a set of more appropriate measurements for mixed use. Increasing the number of POIs has limited power to improve neighbourhood vibrancy compared with encouraging the mixing of complementary POIs. These exploratory findings may be useful for adjusting mixed-use assessments and to help guide urban planning and neighbourhood design.

Due to the high spectral dimensionality of hyperspectral images as well as the difficult and time-consuming process of collecting sufficient labeled samples in practice, the small sample size scenario is one crucial problem and a challenging issue for hyperspectral image classification. Fortunately, the structure information of materials, reflecting region of homogeneity in the spatial domain, offers an invaluable complement to the spectral information. Assuming some spatial regularity and locality of surface materials, it is reasonable to segment the image into different homogeneous parts in advance, called superpixel, which can be used to improve the classification performance. In this paper, a superpixel-based multitask learning framework has been proposed for hyperspectral image classification. Specifically, a set of 2-D Gabor filters are first applied to hyperspectral images to extract discriminative features. Meanwhile, a superpixel map is generated from the hyperspectral images. Second, a superpixel-based spatial-spectral Schroedinger eigenmaps (S4E) method is adopted to effectively reduce the dimensions of each extracted Gabor cube. Finally, the classification is carried out by a support vector machine (SVM)-based multitask learning framework. The proposed approach is thus termed Gabor S4E and SVM-based multitask learning (GS4E-MTLSVM). A series of experiments is conducted on three real hyperspectral image data sets to demonstrate the effectiveness of the proposed GS4E-MTLSVM approach. The experimental results show that the performance of the proposed GS4E-MTLSVM is better than those of several state-of-the-art methods, while the computational complexity has been greatly reduced, compared with the pixel-based spatial-spectral Schroedinger eigenmaps method.

The local binary pattern (LBP) is a simple and efficient texture descriptor for image processing. Recently, LBP has been introduced for feature extraction of hyperspectral imagery. Specifically, the LBP codes are extracted from the 2-D band images to capture the spatial correlation among neighboring pixels, and then the statistical histogram features from all bands, which could estimate the underlying distribution in local area, are concatenated together for pixel-wise classification. However, since hyperspectral imagery contains rich spectral and spatial information, which is actually a 3-D data cube, the 2-D LBP (2-DLBP) model cannot fully exploit the joint spectral-spatial structure. In this paper, the 2-DLBP has been extended into 3-D LBP (3-DLBP) model through forming a 3-D regular octahedral frame to characterize the spectral-spatial relationship. In order to reflect the local continuous property of hyperspectral data in both the spectral and spatial domains, while ensuring the rotational invariance of the 3-DLBP model, the code patterns of 3-DLBP model have been divided into eight groups (including seven groups of “dense” patterns and one group of “nondense” patterns) based on the consistency of spectral-spatial topology structure. Specifically, the patterns in seven “dense” groups correspond to the microstructures in the 3-D domains (such as spots, edges, and flat areas), which has a high percentage in all the 3-DLBP patterns, while the rest patterns are aggregated and treated as the “nondense” patterns. The proposed method is thus called 3-D dense LBP (3-D2LBP) model. Moreover, instead of taking zero as the hard threshold, a slack variable has been introduced to enable the difference between the central pixel and the neighboring ones varying in a small interval, which could greatly decrease the impact of spectral variability and noise, and the discriminative power of the features has been further boosted. The slack threshold-based 3-D2LBP model is named ST-3-D2LBP. A series of experiments is conducted on three real hyperspectral imageries to demonstrate the effectiveness of the proposed two 3-D2LBP-based methods. The experimental results show that the performance of the proposed ST-3-D2LBP is significantly superior to that of 2-DLBP, which is also better than the 3-D2LBP model and several state-of-the-art hyperspectral classification methods.

Fitting polarimetric synthetic aperture radar (PolSAR) data with adaptive scattering models is a promising way to mitigate the deficiencies of the model-based decomposition. Recently, Lee et al. have proposed a generalized decomposition model with several adaptive parameters, whereas the generalized model introduces too much freedom to be solved. In this paper, based on the Lee generalized decomposition model, an adaptive two-component decomposition model is proposed. The PolSAR coherency matrix is represented as the sum of two scattering mechanisms: coherent ground scattering and incoherent volume scattering. The proposed model is under three assumptions: 1) Surface and double scattering are coherent; 2) surface and double scattering are integrated as the ground scattering; and 3) the average polarimetric orientation angle (POA) of the volume (or Bragg) scattering is zero. As the proposed model is very difficult to solve directly, we adopted the exhaustion technique to find the best fit parameter set. The proposed model has three advantages: 1) It can successfully avoid the negative power problem; 2) it is considered without the assumption of reflection symmetry; and 3) the dominant scattering mechanism criterion is not needed in the process of model inversion. However, the proposed model has two disadvantages: 1) the attribution of the volume model becomes ambiguous; and 2) the assumption that sets the POA of the Bragg scattering component to zero is inconsistent with the actual scattering mechanism when there is a slope in the rough surface. The polarimetric AIRSAR L-band data of San Francisco and ESAR L-band data of Oberpfaffenhofen were used to show the efficiency of the proposed decomposition model. Statistical properties of typical areas showed that, except the sea surface and the urban area with building orientation angle about 45°, the proposed model fits the PolSAR data very well.

Region-based hierarchical image representation is crucial in many computer vision applications. However, in practice, an image hierarchy is usually dense, and contains many less informative branches. It is expected that a hierarchy should be accurate and simplified, which is not only desirable for different applications, but also saves considerable computational load for the further analysis. To achieve this target, this paper proposes a novel approach for unsupervised simplification of region-based image hierarchies, which employs the global and local evolution analyses of a hierarchy. First, we introduce a global evolution analysis in the scale-sets framework, which provides clues for eliminating less informative branches. Moreover, a hybrid unsupervised simplification method is designed, utilizing the information from global and local evolution functions. A number of experiments on various images have shown that the proposed approach is effective and efficient in removing less informative nodes (averagely about 90% of the whole nodes), while preserving salient image details and retaining the accuracy.

The 3-D LiDAR scanner and the 2-D charge-coupled device (CCD) camera are two typical types of sensors for surrounding-environment perceiving in robotics or autonomous driving. Commonly, they are jointly used to improve perception accuracy by simultaneously recording the distances of surrounding objects, as well as the color and shape information. In this paper, we use the correspondence between a 3-D LiDAR scanner and a CCD camera to rearrange the captured LiDAR point cloud into a dense depth map, in which each 3-D point corresponds to a pixel at the same location in the RGB image. In this paper, we assume that the LiDAR scanner and the CCD camera are accurately calibrated and synchronized beforehand so that each 3-D LiDAR point cloud is aligned with its corresponding RGB image. Each frame of the LiDAR point cloud is then projected onto the RGB image plane to form a sparse depth map. Then, a self-adaptive method is proposed to upsample the sparse depth map into a dense depth map, in which the RGB image and the anisotropic diffusion tensor are exploited to guide upsampling by reinforcing the RGB-depth compactness. Finally, convex optimization is applied on the dense depth map for global enhancement. Experiments on the KITTI and Middlebury data sets demonstrate that the proposed method outperforms several other relevant state-of-the-art methods in terms of visual comparison and root-mean-square error measurement.

Detecting vehicle turn signals at night is critical for both assistant driving systems and autonomous driving systems. In this paper, we propose a novel method that consists of detection and tracking modules to achieve a high level of robustness. For nighttime vehicle detection, a Nakagami-image-based method is used to locate the regions containing vehicle lights. At the same time, a set of vehicle object proposals is generated using a region proposal network based on convolutional neural network (CNN) feature maps. Then, the light regions and proposals are combined to generate the regions of interest (ROIs) for the further detection. Vehicle candidates are extracted from the ROIs using a softmax classifier with CNN-based features. For the tracking module, we propose a perceptional hashing algorithm to track these vehicle candidates. During the tracking, turn signals are detected by analyzing the continuous intensity variation of the vehicle box sequences. Experimental results for typical sequences show that the proposed method can robustly detect and track a vehicle in front with over 95% accuracy and recognize the turning signals in night scenes with a detection rate of over 90%. The vehicle detection method improves the miss rate of state-of-the-art systems by more than 20%. In addition, the proposed vehicle tracking method outperforms other state-of-the-art systems.

Human vision benefits a lot from pattern collocations in visual activities such as object detection and recognition. Usually, pattern collocations display as the co-occurrences of visual primitives, e.g., colors, gradients, or textons, in neighboring regions. In the past two decades, many sophisticated local feature descriptors have been developed to describe visual primitives, and some of them even take into account the co-occurrence information for improving their discriminative power. However, most of these descriptors only consider feature co-occurrence within a very small neighborhood, e.g., 8-connected or 16-connected area, which would fall short in describing pattern collocations built up by feature co-occurrences in a wider neighborhood. In this paper, we propose to describe local pattern collocations by using a new and general regional co-occurrence approach. In this approach, an input image is first partitioned into a set of homogeneous superpixels. Then, features in each superpixel are extracted by a variety of local feature descriptors, based on which a number of patterns are computed. Finally, pattern co-occurrences within the superpixel and between the neighboring superpixels are calculated and factorized into a final descriptor for local pattern collocation. The proposed regional co-occurrence framework is extensively tested on a wide range of popular shape, color, and texture descriptors in terms of image and object categorizations. The experimental results have shown significant performance improvements by using the proposed framework over the existing popular descriptors.

Shadows often incur uneven illumination to pavement images, which brings great challenges to image-based pavement crack detection. Thus, it is desired to remove pavement shadows before detecting pavement cracks. However, due to the large penumbras cast by trees, light poles, etc., it is difficult to locate shadows in a pavement image. In this paper, an automatic pavement shadow removal method is proposed based on geodesic analysis. First, a geodesic shadow model is used to partition a pavement shadow into a number of geodesic regions. Then, an optimal background region is selected for reference by statistic analysis. Finally, a texture-balanced illuminance compensation is applied on all geodesic regions over the image. Experiments demonstrate the effectiveness of the proposed method.

The assumption of reflection symmetry is one of the major deficiencies for the model based decompositions. The introduction of helix scattering component can mitigate the impact of this assumption limitedly, while it generates some new negative power problems simultaneously. In this paper, we expand the techniques of symmetric scatterer transformation in the Cameron and Huynen decompositions to the multi-look coherency matrix. Two new models of nonlinear programming problem are proposed to mitigate the reflection symmetry assumption and negative power problems for the model based decomposition. Experimental results in the L-band AIRSAR San Francisco data show that the two new models can reduce the correlations between co-polarized and cross-polarized channels to an insignificant level.

Building orientations with respect to the radar look direction is a critical influence on the interpretation of PolSAR data in urban areas. In this paper, we try to analyze thoroughly the problems of polarimetric scattering mechanism in urban areas induced by building orientation. For multi-look PolSAR data, the polarimetric scattering mechanism in urban areas is modeled by two double-bounce scatterings from two orthogonal dihedral structures. From the model, it can be inferred that with the increase of the building orientation, the POA estimation circular-polarization method and the dominant scattering mechanism labeling technique based on the model-based decompositions will gradually become invalid. Moreover, the POA compensation processing is helpful to reduce the impacts of the building orientation, but when the building orientation is increased to some degree, it also become invalid. Three L-band datasets of San Francisco acquired by AIRSAR are used to verify the inferences.

There has been a resurgence of interest in time geography studies due to emerging spatiotemporal big data in urban environments. However, the rapid increase in the volume, diversity, and intensity of spatiotemporal data poses a significant challenge with respect to the representation and computation of time geographic entities and relations in road networks. To address this challenge, a spatiotemporal data model is proposed in this article. The proposed spatiotemporal data model is based on a compressed linear reference (CLR) technique to transform network time geographic entities in three-dimensional (3D) (x, y, t) space to two-dimensional (2D) CLR space. Using the proposed spatiotemporal data model, network time geographic entities can be stored and managed in classical spatial databases. Efficient spatial operations and index structures can be directly utilized to implement spatiotemporal operations and queries for network time geographic entities in CLR space. To validate the proposed spatiotemporal data model, a prototype system is developed using existing 2D GIS techniques. A case study is performed using large-scale datasets of space-time paths and prisms. The case study indicates that the proposed spatiotemporal data model is effective and efficient for storing, managing, and querying large-scale datasets of network time geographic entities.

Kernel Density Estimation (KDE) is an important approach to analyse spatial distribution of point features and linear features over 2-D planar space. Some network-based KDE methods have been developed in recent years, which focus on estimating density distribution of point events over 1-D network space. However, the existing KDE methods are not appropriate for analysing the distribution characteristics of certain kind of features or events, such as traffic jams, queue at intersections and taxi carrying passenger events. These events occur and distribute in 1-D road network space, and present a continuous linear distribution along network. This paper presents a novel Network Kernel Density Estimation method for Linear features (NKDE-L) to analyse the space–time distribution characteristics of linear features over 1-D network space. We first analyse the density distribution of each linear feature along networks, then estimate the density distribution for the whole network space in terms of the network distance and network topology. In the case study, we apply the NKDE-L to analyse the space–time dynamics of taxis’ pick-up events, with real road network and taxi trace data in Wuhan. Taxis’ pick-up events are defined and extracted as linear events (LE) in this paper. We first conduct a space–time statistics of pick-up LE in different temporal granularities. Then we analyse the space–time density distribution of the pick-up events in the road network using the NKDE-L, and uncover some dynamic patterns of people’s activities and traffic condition. In addition, we compare the NKDE-L with quadrat method and planar KDE. The comparison results prove the advantages of the NKDE-L in analysing spatial distribution patterns of linear features in network space.

Map matching method is a fundamental preprocessing technique for massive probe vehicle data. Various transportation applications need map matching methods to provide highly accurate and stable results. However, most current map matching approaches employ elementary geometric or topological measures, which may not be sufficient to encode the characteristic of realistic driving paths, leading to inefficiency and inaccuracy, especially in complex road networks. To address these issues, this article presents a novel map matching method, based on the measure of curvedness of Global Positioning System (GPS) trajectories. The curvature integral, which measures the curvedness feature of GPS trajectories, is considered to be one of the major matching characteristics that constrain pairwise matching between the two adjacent GPS track points. In this article, we propose the definition of the curvature integral in the context of map matching, and develop a novel accurate map matching algorithm based on the curvedness feature. Using real-world probe vehicles data, we show that the curvedness feature (CURF) constrained map matching method outperforms two classical methods for accuracy and stability under complicated road environments.

Sparse coding (SC) is making a significant impact in computer vision and signal processing communities, which achieves the state-of-the-art performance in a variety of applications for images, e.g., denoising, restoration, and synthesis. We propose an adaptive and robust SC algorithm exploiting the characteristics of typical laser range data and the availability of both range and reflectance data to realize range data denoising and inpainting. Specifically, our method estimates the informative level of each patch according to the variation in both range and reflectance modalities, followed by adaptive dictionary training that assigns dynamic sparsity weights to the patches with different informative levels. Furthermore, the l1-norm-based representation fidelity measure is applied to make our method robust to outliers which are common in laser range measurements. Extensive experiments on synthetic and real data demonstrate that our method works effectively, resulting in superior performance both visually and quantitatively, compared with competitive methods including the available sparse-representation-based algorithm, wavelets, partial differential equation, and non-local means.

Due to the diversity of geographical objects, it makes great sense to introduce multiscale segmentation/representation into the analysis and interpretation of high-spatial-resolution remote sensing images. However, with the increasing use of high-resolution images, traditional multiscale segmentation methods gradually show their lack in efficiency, particularly when handling large-scale images. In this paper, a novel bilevel scale-sets model (BSM) is proposed for multiscale region-based representation of large-scale remote sensing images. In the BSM, first, an image is divided into blocks with overlapped margins, and a low-level scale-sets model is blockwisely implemented. Second, a segmentation result is obtained by retrieving and mosaicking the blockwise segmentation results, based on which a high-level scale-sets model is implemented covering the whole image. To further improve the efficiency of the BSM, a parallel implementation is presented for the blockwise scale-sets model. In the experiments, first, the effectiveness of the BSM is validated using a WorldView2 image covering a coastal area of Shenzhen, where the BSM obtains accurate multiscale representation results without any mosaic artifacts. Then, the efficiency of the BSM is demonstrated by comparing with the state-of-the-art multiscale segmentation method, i.e., the one integrated in the commercial software eCognition v9.2, where the proposed BSM takes about 7 min to process a 24 000 × 24 000 multispectral ZY3 image and is two to three times faster than the competing method.

The large amount of spectral and spatial information contained in hyperspectral imagery has provided great opportunity to effectively characterize and identify the surface materials of interest. As a novel feature extraction technique, a series of Gabor wavelet filters with different scales and frequencies was applied on hyperspectral data to extract spectral-spatial-combined features, which produced impressive performance on pixel-oriented classification. However, the incredibly large number of Gabor features could cause too much burden for onboard computation, limiting the efficiency of the method. To make matters worse, due to the nonhomogeneous spatial distribution of materials as well as the different characteristics of the constructed Gabor filters, some Gabor features could have a smaller or even negative impact on material representation, deteriorating the classification accuracy eventually. In this paper, a Gabor cube selection based multitask joint sparse representation approach, abbreviated as GS-MTJSRC, was proposed for hyperspectral image classification. First, based on the Fisher discrimination criterion, the most representative Gabor cubes for each class were picked out. Next, under multitask joint sparse representation framework, a coefficient vector could be obtained for each test sample with the selected Gabor cube features, which could be directly used for the following residual-based classification. Experimental results on three real hyperspectral data sets with different characteristics and spatial resolutions demonstrated the feasibility and efficiency of the proposed method.

Through imaging the same spatial area by hyperspectral sensors at different spectral wavelengths simultaneously, the acquired hyperspectral imagery often contains hundreds of band images, which provide the possibility to accurately analyze and identify a ground object. However, due to the difficulty of obtaining sufficient labeled training samples in practice, the high number of spectral bands unavoidably leads to the problem of a “dimensionality disaster” (also called the Hughes phenomenon), and dimensionality reduction should be applied. Concerning band (or feature) selection, conventional methods choose the representative bands by ranking the bands with defined metrics (such as non-Gaussianity) or by formulating the band selection problem as a clustering procedure. Because of the different but complementary advantages of the two kinds of methods, it can be beneficial to use both methods together to accomplish the band selection task. Recently, a fast density-peak-based clustering (FDPC) algorithm has been proposed. Based on the computation of the local density and the intracluster distance of each point, the product of the two factors is sorted in decreasing order, and cluster centers are recognized as points with anomalously large values; hence, the FDPC algorithm can be considered a ranking-based clustering method. In this paper, the FDPC algorithm has been enhanced to make it suitable for hyperspectral band selection. First, the ranking score of each band is computed by weighting the normalized local density and the intracluster distance rather than equally taking them into account. Second, an exponential-based learning rule is employed to adjust the cutoff threshold for a different number of selected bands, where it is fixed in the FDPC. The proposed approach is thus named the enhanced FDPC (E-FDPC). Furthermore, an effective strategy, which is called the isolated-point-stopping criterion, is developed to automatically determine the appropriate number of bands to be selected. That is, the clustering process will be stopped by the emergence of an isolated point (the only point in one cluster). Experimental results on three real hyperspectral data demonstrate that the bands selected by our E-FDPC approach could achieve higher classification accuracy than the FDPC and other state-of-the-art band selection techniques, whereas the isolated-point-stopping criterion is a reasonable way to determine the preferable number of bands to be selected.

The assumption of reflection symmetry is one of the major deficiencies for model-based decompositions. The introduction of helix scattering components can mitigate some of the impact of this assumption while simultaneously generating some new negative power problems. The helix mechanism has been developed in the Krogager coherent target decomposition where there is no negative power problem. Why do these problems happen in model-based decompositions? In this paper, we review the Krogager decomposition based on the coherency matrix form of the unified CTD model and come to the conclusion that the imaginary part of T23is an overestimate of the helix contribution, which is the main cause of the problem. Furthermore, we review the techniques of symmetric scatterer transformation in the Cameron and Huynen decompositions and expand them to the multilook coherency matrix. Two new models of the nonlinear programming problem are proposed to get the transformed coherency matrix: 1) by subtracting the asymmetric component; and 2) by the procedure of con-diagonalization. A new parameter is established to measure the degree of reflection asymmetry of the coherency matrices. Experimental results in the L-band Airborne SAR (AIRSAR) San Francisco data show that the two new models can reduce the correlations between copolarized and cross-polarized channels significantly. In this case, the assumption of reflection symmetry is reasonable for the transformed coherency matrices, and negative power problems related to helix scattering no longer exist in the model-based decompositions; hence, the helix scattering component is not needed. Lastly, the Freeman and Durden decomposition has been applied to the transformed coherency matrices. Experimental results showed that the decomposition results can reflect the predominant characteristics of the ground objects.

Building orientation with respect to the radar look direction has a critical influence on the interpretation of multilook polarimetric synthetic aperture radar (PolSAR) data in urban areas. In this paper, its impacts on polarimetric orientation angle (POA) estimation and model-based decomposition are discussed. The discussion begins with the analysis of the general double-bounce scattering model, of which the characteristics are dependent on the electromagnetic and geometric parameters of the related dihedral structure. Then, for multilook PolSAR data, the polarimetric scattering mechanism in urban areas is modeled by two double-bounce scatterings from two orthogonal dihedral structures. From the model, the impacts of the building orientation on POA estimation can be revealed. With the increase of the building orientation, the POA difference between the two dihedral structures increases gradually, and the feasibility to estimate the building orientation via the estimated POA is reduced dramatically. Upon further analysis, we illustrate the impacts on the model-based decomposition. With the increase of the building orientation, the dominant scattering mechanism labeling technique based on the model-based decompositions will gradually become invalid. Moreover, the processing of POA compensation, which is helpful in reducing the impacts of the building orientation, also becomes invalid when the building orientation increases to a certain value. At last, three L-band data sets of San Francisco acquired by AIRSAR are used to verify the inferences. The experimental results show that, for L-band PolSAR data in urban areas, when the radar look angle is around 45, the threshold of building orientation for the validity of dominant scattering mechanism labeling is about ±3, and for the POA compensation, the threshold is about ±12.

Coherence loss is a critical issue in interferometric synthetic aperture radar geodesy, particularly when short-wavelength radar images are used to monitor earthquake deformation, and it may result in isolated fringes in an interferogram. The conventional unwrapping algorithms may incompletely unwrap or wrongly estimate the integer jumps between isolated fringes. In this paper, we propose a novel method to connect the isolated fringes in earthquake slip inversion. We use multiple starting points to unwrap the interferogram and then solve the integer ambiguities among the starting points by a dislocation-model-based integer least squares estimator. This estimator allows us to provide a quantitative evaluation of the reliability of the integer solutions in terms of two indicators (the success rate and residual ratio). The algorithm is robust to a certain degree of data noise and fault geometry error, as tested. Simulated experiments and case studies demonstrate that the proposed method can give better unwrapping results than the conventional approaches such as the minimum-cost flow (MCF), statistical-cost network-flow algorithm for phase unwrapping (SNAPHU), and iterative forms of MCF and SNAPHU with the assistance of a slip model. The earthquake slip inversion therefore benefits from the more accurate unwrapping results.

Lane-based road network information, such as the number and locations of traffic lanes on a road, has played an important role in intelligent transportation systems. In this paper, we propose a Collecting Lane-based Road Information via Crowdsourcing (CLRIC) method, which can automatically extract detailed lane structure of roads by using crowdsourcing data collected by vehicles. First, CLRIC filters the high-precision GPS data from the raw trajectories based on region growing clustering with prior knowledge. Second, CLRIC mines the number and locations of traffic lanes through optimized constrained Gaussian mixture model. Experiments are conducted with taxi GPS trajectories in Wuhan, China, and the results show that CLRIC is quantified and displays detailed road networks with the number and locations of traffic lanes comparing with the satellite image and human-interpreted situation.

As a pre-processing tool, superpixel algorithms have been popular used in many computer-vision applications. High efficiency is a desired property of superpixel algorithms, especially in real-time vision systems. In this paper, a novel high-efficient superpixel algorithm is developed based on the watershed algorithm, namely the spatial-constrained watershed (SCoW). SCoW performs watersheding in a marker-controlled manner, with a set of evenly placed markers. To align superpixel boundaries to image edges, an edge-preserving scheme is embedded into the SCoW which makes a balance between the homogeneity and the compactness. Without any complex computing, the proposed superpixel algorithm is found to produce high quality superpixels as traditional superpixel algorithms, while holding much higher efficiency.

Traditional color feature descriptors are focused on color-value distributions in the color space, e.g., color histograms, color bag-of-words, which ignore the spatial location and contextual information of different colors. In this paper, a new regional color co-occurrence feature descriptor (RCC) is proposed to reflect spatial relations of colors in an image. First, we partition an image into a number of disjoint regions using superpixel techniques. Then, we construct a color histogram for each region, based on which we construct a color co-occurrence matrix for each pair of neighboring regions. Finally, all the constructed co-occurrence matrices from an image are summed up and normalized as a color descriptor to represent this image. This new color descriptor reflects the color-collocation patterns in the image. We use this new color descriptor for image/object classification and find that it leads to higher classification accuracies than other competing color descriptors.

Sparse representation (SR) is making significant impact in the computer vision and signal processing communities due to its stunning performance in a variety of applications for images, e.g., denoising, restoration, and synthesis. We propose an adaptive and robust SR algorithm that exploits the characteristics of typical laser range data, i.e., the availability of both range and reflectance data, to realize range data denoising. Specifically, our method estimates the informative level (IL) of each patch according to the variation in both range and reflectance modalities, followed by adaptive dictionary training that assigns dynamic sparsity weights to the patches with different ILs. Furthermore, the l1-norm-based representation fidelity measure is applied to make our method robust to outliers, which are common in laser range measurements. Extensive experiments on synthesized and actual data demonstrate that our method works effectively, resulting in superior performance both visually and quantitatively.

This letter presents preliminary results of sugarcane mapping in its tillering period using quad-polarization data. Three TerraSAR-X images were collected in early, middle, and late tillering periods in the Leizhou Peninsula, South China. Polarization features based on Cloude-Pottier decomposition, such as scattering angle and polarization entropy, were used to analyze sugarcane scattering mechanism temporal behavior and the differences between sugarcane and other typical land-cover types. Results show good separability of sugarcane from other land-cover types. At last, based on common features of sugarcane in tillering period, a sugarcane mapping method was proposed with certain high accuracy and good stability. The results suggest that the tillering period, from early April to late May, is a suitable growth period for sugarcane mapping, and polarization features appear promising for crop mapping and monitoring.

Global Positioning System/Inertial Navigation System (GPS/INS) integrated navigation systems play a very important role in modern photogrammetry and laser scanning by virtue of their capability of direct measurement of high-precision position and attitude data in the WGS 84 datum. In practice, as georeferencing is often conducted in national coordinates, there is a need to transform GPS/INS data to the required national map projection frame first. This letter presents an improved coordinate-transformation-based method for the GPS/INS attitude transformation by taking the datum scale distortion and the length distortion into account. Experimental results show that the transformation errors of our improved method are on the order of magnitude of 1 × 10-5°, which can be safely ignored in aerial photogrammetric processing, whereas the maximum error of the previous coordinate-transformation-based method can be up to several 0.001°.

Sparse-representation-based classification (SRC) assigns a test sample to the class with minimum representation error via a sparse linear combination of all the training samples, which has successfully been applied to several pattern recognition problems. According to compressive sensing theory, the l1-norm minimization could yield the same sparse solution as the l0norm under certain conditions. However, the computational complexity of the l1-norm optimization process is often too high for large-scale high-dimensional data, such as hyperspectral imagery (HSI). To make matter worse, a large number of training data are required to cover the whole sample space, which is difficult to obtain for hyperspectral data in practice. Recent advances have revealed that it is the collaborative representation but not the l1-norm sparsity that makes the SRC scheme powerful. Therefore, in this paper, a 3-D Gabor feature-based collaborative representation (3GCR) approach is proposed for HSI classification. When 3-D Gabor transformation could significantly increase the discrimination power of material features, a nonparametric and effective l2-norm collaborative representation method is developed to calculate the coefficients. Due to the simplicity of the method, the computational cost has been substantially reduced; thus, all the extracted Gabor features can be directly utilized to code the test sample, which conversely makes the l2-norm collaborative representation robust to noise and greatly improves the classification accuracy. The extensive experiments on two real hyperspectral data sets have shown higher performance of the proposed 3GCR over the state-of-the-art methods in the literature, in terms of both the classifier complexity and generalization ability from very small training sets.

This paper presents an activity sequence-based indoor pedestrian localization approach using smartphones. The activity sequence consists of several continuous activities during the walking process, such as turning at a corner, taking the elevator, taking the escalator, and walking stairs. These activities take place when a user walks at some special points in the building, like corners, elevators, escalators, and stairs. The special points form an indoor road network. In our approach, we first detect the user's activities using the built-in sensors in a smartphone. The detected activities constitute the activity sequence. Meanwhile, the user's trajectory is reckoned by Pedestrian Dead Reckoning (PDR). Based on the detected activity sequence and reckoned trajectory, we realize pedestrian localization by matching them to the indoor road network using a Hidden Markov Model. After encountering several special points, the location of the user would converge on the true one. We evaluate our proposed approach using smartphones in two buildings: an office building and a shopping mall. The results show that the proposed approach can realize autonomous pedestrian localization even without knowing the initial point in the environments. The mean offline localization error is about 1.3 m. The results also demonstrate that the proposed approach is robust to activity detection error and PDR estimation error.

Indoor maps are integral to pedestrian navigation systems, an essential element of intelligent transportation systems (ITS). In this paper, we propose ALIMC, i.e., Activity Landmark-based Indoor Mapping system via Crowdsourcing. ALIMC can automatically construct indoor maps for anonymous buildings without any prior knowledge using crowdsourcing data collected by smartphones. ALIMC abstracts the indoor map using a link-node model in which the pathways are the links and the intersections of the pathways are the nodes, such as corners, elevators, and stairs. When passing through the nodes, pedestrians do the corresponding activities, which are detected by smartphones. After activity detection, ALIMC extracts the activity landmarks from the crowdsourcing data and clusters the activity landmarks into different clusters, each of which is treated as a node of the indoor map. ALIMC then estimates the relative distances between all the nodes and obtains a distance matrix. Based on the distance matrix, ALIMC generates a relative indoor map using the multidimensional scaling technique. Finally, ALIMC converts the relative indoor map into an absolute one based on several reference points. To evaluate ALIMC, we implement ALIMC in an office building. Experiment results show that the 80th percentile error of the mapping accuracy is about 0.8-1.5 m.

Large-scale global positioning system (GPS) positioning information of floating cars has been recognised as a major data source for many transportation applications. Mapping large-scale low-frequency floating car data (FCD) onto the road network is very challenging for traditional map-matching (MM) algorithms developed for in-vehicle navigation. In this paper, a multi-criteria dynamic programming map-matching (MDP-MM) algorithm is proposed for online matching FCD. In the proposed MDP-MM algorithm, the MDP technique is used to minimise the number of candidate routes maintained at each GPS point, while guaranteeing to determine the best matching route. In addition, several useful techniques are developed to improve running time of the shortest path calculation in the MM process. Case studies based on real FCD demonstrate the accuracy and computational performance of the MDP-MM algorithm. Results indicated that the MDP-MM algorithm is competitive with existing algorithms in both accuracy and computational performance.

Downsampling is a routine step before applying interferometric synthetic aperture radar (InSAR) data to earthquake inversion because of the high computational burden. In this letter, we make use of the matrix perturbation theory to describe the downsampling process, which is considered as matrix perturbation on inversion equation. First, we derive a formula to quantitatively assess the perturbation on the inversion solution caused by data downsampling. Next, we propose an equation-based InSAR data downsampling algorithm to better reduce the perturbation. The experiment with simulated data demonstrates that our new algorithm preserves the most details from full data inversion comparing with previous algorithms. Finally, we use our method to study the slip distribution of the 2008 Mw 6.3 Dangxiong earthquake.

Local search heuristics for very large-scale vehicle routing problems (VRPs) have made remarkable advances in recent years. However, few local search heuristics have focused on the use of the spatial neighborhood in Voronoi diagrams to improve local searches. Based on the concept of a k-ring shaped Voronoi neighbor, we propose a Voronoi spatial neighborhood-based search heuristic and algorithm to solve very large-scale VRPs. In this algorithm, k-ring Voronoi neighbors of a customer are limited to building and updating local routings, and rearranging local routings with improper links. This algorithm was evaluated using four sets of benchmark tests for 200–8683 customers. Solutions were compared with specific examples in the literature, such as the one-depot VRP. This algorithm produced better solutions than some of the best-known benchmark VRP solutions and requires less computational time. The algorithm outperformed previous methods used to solve very large-scale, real-world distance constrained capacitated VRP.

This letter presents a novel method for automated footprint extraction of building facades from mobile LiDAR point clouds. The proposed method first generates the georeferenced feature image of a mobile LiDAR point cloud and then uses image segmentation to extract contour areas which contain facade points of buildings, points of trees, and points of other objects in the georeferenced feature image. After all the points in each contour area are extracted, a classification based on principal component analysis (PCA) method is adopted to identify building objects from point clouds extracted in contour areas. Then, all the points in a building object are segmented into different planes using the random sample consensus algorithm. For each building, points in facade planes are chosen to calculate the direction, the start point, and the end point of the facade footprints using PCA. Finally, footprints of different facades of building are refined, harmonized, and joined. Two data sets of downtown areas and one data set of a residential area captured by Optech's LYNX mobile mapping system were tested to verify the validities of the proposed method. Experimental results show that the proposed method provides a promising and valid solution for automatically extracting building facade footprints from mobile LiDAR point clouds.

In this paper, a bottom–up vehicle emission model is proposed to estimate real-time$\hbox{CO}_{2}$emissions using intelligent transportation system (ITS) technologies. In the proposed model, traffic data that were collected by ITS are fully utilized to estimate detailed vehicle technology data (e.g., vehicle type) and driving pattern data (e.g., speed, acceleration, and road slope) in the road network. The road network is divided into a set of small road segments to consider the effects of heterogeneous speeds within a road link. A real-world case study in Beijing, China, is carried out to demonstrate the applicability of the proposed model. The spatiotemporal distributions of$ \hbox{CO}_{2}$emissions in Beijing are analyzed and discussed. The results of the case study indicate that ITS technologies can be a useful tool for real-time estimations of$\hbox{CO}_{2}$emissions with a high spatiotemporal resolution.

As travel times in road networks are dynamic and uncertain, it is difficult and time-consuming to search for the least expected time path in large-scale networks. This paper addresses the problem of finding the least expected time path in stochastic time-dependent (STD) road networks. A stochastic travel speed model is proposed to represent STD link travel times. It is proved that the link travel times in STD networks satisfy the stochastic first-in-first-out (S-FIFO) property. Based on this S-FIFO property, an efficient multicriteria A* algorithm is proposed to exactly determine the least expected time path in STD networks. Computational results using several large-scale road networks show that the proposed algorithm has a significant computational advantage over existing solution algorithms without the S-FIFO property.

Finding road intersections in advance is crucial for navigation and path planning of moving autonomous vehicles, especially when there is no position or geographic auxiliary information available. In this paper, we investigate the use of a 3D point cloud based solution for intersection and road segment classification in front of an autonomous vehicle. It is based on the analysis of the features from the designed beam model. First, we build a grid map of the point cloud and clear the cells which belong to other vehicles. Then, the proposed beam model is applied with a specified distance in front of autonomous vehicle. A feature set based on the length distribution of the beam is extracted from the current frame and combined with a trained classifier to solve the road-type classification problem, i.e., segment and intersection. In addition, we also make the distinction between +-shaped and T-shaped intersections. The results are reported over a series of real-world data. A performance of above 80% correct classification is reported at a real-time classification rate of 5 Hz.

Landmarks provide the most predominant navigation cue for pedestrian navigation. Very few navigation data models in the geographical information science and transportation communities support modeling of landmarks and use of landmark-based route instructions for pedestrian navigation services. This article proposes a landmark-based pedestrian navigation data model to fill this gap. This data model can model landmarks in several pedestrian navigation scenarios (buildings, open spaces, multimodal transportation systems, and urban streets). This article implements the proposed model in the ArcGIS software environment and demonstrates two typical pedestrian navigation scenarios: (1) a multimodal pedestrian navigation environment involving bus lines, parks, and indoor spaces and (2) a subway system in a metropolitan environment. These two scenarios illustrate the feasibility of the proposed data model in real-world environments. Further improvements of this model could lead to more intuitive and user-friendly landmark-based pedestrian navigation services than the functions supported by current map-based navigation systems.

The interpolation of link-based traffic data is an important topic for transportation researchers and engineers. In recent years the kriging method has been used in traffic data interpolation from the viewpoint of spatial analysis. This method has shown promising results, especially for a large-scale road network. However, existing studies using the Euclidean distance metric, which is widely used in traditional kriging, fail to accurately describe the spatial distance in a road network. In this article we introduce road network distance to describe spatial distance between road links, and we propose an improved distance metric called approximate road network distance (ARND), based on the isometric embedding theory, for solving the problem of the invalid spatial covariance function in kriging caused by the non-Euclidean distance metric. An improved Isomap algorithm is also proposed for obtaining the ARND metric. This study is tested on a large-scale urban road network with sparse road-link travel speeds derived from approximately 1200 ‘floating cars’ (GPS-enabled taxis). Comparison was conducted on both the Euclidean distance metric and the ARND metric. The validation results show that the use of the ARND metric can obtain better interpolation accuracy in different time periods and urban regions with different road network structures. Therefore, we conclude that the improved distance metric has the ability for improving kriging interpolation accuracy for link-based traffic data within real situations, providing more reliable basic traffic data for various traffic applications.

In this paper, we propose a computer vision based system for real-time robust traffic sign detection and recognition, especially developed for intelligent vehicle. In detection phase, a color-based segmentation method is used to scan the scene in order to quickly establish regions of interest (ROI). Sign candidates within ROIs are detected by a set of Haar wavelet features obtained from AdaBoost training. Then, the Speeded Up Robust Features (SURF) is applied for the sign recognition. SURF finds local invariant features in a candidate sign and matches these features to the features of template images that exist in data set. The recognition is performed by finding out the template image that gives the maximum number of matches. We have evaluated the proposed system on our intelligent vehicle SmartVII. A recognition accuracy of over 90% in real-time has been achieved.

Landmarks provide the most predominant navigational cue for pedestrian navigation. The choice and representation of landmarks require an optimal approach to meet the needs of pedestrians, for example, shorter distances, fewer turns, and easy confirmation. This article proposes a multiobjective model to generate optimal landmark sequences for pedestrian route instructions. This model offers a general approach to meet the diverse needs of pedestrians. A modified ant colony optimization (ACO) algorithm is used to implement the proposed model. This research determined the parameters of the modified ACO algorithm by testing the whole study area using various weight combinations. This article also discusses the optimization process for various cases, for example, different origin–destination (OD) pairs, different pedestrian study areas, the influence of landmark density, and the overall performance comparison in achieving four objectives. Experimental results have confirmed that this proposed model can optimize landmark sequences for pedestrian route instructions.

Strokes are products of a higher-level aggregation of street segments that can reflect functional importance and perceptual significance that is associated with them in human spatial mental conceptualizations, which is of vital importance for network analysis, street selection, and map generalization. Street properties (e.g., street names) and angles between street segments are the two main elements used for generating street strokes according to the continuity principle of perceptual grouping into networks. However, it is difficult to automatically generate strokes with good continuity from street networks with multiple lanes such as dual carriageways or complex street junctions. This article proposes a method for generating street strokes that maintain good continuity across multiple lanes and complex street junctions. The proposed method first detects dual carriageways and complex junctions in street networks and then generates strokes according to the continuity principle of perceptual grouping. Finally, it groups the generated street strokes across the dual carriageways and complex street junctions to maintain good continuity. Moreover, the generated strokes are hierarchically ranked based on stroke length and centrality measurements. Experimental studies demonstrate the validity and effectiveness of the proposed method. The result shows that the generated street strokes maintain good continuity and reflect well the hierarchical structure of the street networks.

Considering the plentiful road markings in China, we present a Block-Constraint Line scanning (BCLS) method for lane detection in this paper. In this method, images are firstly pre-processed by a morphological top-hat transform, and then an imaging model is created for building relationship between lane parameters of the image coordinate and the WGS coordinate, from which target points on lane lines could be retained by a block-constraint line scanning algorithm. Finally, lanes could be extracted by a Progressive Probabilistic Hough Transform (PPHT) and the number of lanes is figured out through clustering. Our method is fast enough to meet real-time requirement. Experiments were carried out on the intelligent vehicle SmartV (Fig.1) on the Wuhan urban roads in China and the results show that this method can efficiently and accurately extract lanes in complex environments, even with the presence of non-lane road markings.

Multi‐resolution terrain models are an efficient approach to improve the speed of three‐dimensional (3D) visualizations, especially for terrain visualization in Geographical Information Systems (GIS). As a further development to existing algorithms and models, a new model is proposed for the construction of multi‐resolution terrain models in a 3D GIS. The new model represents multi‐resolution terrains using two major methods for terrain representation: Triangulated Irregular Network (TIN) and regular grid (Grid). In this paper, first, the concepts and formal definitions of the new model are presented. Second, the methodology for constructing multi‐resolution terrain models based on the new model is proposed. Third, the error of multi‐resolution terrain models is analysed, and a set of rules is proposed to retain the important features (e.g. boundaries of man‐made objects) within the multi‐resolution terrain models. Finally, several experiments are undertaken to test the performance of the new model. The experimental results demonstrate that the new model can be applied to construct multi‐resolution terrain models with good performance in terms of time cost and maintenance of the important features. Furthermore, a comparison with previous algorithms/models shows that the speed of rendering for 3D walking/flying through has been greatly improved by applying the new model.

There are many image fusion methods that can be used to produce high-resolution multispectral images from a high-resolution panchromatic image and low-resolution multispectral images. Starting from the physical principle of image formation, this paper presents a comprehensive framework, the general image fusion (GIF) method, which makes it possible to categorize, compare, and evaluate the existing image fusion methods. Using the GIF method, it is shown that the pixel values of the high-resolution multispectral images are determined by the corresponding pixel values of the low-resolution panchromatic image, the approximation of the high-resolution panchromatic image at the low-resolution level. Many of the existing image fusion methods, including, but not limited to, intensity-hue-saturation, Brovey transform, principal component analysis, high-pass filtering, high-pass modulation, the a/spl grave/ trous algorithm-based wavelet transform, and multiresolution analysis-based intensity modulation (MRAIM), are evaluated and found to be particular cases of the GIF method. The performance of each image fusion method is theoretically analyzed based on how the corresponding low-resolution panchromatic image is computed and how the modulation coefficients are set. An experiment based on IKONOS images shows that there is consistency between the theoretical analysis and the experimental results and that the MRAIM method synthesizes the images closest to those the corresponding multisensors would observe at the high-resolution level.

Developing a three-dimensional (3D) data model for Geographic Information Systems (GIS) is an essential and complex issue. 3D modelling in GIS is becoming ever more important for the development of cyber cities and digital earth, which have recently become feasible. A competent 3D model forms an efficient foundation for 3D visualization, query and spatial analysis. As a development of the existing 3D models, this study proposes particular improvements in handling complex 3D objects. We present an object-oriented data model for handling complex 3D objects in GIS. First, the conceptual data model is developed based on the principle of object-oriented (OO) data modelling. This model is designed based on the following three basic geometric elements: node, segment and triangle. Accordingly, the abstract geometric objects are defined: including points, lines, surfaces and volumes. Second, the corresponding 3D logical model is designed based on the defined abstract objects and the relationships between them. Third, a formal representation of the 3D spatial objects is described in detail. Fourth, a prototype 3D GIS is developed based on the proposed 3D data model. Finally, we describe the results of an experimental study to reconstruct 3D objects using this 3D GIS and a comparison with the performance of other 3D data models. The proposed model is able to handle complex objects, such as complex buildings and TV towers, which is an essential functionality for building large-scale cyber cities, such as for Hong Kong. The proposed data model proves to be very efficient, particularly in visualization and rendering. The experimental results show which the data volume of the proposed model is compacted and the visualization speed for 3D objects is improved, compared with the existing models.