Executing neural networks on resource-constrained embedded devices faces challenges. Efforts have been made at the application and system levels to reduce the execution cost. Among them, the early-exit networks reduce computational cost through intermediate exits, while Dynamic Voltage and Frequency Scaling (DVFS) offers system energy reduction. Existing works strive to unify early-exit and DVFS for combined benefits on both timing and energy flexibility, yet limitations exist: 1) varying time constraints that make different exit points become more, or less, important in terms of inference accuracy, are not taken care of, and 2) the optimal decisions of unifying DVFS and early-exit as a multi-objective optimization problem are not achieved due to the large configuration space. To address these challenges, we propose Dr2r, a reinforcement learning-based framework that jointly optimizes early-exit points and DVFS settings for continuous inference. In particular, Dr2r includes a cross-training mechanism that fine-tunes the early-exit network to accommodate dynamic time constraints and system conditions. Experimental results demonstrate that Dr2r achieves up to 22.03% energy reduction and 3.23% accuracy gain compared to contemporary techniques.

Although 2D radar signal representations, such as spectrograms and range-Doppler maps have been widely used for target recognition, 3D time-range-Doppler (TRD) has been less studied, partially because of the difficulties in extracting features from the TRD representation, i.e., shallow 3D neural networks have limited discriminant power, but repeatedly applying 3D convolutions will lead to an oversized 3D network. A hybrid 3D–2D network architecture, Progressively-Orthogonally-Mapped EfficientNet (POMEN), is proposed to address these challenges. More specifically, the proposed POMEN utilizes 3D convolutions in the earlier stages to capture the information embedded in the sparse 3D TRD representation, and to avoid the oversized feature map caused by excessively applying 3D convolutions, we propose to progressively map the 3D features into three sets of 2D features corresponding to the range-time signature, range-Doppler map and time-Doppler signature (spectrogram), respectively. Subsequently, 2D EfficientNet blocks were designed to extract discriminant information from the three sets of 2D feature maps. This hybrid 3D–2D network design effectively extracts features from the 3D TRD representation, thereby avoiding oversized features from full-sized 3D networks and the information loss of 2D networks on 2D representations. Finally, a homogeneous gated fusion network was designed to fuse the three sets of 2D features. The proposed method was evaluated on the UGRS, MIMOGR, and mmWRWD datasets. The experimental results for all datasets demonstrate that the proposed POMEN significantly and consistently outperforms the state-of-the-art models in both 2D and 3D representations.

In recent years, deep reinforcement learning has shown great potential in solving computer games with sequential decision-making scenarios. Hyper-heuristic is a generic search framework, capable of intelligently selecting or generating algorithms to solve a class of optimisation problems with stochastic or dynamic settings. This paper proposes a new general framework for solving online packing problems using deep reinforcement learning hyper-heuristics. Although analytical approaches can address most offline packing problems successfully, their online versions have proved much more challenging and the performance of the existing methods is often not satisfactory. In this paper, we extend a recent deep reinforcement learning hyper-heuristic framework by fusing the visual information of real-time packing with distributional information of random parameters of the problem. Computational experiments show that our method outperforms the state of the art online methods with reductions in optimality gap between 2%–19% for knapsack problem and 0.7% for the online strip packing problem. In addition, a new visual analysis presentation is also devised to better interpret the learned packing strategies, which can reveal more information than the widely used landscape analysis. As online packing problems are widely available in production environments, the proposed approach can serve as an important reference to solve other similar combinatorial optimisation problems for which visual layout inputs would aid learning.

We propose a logic of east and west (LEW ) for points in 1D Euclidean space. It formalises primitive direction relations: east (E), west (W) and indeterminate east/west (Iew). It has a parameter τ ∈ N>1, which is referred to as the level of indeterminacy in directions. For every τ ∈ N>1, we provide a sound and complete axiomatisation of LEW , and prove that its satisfiability problem is NP-complete. In addition, we show that the finite axiomatisability of LEW depends on τ : if τ = 2 or τ = 3, then there exists a finite sound and complete axiomatisation; if τ > 3, then the logic is not finitely axiomatisable. LEW can be easily extended to higher-dimensional Euclidean spaces. Extending LEW to 2D Euclidean space makes it suitable for reasoning about not perfectly aligned representations of the same spatial objects in different datasets, for example, in crowd-sourced digital maps.

Detecting 3D mask attacks to a face recognition system is challenging. Although genuine faces and 3D face masks show significantly different remote photoplethysmography (rPPG) signals, rPPG-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence and weak rPPG signals. To enhance the rPPG signal in a motion-robust way, a landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. To better encode the rPPG signal, a weighted spatial-temporal representation is proposed, which emphasizes the face regions with rich blood vessels. In addition, characteristics of rPPG signals in different color spaces are jointly utilized. To improve the generalization capability, a lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features from the rPPG spatial-temporal representation for classification. The proposed method is compared with the state-of-the-art methods on five benchmark datasets under both intra-dataset and cross-dataset evaluations. The proposed method shows a significant and consistent improvement in performance over other state-of-the-art rPPG-based methods for face spoofing detection.

Entity re-identification is the foundation of tracking- and matching-based computer vision tasks, which are widely employed in a variety of applications. However, when trained exclusively on clear images, the models capacity to generalize is significantly affected by the presence of occlusion at referencing time, whereas data argumentation-based approaches are costly to construct without guaranteeing a test-time improvement. To tackle this problem, we propose a domain adaptation framework based on learning representations that generates occlusion-invariant feature representations by aligning the clean image embedding distribution with the occluded one, using a disparity discrepancy metric derived from the siamese network architecture. Without the need for additional processing modules during the inference stage or an expensive occlusion-augmentation-enlarged dataset during the training stage, we could obtain occlusion invariant embeddings that are free of the impact of occluders. Extensive experimental results for two tasks across three datasets indicate the proposed method’s robustness and effectiveness to a variety of occlusions at all levels.

Face spoofing detection against paper attack and video-replay attack has been well studied, whereas detecting 3D face mask attack remains challenging. Remote photoplethysmography (rPPG) signal is a recently developed liveness clue for face-spoofing detection. The main challenge of existing rPPG-based methods is that the signal can be easily distorted by background noise or object motion. To address this problem, in this work, we propose an rPPG-based face-spoofing detection method using multiple regions of interests (ROIs) covering entire face, and emphasize the regions containing richer rPPG signals using larger weights. The rPPG signals of these regions form a weighted spatial-temporal map. In view of the discriminant power of EfficientNet over other deep convolutional neural networks, we propose a domain-specific EfficientNet as the classification method. Extensive experiments on two databases namely 3DMAD and HKBU-Mars V2 demonstrate the superior performance of the proposed method over state-of-the-art rPPG-based face-spoofing-detection algorithms.

We propose a logic of directions for points (LD) over 2D Euclidean space, which formalises primary direction relations east (E), west (W), and indeterminate east/west (Iew), north (N), south (S) and indeterminate north/south (Ins). We provide a sound and complete axiomatisation of it, and prove that its satisfiability problem is NP-complete.

Urban infrastructure assets (e.g. roads, water pipes) perform critical functions to the health and well-being of society. Although it has been widely recognised that different infrastructure assets are highly interconnected, infrastructure management in practice such as planning, installation and maintenance are often undertaken by different stakeholders without considering these dependencies due to the lack of relevant data and cross-domain knowledge, which may cause unexpected cascading social, economic and environmental effects. In this paper, we present a knowledge based decision support system for urban infrastructure inter-asset management. By considering various infrastructure assets (e.g. road, ground, cable), triggers (e.g. pipe leaking) and potential consequences (e.g. traffic disruption) as a holistic system, we model each sub-domain using a modular ontology and encapsulate the interdependence between them using a set of rules. Moreover, qualitative likelihood is assigned to each rule by domain experts (e.g. civil engineers) to encode the uncertainty of knowledge, and an inference engine is applied to predict the potential consequences of a given trigger with location specific data and the encoded rules. A web-based prototype system has been developed based on the above concept and demonstrated to a wide range of stakeholders. The system can assist in the process of decision making by aiding data collation and integration, as well as presenting potential consequences of possible triggers, advising on whether additional information is needed or suggesting ways of obtaining such information. The work shows an intelligent approach to integrate and process multi-source data to pioneer a novel way to aid a complex decision process with a high social impact.

We present an interactive decision support system for assisting city infrastructure inter-asset management. It combines real-time site specific data retrieval, a knowledge base co-created with domain experts and an inference engine capable of predicting potential consequences and risks resulting from the available data and knowledge. The system can give explanations of each consequence, cope with incomplete and uncertain data by making assumptions about what might be the worst case scenario, and making suggestions for further investigation. This demo presents multiple real-world scenarios, and demonstrates how modifying assumptions (parameter values) can lead to different consequences.

This paper aims to lay a foundation for a systematic study of mechanisms for construction of definitions within a formal theory, by investigating operators for incremental construction of definitions of new relations from an existing set of primitives and previously defined relations. To illustrate our method, we apply it to two of the best known relation sets studied in KRR: Allen's Interval Algebra and Region Connection Calculus. We also show that systematic exploration of definitional possibilities can yield interesting insights into relation sets that were originally defined in a more ad hoc way, and opens the possibility for discovering new vocabulary for extending or refining existing calculi or for developing completely new calculi.

Assessing the Underworld (ATU) is a large interdisciplinary UK research project, which addresses challenges in integrated inter-asset maintenance. As assets on the surface of the ground (e.g. roads or pavements) and those buried under it (e.g. pipes and cables) are supported by the ground, the properties and processes of soil affect the performance of these assets to a significant degree. In order to make integrated decisions, it is necessary to combine the knowledge and expertise in multiple areas, such as roads, soil, buried assets, sensing, etc. This requires an underpinning knowledge model, in the form of an ontology. Within this context, we present a new ontology for describing soil properties (e.g. soil strength) and processes (e.g. soil compaction), as well as how they affect each other. This ontology can be used to express how the ground affects and is affected by assets buried under the ground or on the ground surface. The ontology is written in OWL 2 and openly available from the University of Leeds data repository: http://doi.org/10.5518/54 .

This paper describes a series of new qualitative spatial logics for checking consistency of sameAs and partOf matches between spatial objects from different geospatial datasets, especially from crowd-sourced datasets. Since geometries in crowd-sourced data are usually not very accurate or precise, we buffer geometries by a margin of error or a level of tolerance, and define spatial relations for buffered geometries. The spatial logics formalize the notions of `buffered equal' (intuitively corresponding to `possibly sameAs'), `buffered part of' (`possibly partOf'), `near' (`possibly connected') and `far' (`definitely disconnected'). A sound and complete axiomatisation of each logic is provided with respect to models based on metric spaces. For each of the logics, the satisfiability problem is shown to be NP-complete. Finally, we briefly describe how the logics are used in a system for generating and debugging matches between spatial objects, and report positive experimental evaluation results for the system.

We describe a tool, MatchMaps, that generates sameAs and partOf matches between spatial objects (such as shops, shopping centres, etc.) in crowd-sourced and authoritative geospatial datasets. MatchMaps uses reasoning in qualitative spatial logic, description logic and truth maintenance techniques, to produce a consistent set of matches. We report the results of an initial evaluation of MatchMaps by experts from Ordnance Survey (Great Britain’s National Mapping Authority). In both the case studies considered, MatchMaps was able to correctly match spatial objects (high precision and recall) with minimal human intervention.

We propose a new qualitative spatial logic for reasoning about part-whole relations between geometries (sets of points) represented in different geospatial datasets, in particular crowd-sourced datasets. Since geometries in crowd-sourced data can be less inaccurate or precise, we buffer geometries by a margin of error or level of tolerance σ, and define part-whole relation for buffered geometries. The relations between geometries considered in the logic are: buffered part of (BPT), Near and Far. We provide a sound and complete axiomatisation of the logic with respect to metric models, and show that its satisfiability problem is NP-complete.