Effectively transferring knowledge from one task to another in cooperative Multi-Agent Reinforcement Learning (MARL) can be key to accelerate learning in complex scenarios. For instance, in sports it is common to train in simpler situations and then apply what was learned in the real game. The same logic applies to other scenarios that involve increasing levels of difficulty, such as robotics tasks, or healthcare applications. In the realm of MARL, transferring knowledge can become extremely challenging due to factors such as changes in the observation and action spaces, or the underlying dynamics of the environment. Consequently, most current methods still opt to train each new task from scratch. However, as tasks become increasingly complex, there is a growing interest in leveraging behavioural similarities that emerge among semantically similar tasks. In this context, we explore techniques to facilitate the rapid transfer of knowledge from one policy network to another within off-policy value-based methods. Additionally, we introduce a special case that enables function-preserving transfers of centralised functions between tasks. Our work offers a promising strategy to reduce training time, enabling zero-shot task transfers.

Neuromorphic computing is revolutionising artificial vision by emulating the human brain’s remarkable efficiency, adaptability, and spatio-temporal processing. This review synthesises recent advances in neuromorphic vision, with a special focus on wave-based dynamics; particularly the role of cortical travelling waves and neural oscillations in coordinating activity across the visual cortex. We examine how these biological mechanisms inspire cutting-edge computational models, including Physics-Informed Neural Networks, reservoir computing, and spiking neural networks, each enabling real-time, energy-efficient visual processing. The review also highlights breakthroughs in hardware, from memristive devices and photonic circuits to optoelectronic polymers, which support in-sensor and event-based processing while dramatically reducing power consumption. By integrating insights from computational neuroscience, materials science, and machine intelligence, we identify persistent challenges; such as scalable training, robust hardware integration, and biologically plausible modelling and outline actionable directions for future research. Our synthesis provides a comprehensive roadmap for the next generation of neuromorphic vision systems, paving the way toward artificial perception that rivals the efficiency and adaptability of biological vision.

In cooperative Multi-Agent Reinforcement Learning (MARL) agents are required to learn behaviours as a team to achieve a common goal. However, while learning a task, some agents may end up learning sub-optimal policies, not contributing to the objective of the team. Such agents are called lazy agents due to their non-cooperative behaviours that may arise from failing to understand whether they caused the rewards. As a consequence, we observe that the emergence of cooperative behaviours is not necessarily a byproduct of being able to solve a task as a team. In this paper, we investigate the applications of causality in MARL and how it can be applied in MARL to penalise these lazy agents. We observe that causality estimations can be used to improve the credit assignment to the agents and show how it can be leveraged to improve independent learning in MARL. Furthermore, we investigate how Amortised Causal Discovery can be used to automate causality detection within MARL environments. The results demonstrate that causality relations between individual observations and the team reward can be used to detect and punish lazy agents, making them develop more intelligent behaviours. This results in improvements not only in the overall performances of the team but also in their individual capabilities. In addition, results show that Amortised Causal Discovery can be used efficiently to find causal relations in MARL.

Analysis of player tracking and event data in football matches is used by the coaching staff to evaluate team performance and to inform tactical decision-making, whereas using Machine Learning methods to gain useful insights from the data is still an open research question. The objective of our research is to discover the football team's space-control efficiency using a novel Machine Learning approach and evaluate the team performance based on its space-control efficiency. We develop a novel Possession Evaluation Model through deep generative machine learning to predict the football team's space-control capability utilizing tracking and event data. The developed model is used to quantify the efficiency of attacking and defending for a given sequence of play. Performance analysis results demonstrate that this novel method of space-control efficiency quantification is objective and precise. The superior performance of the model is attributed to the utilization of deep generative modelling on image datasets and conditioning in the prediction with contextual factors. This study presents a novel approach to football analysis in evaluating team performance and providing tactical insights for the coach to make data-informed adjustments.

Multiagent reinforcement learning (MARL) is useful in many problems that require the cooperation and coordination of multiple agents. Learning optimal policies using reinforcement learning in a multiagent setting can be very difficult as the number of agents increases. Recent solutions such as value decomposition networks (VDNs), QMIX, QTRAN, and QPLEX adhere to the centralized training and decentralized execution (CTDE) scheme and perform factorization of the joint action-value functions. However, these methods still suffer from increased environmental complexity, and at times fail to converge in a stable manner. We propose a novel concept of residual Q-networks (RQNs) for MARL, which learns to transform the individual Q -value trajectories in a way that preserves the individual-global-max (IGM) criteria, but is more robust in factorizing action-value functions. The RQN acts as an auxiliary network that accelerates convergence and will become obsolete as the agents reach the training objectives. The performance of the proposed method is compared against several state-of-the-art techniques such as QPLEX, QMIX, QTRAN, and VDN, in a range of multiagent cooperative tasks. The results illustrate that the proposed method, in general, converges faster, with increased stability, and shows robust performance in a wider family of environments. The improvements in results are more prominent in environments with severe punishments for noncooperative behaviors and especially in the absence of complete state information during training time.

Increasingly, the task of detecting and recognizing the actions of a human has been delegated to some form of neural network processing camera or wearable sensor data. Due to the degree to which the camera can be affected by lighting and wearable sensors scantiness, neither one modality can capture the required data to perform the task confidently. That being the case, range sensors, like light detection and ranging (LiDAR), can complement the process to perceive the environment more robustly. Most recently, researchers have been exploring ways to apply convolutional neural networks to 3-D data. These methods typically rely on a single modality and cannot draw on information from complementing sensor streams to improve accuracy. This article proposes a framework to tackle human activity recognition by leveraging the benefits of sensor fusion and multimodal machine learning. Given both RGB and point cloud data, our method describes the activities being performed by subjects using regions with a convolutional neural network (R-CNN) and a 3-D modified Fisher vector network. Evaluated on a custom captured multimodal dataset demonstrates that the model outputs remarkably accurate human activity classification (90%). Furthermore, this framework can be used for sports analytics, understanding social behavior, surveillance, and perhaps most notably by autonomous vehicles (AVs) to data-driven decision-making policies in urban areas and indoor environments.

Increasingly complex automated driving functions, specifically those associated with free space detection (FSD), are delegated to convolutional neural networks (CNNs). If the dataset used to train the network lacks diversity, modality, or sufficient quantities, the driver policy that controls the vehicle may induce safety risks. Although most autonomous ground vehicles (AGVs) perform well in structured surroundings, the need for human intervention significantly rises when presented with unstructured niche environments. To this end, we developed an AGV for seamless indoor and outdoor navigation to collect realistic multimodal data streams. We demonstrate one application of the AGV when applied to a self-evolving FSD framework that leverages online active machine-learning (ML) paradigms and sensor data fusion. In essence, the self-evolving AGV queries image data against a reliable data stream, ultrasound, before fusing the sensor data to improve robustness. We compare the proposed framework to one of the most prominent free space segmentation methods, DeepLabV3+ [1]. DeepLabV3+ [1] is a state-of-the-art semantic segmentation model composed of a CNN and an autodecoder. In consonance with the results, the proposed framework outperforms DeepLabV3+ [1]. The performance of the proposed framework is attributed to its ability to self-learn free space. This combination of online and active ML removes the need for large datasets typically required by a CNN. Moreover, this technique provides case-specific free space classifications based on the information gathered from the scenario at hand.

Humans live among other humans, not in isolation. Therefore, the ability to learn and behave in multi-agent environments is essential for any autonomous system that intends to interact with people. Due to the presence of multiple simultaneous learners in a multi-agent learning environment, the Markov assumption used for single-agent environments is not tenable, necessitating the development of new Policy Learning algorithms. Recent Actor-Critic algorithms proposed for multi-agent environments, such as Multi-Agent Deep Deterministic Policy Gradients and Counterfactual Multi-Agent Policy Gradients, find a way to use the same mathematical framework as single agent environments by augmenting the Critic with extra information. However, this extra information can slow down the learning process and afflict the Critic with Curse of Dimensionality. To combat this, we propose a novel Deep Neural Network configuration called Deep Multi-Critic Network. This architecture works by taking a weighted sum over the outputs of multiple critic networks of varying complexity and size. The configuration was tested on data collected from a real-world multi-agent environment. The results illustrate that by using Deep Multi-Critic Network, less data is needed to reach the same level of performance as when not using the configuration. This suggests that as the configuration learns faster from less data, then the Critic may be able to learn Q-values faster, accelerating Actor training as well.

With the exponential growth of stereoscopic imaging in various applications, it has become very demanding to have a reliable quality assessment technique to measure the human perception of stereoscopic images. Quality assessment of stereoscopic visual content in the presence of artefacts caused by compression and transmission is a key component of end-to-end 3D media delivery systems. Despite a few recent attempts to develop stereoscopic image/video quality metrics, there is still a lack of a robust stereoscopic image quality metric. Towards addressing this issue, this paper proposes a full reference stereoscopic image quality metric, which mimics the human perception while viewing stereoscopic images. A signal processing model that is consistent with physiological literature is developed in the paper to simulate the behaviour of simple and complex cells of the primary visual cortex in the Human Visual System (HVS). The model is trained with two publicly available stereoscopic image databases to match the perceptual judgement of impaired stereoscopic images. The experimental results demonstrate a significant improvement in prediction performance as compared with several state-of-the-art stereoscopic image quality metrics.

This paper addresses the problem caused by motion vector coding dependencies on the error resilience performance of the emergent High Efficiency Video Coding (HEVC) standard. We propose a method based on the prediction dependency of motion vectors (MV) to select the most relevant ones for redundant coding with reduced overhead. The spatial dependencies are analysed in the encoder to prioritise the MVs that should be selected for redundancy, based on the number of subsequent dependent coding units. Then, a subset of prioritised MVs is transmitted as redundancy (referred to as side information in the paper), to reduce the use and propagation of mismatched MV predictions in case of transmission errors or data loss. The simulation results show that the proposed MV selection method can effectively identify the most relevant motion field, achieving improved error robustness with a reduced redundancy overhead. Exploiting only 30% of the generated MVs for redundancy, average quality gains of up to 1 dB are achieved compared to a uniform MV selection scheme, and up to 2 dB compared to the original HEVC standard with no redundant encoded information.

The quality assessment of impaired stereoscopic video is a key element in designing and deploying advanced immersive media distribution platforms. A widely accepted quality metric to measure impairments of stereoscopic video is, however, still to be developed. As a step toward finding a solution to this problem, this paper proposes a full reference stereoscopic video quality metric to measure the perceptual quality of compressed stereoscopic video. A comprehensive set of subjective experiments is performed with 14 different stereoscopic video sequences, which are encoded using both the H.264 and high efficiency video coding compliant video codecs, to develop a subjective test results database of 116 test stimuli. The subjective results are analyzed using statistical techniques to uncover different patterns of subjective scoring for symmetrically and asymmetrically encoded stereoscopic video. The subjective result database is subsequently used for training and validating a simple but effective stereoscopic video quality metric considering heuristics of binocular vision. The proposed metric performs significantly better than state-of-the-art stereoscopic image and video quality metrics in predicting the subjective scores. The proposed metric and the subjective result database will be made publicly available, and it is expected that the proposed metric and the subjective assessments will have important uses in advanced 3D media delivery systems.

With the increasing popularity of online video streaming and broadcasting applications, video transmission has become the most bandwidth consumed application in the Internet. Unless resource utilisation is further optimised, existing networks would not be able to accommodate the forecasted demand for high volume video traffic. Random Linear Network Coding (RLNC) is a promising approach to increase the network throughput and robustness in order to facilitate high volume traffic. However, the performance gain obtained using RLNC decreases when the network packet loss rate increases. In response, this paper proposes to incorporate RLNC with a combined interleaving and a coding technique to improve the effectiveness of generating redundant packet combinations over classical RLNC to enhance the robustness against packet losses. Simulation results illustrate that the proposed method improves the probability of recovering lost data over the classical RLNC and the video transmission simulation results exhibit up to 5dB increase in PSNR and up to 14% increase in VQM utilising the same network resources at the same network conditions.

Three-dimensional (3-D) displays provide a more realistic experience of entertainment by providing its viewers an added sensation of depth by artificially exploiting light rays to stimulate certain depth cues in the human visual system, especially binocular stereopsis. Due to its close relationship with human visual perception, mass market deployment of 3-D displays will be significantly dependant upon addressing the related perceptual factors such as visual comfort. In order to address the perceptual factors, it is very important to understand how humans experience depth on 3-D displays and how sensitive they are for different depth cues. In this paper, the sensitivity of humans for different depth cues is analyzed as applicable to 3-D viewing on stereoscopic displays. Mathematical models are derived to explain the just noticeable difference in depth (JNDD) for three different depth cues, namely binocular disparity, retinal blur, and relative size. Extensive subjective assessments are performed on a stereoscopic display with passive polarized glasses and on an auto-stereoscopic display to validate the mathematical models for JNDD. It is expected that the proposed models will have important use cases in 3-D display designing as well as 3-D content production.

Compression of Depth maps that are used in 3D video systems based on Depth Image Based Rendering (DIBR) poses a new challenge in video coding, since it is not a sequence of images for final viewing by end users rather an aid for rendering. Therefore, compressing depth maps using existing video coding techniques yields unacceptable distortions while rendering virtual views. In this paper we propose a novel mode selection method for offline compression of depth maps by selecting modes collaboratively considering an entire row of macroblocks together. For selecting these modes while encoding, we propose a novel distortion criteria that incorporates rendering distortions instead of distortion of depth map itself. A genetic algorithm based optimization technique is used for the mode selection. The simulation results suggest that the proposed technique can improve the PSNR up to 1.6dB in the rendered stereoscopic views in comparison to the block wise mode selection method based on Lagrange Optimization and the distortion of the depth map itself.

The ability to provide a realistic perception of depth is the core added functionality of modern 3D video display systems. At present, there is no standard method to assess the perception of depth in 3D video. Existence of such methods would immensely enhance the progression of 3D video research. This paper focuses on the depth perception assessment in color plus depth representation of 3D video. In this paper, we subjectively evaluate the depth perceived by the users on an auto stereoscopic display, and analyze its variation with the impairments introduced during the compression of the depth images. The variation of the subjective perception of depth is explained based on another evaluation that is carried out to identify the Just Noticeable Difference in Depth (JNDD) perceived by the subjects. The JNDD corresponds to the sensitivity of the observers to the changes in depth in a 3D video scene. Even though only the effects of compression artifacts are considered in this paper, the proposed assessment technique, based on the JNDD values can be used in any future depth perception assessment work.

Research on error resilience in multi-view coding is currently receiving considerable interest. While there is a multitude of literature concerning error recovery in 2D video, due to the statistical difference in motion compensation among temporal frames and disparity compensation among view points, such methods are inadequate to cater to the requirements of multiview video transmission. This paper addresses the above issue by transmission of redundant disparity vectors for error recovery purposes. The proposed system, which is implemented using the Joint Scalable Video Model (JSVM) codec and tested using a simulated Internet Protocol (IP) packet network environment, can be used along with a suitable error concealment scheme to provide robust multi-view video transmission. The experimental results suggest that the proposed algorithm experiences a slight degradation of quality in error free environments due to the inclusion of redundant data. However, it improves the reconstructed picture quality significantly in error prone environments, specifically for Packet Loss Rates (PLRs) greater than 7%.

Motion estimation is the most computationally intensive element in a block based video codec. In this paper, a novel Predictive Intensive Direction Searching (PIDS) algorithm is proposed to reduce the computational load of H.264/AVC video codec. Based on the direction of predicted motion vector, the search area is adaptively divided into one region of intensive-direction-searching and several regions of' coarse-direction-searching. Then the extended hexagon search is used to refine the final optimal motion vector. The experimental results indicate that the proposed algorithm achieves a reduction of 15% motion estimation time, while incurring only 0.5% increment on the bite rate compared with that of UMHexagonS algorithm, which is adopted by the H.264/AVC reference software.

The most distinguishing feature of 3D display systems, compared to their traditional 2D counterparts, is their ability to provide an additional perception of depth to its viewers. Thus, the mechanisms behind human depth perception play a significant role in 3D video systems. While there has been significant amounts of research carried out to understand human depth perception, in the areas of physiology and psychology, its applicability to 3D display systems is seldom spoken. Understanding the mechanisms of depth perception is of utmost importance to the development of 3D video technologies that are heavily based on exploitation of human perception. In this paper, it is explained with the aid of existing physiological and psychological models how humans perceive depth in 3D video displays. Based on these explanations a mathematical model is derived to explain the just noticeable difference in depth (JNDD) as perceived by a viewer, watching 3D video. The derived model is experimentally validated on an auto-stereoscopic display. This model is expected to be useful in both 3D content productions as well as in 3D content processing and compression.