An important source of information for ensuring public safety is control room video surveillance. A Decision-Support System (DSS) designed for the security task force is vital and necessary to take decisions rapidly using a sea of information. In case of mission-critical operation, Situational Awareness (SA) which consists of being aware of what is going on around you in real time plays a crucial role across a variety of industries and should be placed at the center of our system. In this paper, in order to satisfy the understanding and projection levels of SA in real time, we propose a method based on Reinforcement Learning (RL) using an Actor-Critic algorithm. This algorithm permits our DSS to first, keep SA knowledge up to date by performing online adaptive learning and second, develop a long-term vision and strategy by exploiting a non-myopic agent for real-time sequential decision-making. In contrast to other approaches, our results permit demonstrating the practical potential of our method in real-world scenarios by preventing abnormal situations in an ever-changing environment.

International audience

When several networks (e.g., Wi-Fi, UMTS, and LTE) cover the same region, the mobile terminals that are equipped with multiple network interfaces provide the possibility for mobile end-users to select their believed best network. This is known as the network selection problem, which is a decision making problem with multiple criteria (network conditions, service requirements, terminal characteristics, and user needs). Many network selection solutions using different mathematical theories have been proposed in the literature to allow the best connectivity for applications, users, and terminals. Unfortunately, most approaches for the network selection do not make effective selection decisions, since they are vulnerable to the uncertainty and imprecision related to network state information. In this paper, we investigate the belief functions theory to devise an efficient lightweight uncertainty- aware network interface selection scheme. We provide analytical studies and simulation experiments to demonstrate the efficiency of the proposed solution.

The user satisfaction measurement has gained high attention from Network Operators (NOs) and Service Providers (SPs) because their businesses are highly dependent on the user's satisfaction. Generally, the traditional strategies to measure the user's perception are based on Quality of Service (QoS), which is not sufficient to reflect the real user's perceived quality. Therefore, NOs and SPs start to develop new strategies based on the Quality of Experience (QoE) metric to analyze the relationship between the user's satisfaction and influence factors (QoE IFs). In this paper, a new method to build a predictive model to estimate user's satisfaction in terms of Mean Opinion Score (MOS) is proposed. The proposed method uses the dataset collected using the controlled testbed based on the YouTube video service. In the proposed model, the correlation matrix is used to develop a new heuristic method that used back-jumping technique to select the most beneficial factors to predict the optimal user's satisfaction.

In Heterogeneous Wireless Networks (HWNs), the mobile terminals are equipped with multiple access network interfaces (GSM, UMTS, LTE, WiFi, Bluetooth, etc.), to provide the possibility for mobile end-users to rank the networks and dynamically select the best one at anytime and anywhere, which is well known as Always Best Connected (ABC). In such environment, the major issue is network interface selection, which is a decision making problem with multiple alternatives (networks) and attributes (network characteristics, application requirements, terminal capacities, and user needs). In this context, many approaches have been proposed. Multi Attribute Decision Making (MADM) algorithms present a promising solution for multi-criteria decision making problems. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is one of MADM algorithms, which is widely adopted. TOPSIS ranks the available networks based on their scores, with the highest being the best. TOPSIS suffers from couple limitations. First is the ranking abnormality, e.g. if a low ranking network is disconnected then the order of higher ranking networks changes, which results in the selection of a less desirable network. Second is the selection strategy, where TOPSIS simply selects the network with highest score regardless of whether or not it satisfies the user and/or application needs. In this paper, we propose a new strategy based on utility function to remedy these shortcomings. The effectiveness of our strategy is evaluated through simulations. Obtained results show clearly that our strategy eliminates the rank reversal (ranking abnormality) phenomenon, and enhances the ranking quality by considering application and/or user needs.

In Heterogeneous Wireless Networks, mobile users use a terminal with multiple access interfaces for non-real-time or real-time applications (services). In such environment, the major issue is Always Best Connected (ABC), which means that the mobile nodes rank the network interfaces and select the best one at anytime and anywhere. To meet the ABC requirements, many network interface selection strategies have been proposed in the literature, using various technologies. This paper surveys existing approaches and discusses their advantages and limitations. The paper also highlights open issues in this area of research and proposes a new QoE-based approach for interface selection based on TOPSIS algorithm for e-Health use case. The effectiveness of our approach is evaluated through simulations. Obtained results show clearly that our approach ensures the best QoE for user, and eliminates a major inconvenient due to rank reversal (ranking abnormality).

Traditionally, companies host their own services, platforms and infrastructures on their own servers. This policy results in high costs in terms of material and human resources. It may also be inadequate to the real needs of the company. In this context, one solution is to use cloud computing to outsource their services. The latter is defined by making available to the customer high-performance servers and high bandwidth. The cloud is also defined by renting software and hardware infrastructure to customers according to their needs. Cloud computing is made possible by the improvement of computer networks infrastructures. Indeed, broadband connections have reduced latency and thus enabled the use of remote resources. The success of cloud computing has led to a significant increase in the providers number offering many and varied cloud services. While the access to these services is made possible through a simple subscription, no technique is currently available to select the cloud provider that best fits their needs. Selecting a provider is an optimization problem that has been studied in several areas. Given the large number of parameters and actors in the cloud, this problem is known as NP-complete one. In this work, we propose a new developed platform which plays the role of a broker between clients and cloud providers. Based on a set of benchmark tasks on provider services, it performs an adaptive cloud provider selection in accordance with the client needs. The experimental results show that the proposed approach gives benefits to subscribers in terms of QoE.

The tremendous growth in video services, specially in the context of mobile usage, creates new challenges for network service providers: How to enhance the user's Quality of Experience (QoE) in dynamic wireless networks (UMTS, HSPA, LTE/LTE-A). The network operators use different methods to predict the user's QoE. Generally to predict the user's QoE, methods are based on collecting subjective QoE scores given by users. Basically, these approaches need a large dataset to predict a good perceived quality of the service. In this paper, we setup an experimental test based on crowdsourcing approach and we build a large dataset in order to predict the user's QoE in mobile environment in term of Mean Opinion Score (MOS). The main objective of this study is to measure the individual/global impact of QoE Influence Factors (QoE IFs) in a real environment. Based on the collective dataset, we perform 5 testing scenarios to compare 2 estimation methods (SVM and ANFIS) to study the impact of the number of the considered parameters on the estimation. It became clear that using more parameters without any weighing mechanisms can produce bad results.

As current server capacity and network bandwidth become increasingly overloaded by the rapid growth of high quality emerging multimedia services such as mobile online gaming, social networking or IPTV, a critical factor of success of these multimedia services becomes the end-user perception of quality while them using the service. As a result, user-centered approaches that consider quality of experience (QoE) constitute the current design trend for network systems of content providers and network operators. A content distribution network (CDN) that replicates the content from original servers to the replicated servers close to end users is actually an effective solution to improve network quality. We propose a QoE-based server selection algorithm in the context of a CDN architecture. Using realistic characteristics of the server selection process, we formalize our selection model as a sequential decision problem solved by the multi-armed bandit (MAB) paradigm. By using realistic experiments, we demonstrate that our approach yields significant improvements in term of user perception compared to traditional methods (such as Fastest, Closest and Round Robin).

A main objective of an Information Centric Network (ICN) is to improve the network by placing the knowledge in center of the network design. This vision of the network needs an efficient distributed and decentralized knowledge plane. So, an important amount of knowledge should be disseminated over the supervised network, which remains an open problem. Indeed, the dissemination infrastructure must be able to ensure the transport of all information types including knowledge information, throughout the network, and guarantee its freshness. Another crucial aspect of the problem is related to the network robustness with respect to network failures. In this paper, we propose a new model of knowledge dissemination based on super peers architecture. We formalize the super peer selection problem as a K-medoids clustering task. Furthermore, to handle the dynamicity of the network and especially the changes of the end-user network topology, we improved the selection mechanism by adding an adaptive mechanism based on the Page-Hinkley statistical test. Experimental results show that the proposed approach significantly improves performances compared to other current approaches.

This article proposes a state-dependent routing algorithm based on a global optimization cost function whose parameters are learned from the real-time state of the network with no a priori model. The proposed approach samples, estimates, and builds the model of pertinent and important aspects of the network environment such as type of traffic, QoS policies, resources, etc. It is based on the trial/error paradigm combined with swarm-adaptive approaches. The global system uses a model that combines both a stochastic planned prenavigation for the exploration phase with a deterministic approach for the backward phase. We conducted a performance analysis of the proposed algorithm using OPNET based on several topologies such as the Nippon telephone and telegraph network. The simulation results obtained demonstrate substantial performance improvements over traditional routing approaches as well as the benefits of learning approaches for networks with dynamically changing traffic.

Autonomic computing is a new paradigm inspired by the biological world. It aims at making a network independent of any human monitoring. To reach such autonomy, knowledge should be disseminated over the network, which remains an open problem. Our solution consists in proposing a new model of knowledge dissemination based on three key ideas: a hierarchical architecture, a specific-service overlay network (SSON) and a multi-criteria selection of a subset of nodes responsible for knowledge management. The simulation results show that the proposed approach significantly improves performances compared to other approaches.

Autonomic Networks represent a concept inspired by the biological world that aims at making a network independent of any human monitoring. To reach such autonomy, knowledge should be disseminated over the network, which remains an open problem. In fact, disseminating the knowledge over all nodes leads to a big overhead. That is why we need to select a subset of nodes in charge of knowledge management. A single criterion-based selection mechanism has been proposed in a previous work but such a mechanism seems to be very simplistic. In this paper, we present a new multi-criteria selection mechanism based on Pareto. The simulation results show that the proposed approach significantly improves performances compared to single criterion-based selection mechanism.

For years, wireless network systems have been trying to satisfy end-users and support high quality multimedia applications such as Mobile TV, VoIP, etc. Combining wireless networks with multimedia content distribution needs efficient routing protocols. We develop in this paper a new routing protocol, namely DOQAR (Dynamic Optimized QoE Adaptive Routing), to improve the user perception and optimize the usage of network resources. In our end-to-end model, smartphone users connect to content servers in a wired network across a wireless access network. In order to evaluate the QoE, we use a Multi-Layer Perception-based (MLP) method. Experimental results show a significant performance against other traditional routing protocols.

Computing constrained shortest paths is fundamental to some important network functions such as QoS routing and traffic engineering. This paper introduces a polynomial time approximation Quality of Service (QoS) routing algorithm and constructs dynamic state-dependent routing policies. The proposed algorithm uses a bio-inspired approach based on the trial/error paradigm combined with swarm adaptive approaches to optimize three QoS different criteria: static cumulative cost path, dynamic residual bandwidth, and end-to-end delay. The approach uses a model that combines both a stochastic planned pre-navigation for the exploration phase and a deterministic approach for the backward phase. In this paper, we adopt the unified framework of online learning to consider a global cost function. Numerical results obtained with OPNET simulator for different levels of traffic's load show that the new added module improves clearly performances of our earlier KOQRA.

In the last decade, due to emerging real-time and multimedia applications, there has been much interest for developing mechanisms to take into account the quality of service required by these applications. One of these mechanisms consists to integrate simultaneously many criteria of quality of service (QoS) in routing decision. Efficient routing of information packets in dynamically changing communication network requires that as the load levels, traffic patterns and topology of the network change, the routing policy also adapts. We propose in this paper an approach used an adaptive algorithm for packet routing using reinforcement learning called AV-BW Delay Q- Routing. This approach is based on earlier developed K-Optimal path Q-Routing Algorithm (KOQRA) which optimizes simultaneously two additive QoS criteria: cumulative cost path and end-to-end delay. The approach developed here adds a third criterion to KOQRA regarding residual bandwidth. This proposed technique uses an inductive approach based on trial/error paradigm combined with swarm adaptive approaches. The whole algorithm uses a model combining both a stochastic planned pre-navigation for the exploration phase and a deterministic approach for the backward phase. Numerical results obtained with OPNET simulator for different levels of traffic's load show that AV-BW Delay Q-Routing improves clearly performances of our earlier KOQRA.

High speed modern communication networks are required to integrate and support multimedia application which requires differentiated Quality-of-Service (QoS) guarantees. Routing mechanism is a key to success of future communication networking. However, it is often complicated by the notion of guaranteed QOS, which can either be related to time, cost, packet loss or bandwidth requirements. Communication network requires that as the load levels, traffic patterns and topology of the network change, the routing policy also adapts. In this paper, we present a QoS based routing to construct dynamic state-dependent routing policies. The proposed algorithm used a reinforcement learning paradigm to optimize two QoS criteria: cumulative cost path based on hop count and end-to-end delay. Multiple paths are searched in parallel to find the N best qualified ones. In order to improve the overall network performance, a load balancing policy is defined and depends on a dynamical traffic path probability distribution function. The performance of our algorithm for different levels of traffic’s load is compared experimentally with standard optimal path routing algorithms for the same problem.

Routing is a relevant issue for maintaining good performance and successfully operating in a network. We focused in this paper on neuro-dynamic programming to construct dynamic state-dependent routing policies which offer several advantages, including a stochastic modelization of the environment, learning and evaluation are assumed to happen continually, multi-paths routing and minimizing state overhead. This paper describe an adaptive algorithm for high speed irregular packet routing using reinforcement learning called N Q-routing Optimal Shortest Paths (NQOSP). In contrast with other algorithms that are also based on Reinforcement Learning methods, NQOSP is based on a multi-paths routing technique combined with the Q-Routing algorithm. In this case, the exploration space is limited to N-Optimal non loop paths in term of hops number (number of routers in a path) leading to a substantial reduction of convergence time. We propose here a framework to describe our algorithm and focus to improve scalability, robustness of our approach. We also integrate a module to compute dynamically a probability in order to better distribute traffic on best paths. The performance of NQOSP is evaluated experimentally with OPNET simulator for different levels of traffic’s load and compared to standard shortest path and Q-routing algorithms on large interconnected network. Our approach prove superior to a classical algorithms and is able to route efficiently in large networks even when critical aspects, such as the link broken network, are allowed to vary dynamically.