Reinforcement learning is widely applied in robotic arm manipulation tasks. However, most of these tasks focus on single and simple end effector. When facing heavy robotic arm hoisting tasks, which are usually manipulated by robotic arms with multi-end-effectors and more degrees of freedom, the single-agent-based reinforcement learning method performs relatively ineffective. In this paper, we propose a multi-agent reinforcement learning approach for hoisting tasks manipulated by robotic arm with multi-end-effectors. The method decomposes the robotic arm into global and local agents based on the degrees of freedom, with one agent controlling global and rough movement, and the other controlling local and fine movement. In this way, the multi-end-effectors’ spatial trajectory can be accurately manipulated. Moreover, in the training process, a four levels curriculum learning strategy is introduced, in which different reward functions are designed respectively, to make the training efficiency and effectiveness. We develop a Unity engine environment-based simulation and perform several comparison experiments. The results demonstrate that the proposed approach outperforms conventional single-agent-based methods.

This research tackles challenges in traditional fuzzing, such as limited coverage, instability, and inefficiency in bug discovery. We propose two novel models and their combination to enhance mutation processes and improve its reliability through unbiased randomisation, building on cryptographic techniques from our prior work. To our knowledge, we are the first to apply this approach to AFL++ , extending Feistel-inspired mutation and high-performance randomisation to generate high-quality test cases, with potential to attract attention in the fuzzing community. • Integrate and assess Feistel-inspired mutations’ impact on AFL++ performance, focusing on code coverage and stability. • Integrate the Permuted Congruential Generator ( PCG ) into AFL++ and evaluate its performance compared to traditional random number generators ( RNGs ). • Evaluate a hybrid model combining Feistel and PCG randomness for better stability and coverage. We enhance AFL++ with algorithmic improvements and RNGs modifications. Our models include: • CAFL++ ( Cryptographic - AFL++ ): Integrates Feistel-inspired transformations for improved coverage. • PCGAFL++ : Refines the AFL ’s RNG with PCG to reduce bias. • CPCGAFL++ : Combines Feistel-inspired swaps and PCG -based RNG for a robust fuzzing approach. Performance was analysed using metrics like Code Coverage and the Vargha-Delaney_A12 statistic across 20 Fuzzbench targets, and bug discovery on three targets. Our models showed significant improvements over AFL++ . CAFL++ outperformed AFL++ in 75% of test targets, offering better code coverage and stability. PCGAFL++ surpassed AFL++ in 60% of targets by enhancing randomness, resulting in more efficient fuzzing. CPCGAFL++ demonstrated improved stability and enhanced bug discovery performance, while achieving code coverage comparable to AFL++ . These results highlight the key improvements introduced by our two models for fuzz testing. Our models advance fuzzing by improving code coverage and stability. Integrating Feistel-inspired swaps and PCG -based RNG overcomes traditional fuzzing limitations, offering a more efficient and reliable method. These models represent a step forward in fuzzing techniques, influencing both academic research and industrial practices.

Modern software and networks underpin our digital society, yet the rapid growth of vulnerabilities that are uncovered within these threaten our cyber security posture. Addressing these issues at scale requires automated proactive approaches that can identify and mitigate these vulnerabilities in a suitable time frame. Fuzzing techniques have emerged as crucial methods to preemptively tackle these risks. However, traditional fuzzing methods encounter various challenges, such as a lack of strategy for deep bug identification, time-intensive bug analysis, quality of inputs, seed scheduling and others. To overcome these challenges, diverse Machine Learning (ML) models and optimisation techniques have been employed, including advanced feature engineering, optimised seed selection, refined predictive/fitness models, and Gradient-based optimisation. Furthermore, the use of ML architectures such as Long Short-Term Memory (LSTM), Generative Adversarial Network (GAN), Sequence-to-Sequence (Seq2Seq), and Generative Randomised Unit (GRU), have demonstrated greater effectiveness within ML-based fuzzing. In this paper, we delve into this paradigm shift, aiming to address fundamental challenges across different ML categories. We survey popular ML categories such as Traditional Machine Learning (TML), Deep Learning (DL), Reinforcement Learning (RL), and Deep Reinforcement Learning (DRL), to investigate their potential for enhancing traditional fuzzing approaches. We explore the respective advantages in each category of ML-based fuzzing, while also analysing the challenges unique to each category. Our work provides a comprehensive survey across the fuzzing domain and how machine learning techniques have been utilised, that we believe will be of use to future researchers in this domain.

Online social network platforms have a problem with misinformation. One popular way of addressing this problem is via the use of machine learning based automated misinformation detection systems to classify if a post is misinformation. Instead of post hoc detection, we propose to predict if a user will engage with misinformation in advance and design an effective graph neural network classifier based on ego-graphs for this task. However, social networks are highly dynamic, reflecting continual changes in user behaviour, as well as the content being posted. This is problematic for machine learning models which are typically trained on a static training dataset, and can thus become outdated when the social network changes. Inspired by the success of continual learning on such problems, we propose an ego-graphs replay strategy in continual learning (EgoCL) using graph neural networks to effectively address this issue. We have evaluated the performance of our method on user engagement with misinformation on two Twitter datasets across nineteen misinformation and conspiracy topics. Our experimental results show that our approach EgoCL has better performance in terms of predictive accuracy and computational resources than the state of the art.

Data augmentation has been widely used in machine learning for natural language processing and computer vision tasks to improve model performance. However, little research has studied data augmentation on graph neural networks, particularly using augmentation at both train- and test-time. Inspired by the success of augmentation in other domains, we have designed a method for social influence prediction using graph neural networks with train- and test-time augmentation, which can effectively generate multiple augmented graphs for social networks by utilising a variational graph autoencoder in both scenarios. We have evaluated the performance of our method on predicting user influence on multiple social network datasets. Our experimental results show that our end-to-end approach, which jointly trains a graph autoencoder and social influence behaviour classification network, can outperform state-of-the-art approaches, demonstrating the effectiveness of train-and test-time augmentation on graph neural networks for social influence prediction. We observe that this is particularly effective on smaller graphs.

There is no agreed definition of social capital in the literature. However, one interpretation is that it refers to those resources embedded in an individual's social network offering benefits to that individual in relation to achieving goals and facilitating actions. This can be viewed as a resource-based interpretation of social capital aimed at the level of individuals. In this paper, we propose a family of social capital measures in line with this interpretation. Our measures are designed for a model of social networks based on weighted and attributed graphs, and cover four dimensions of social capital: (i) access to resources, (ii) access to superiors, (iii) homogeneity of ties, and (iv) heterogeneity of ties. We demonstrate the real-world application of our measures by exploring an illustrative use case in the form of a workplace social network.

Cloud and cloud shadow detection in remote sensing imagery is important for its wide range of applications. Traditionally, the detection is usually based on the manually designed thresholds from multiband, which is complicated and of multistage. To simplify the process of cloud and cloud shadow detection and improve the performance, we propose a multilevel feature fused segmentation network (MFFSNet), which can be trained end-to-end without any hand-tuned parameters. Specifically, a fully convolutional network is proposed for cloud and cloud shadow features learning. Then, we utilize a novel pyramid pooling module to extract contextual relation between cloud and shadow. Furthermore, a special multilevel feature fused structure is designed to combine semantic information with spatial information from different levels, so that we can better handle the multiscale objects and produce detailed segmentation boundaries. Experiments show that the MFFSNet outperforms the state-of-the-art methods and achieves high accuracies of 98.69% and 98.92% for cloud and cloud shadow detection.

This paper presents an event modelling and reasoning framework where event-observations obtained from heterogeneous sources may be uncertain or incomplete, while sensors may be unreliable or in conflict. To address these issues we apply Dempster-Shafer (DS) theory to correctly model the event-observations so that they can be combined in a consistent way. Unfortunately, existing frameworks do not specify which event-observations should be selected to combine. Our framework provides a rule-based approach to ensure combination occurs on event-observations from multiple sources corresponding to the same event of an individual subject. In addition, our framework provides an inference rule set to infer higher level inferred events by reasoning over the uncertain event-observations as epistemic states using a formal language. Finally, we illustrate the usefulness of the framework using a sensor-based surveillance scenario.

Factored MDPs provide an efficient way to reduce the complexity of large, real-world domains by exploiting structure within the state space. This avoids the need for the state space to be fully enumerated, which is impractical in large domains. However, defining a reward function for state transitions is difficult in a factored MDP since transitions are not known prior to execution. In this paper, we provide a novel method for deriving rewards from information within the states in order to determine intermediate rewards for state transitions. We do this by treating some specific state variables as resources, allowing costs and rewards to be inferred from changes to the resources and ensuring the agent is resource-aware while also being goal-oriented. To facilitate this, we propose a novel variant of Dynamic Bayesian Networks specifically for modelling action transitions and capable of dealing with relative changes to realvalued state variables (such as resources) in a compact fashion. We also propose a number of reward functions which model resource types commonly found in real-world situations. We go on to show that our proposed framework offers an improvement over existing techniques involving reward functions for factored MDPs as it improves both the efficiency and decision quality of online planners when operating on these models.

The Belief-Desire-Intention (BDI) architecture is a practical approach for modelling large-scale intelligent systems. In the BDI setting, a complex system is represented as a network of interacting agents - or components - each one modelled based on its beliefs, desires and intentions. However, current BDI implementations are not well-suited for modelling more realistic intelligent systems which operate in environments pervaded by different types of uncertainty. Furthermore, existing approaches for dealing with uncertainty typically do not offer syntactical or tractable ways of reasoning about uncertainty. This complicates their integration with BDI implementations, which heavily rely on fast and reactive decisions. In this paper, we advance the state-of-the-art w.r.t. handling different types of uncertainty in BDI agents. The contributions of this paper are, first, a new way of modelling the beliefs of an agent as a set of epistemic states. Each epistemic state can use a distinct underlying uncertainty theory and revision strategy, and commensurability between epistemic states is achieved through a stratification approach. Second, we present a novel syntactic approach to revising beliefs given unreliable input. We prove that this syntactic approach agrees with the semantic definition, and we identify expressive fragments that are particularly useful for resource-bounded agents. Third, we introduce full operational semantics that extend CAN, a popular semantics for BDI, to establish how reasoning about uncertainty can be tightly integrated into the BDI framework. Fourth, we provide comprehensive experimental results to highlight the usefulness and feasibility of our approach, and explain how the generic epistemic state can be instantiated into various representations.

There has been much interest in the belief–desire–intention (BDI) agent-based model for developing scalable intelligent systems, e.g. using the AgentSpeak framework. However, reasoning from sensor information in these large-scale systems remains a significant challenge. For example, agents may be faced with information from heterogeneous sources which is uncertain and incomplete, while the sources themselves may be unreliable or conflicting. In order to derive meaningful conclusions, it is important that such information be correctly modelled and combined. In this paper, we choose to model uncertain sensor information in Dempster–Shafer (DS) theory. Unfortunately, as in other uncertainty theories, simple combination strategies in DS theory are often too restrictive (losing valuable information) or too permissive (resulting in ignorance). For this reason, we investigate how a context-dependent strategy originally defined for possibility theory can be adapted to DS theory. In particular, we use the notion of largely partially maximal consistent subsets (LPMCSes) to characterise the context for when to use Dempster’s original rule of combination and for when to resort to an alternative. To guide this process, we identify existing measures of similarity and conflict for finding LPMCSes along with quality of information heuristics to ensure that LPMCSes are formed around high-quality information. We then propose an intelligent sensor model for integrating this information into the AgentSpeak framework which is responsible for applying evidence propagation to construct compatible information, for performing context-dependent combination and for deriving beliefs for revising an agent’s belief base. Finally, we present a power grid scenario inspired by a real-world case study to demonstrate our work.

The ability of an autonomous agent to select rational actions is vital in enabling it to achieve its goals. To do so effectively in a high-stakes setting, the agent must be capable of considering the risk and potential reward of both immediate and future actions. In this paper we provide a novel method for calculating risk alongside utility in online planning algorithms. We integrate such a risk-aware planner with a BDI agent, allowing us to build agents that can set their risk aversion levels dynamically based on their changing beliefs about the environment. To guide the design of a risk-aware agent we propose a number of principles which such an agent should adhere to and show how our proposed framework satisfies these principles. Finally, we evaluate our approach and demonstrate that a dynamically risk-averse agent is capable of achieving a higher success rate than an agent that ignores risk, while obtaining a higher utility than an agent with a static risk attitude.

Planning is an essential process in teams of multiple agents pursuing a common goal. When the effects of actions undertaken by agents are uncertain, evaluating the potential risk of such actions alongside their utility might lead to more rational decisions upon planning. This challenge has been recently tackled for single agent settings, yet domains with multiple agents that present diverse viewpoints towards risk still necessitate comprehensive decision making mechanisms that balance the utility and risk of actions. In this work, we propose a novel collaborative multi-agent planning framework that integrates (i) a team-level online planner under uncertainty that extends the classical UCT approximate algorithm, and (ii) a preference modeling and multicriteria group decision making approach that allows agents to find accepted and rational solutions for planning problems, predicated on the attitude each agent adopts towards risk. When utilised in risk-pervaded scenarios, the proposed framework can reduce the cost of reaching the common goal sought and increase effectiveness, before making collective decisions by appropriately balancing risk and utility of actions.

InSAR interferogram quality assessment is a key step for the using of interferogram map. Traditionally, the interferogram is qualitatively assessed visually and quantitatively assessed by the number of residues. However, the important structure information is hardly quantifiable. This paper presents a novel index to evaluate the quality of InSAR interferogram based on connected area. After discomposing the fringes into independent connected areas, we analyze the statistical ratio of an area to its margin. Then we use the ratio as an index to quantitatively evaluate the interferogram. In the end, the presented index is used for the filtered interferogram of popular filters, and the results fit the visual judging.

Two approximations, center-beam approximation and reference digital elevation model (DEM) approximation, are used in synthetic aperture radar (SAR) motion compensation procedures. They usually introduce residual motion compensation errors for airborne single-antenna SAR imaging and SAR interferometry. In this paper, we investigate the effects of residual uncompensated motion errors, which are caused by the above two approximations, on the performance of airborne along-track interferometric SAR (ATI-SAR). The residual uncompensated errors caused by center-beam approximation in the absence and in the presence of elevation errors are derived, respectively. Airborne simulation parameters are used to verify the correctness of the analysis and to show the impacts of residual uncompensated errors on the interferometric phase errors for ATI-SAR. It is shown that the interferometric phase errors caused by the center-beam approximation with an accurate DEM could be neglected, while the interferometric phase errors caused by the center-beam approximation with an inaccurate DEM cannot be neglected when the elevation errors exceed a threshold. This research provides theoretical bases for the error source analysis and signal processing of airborne ATI-SAR.

Recommending users for a new social network user to follow is a topic of interest at present. The existing approaches rely on using various types of information about the new user to determine recommended users who have similar interests to the new user. However, this presents a problem when a new user joins a social network, who is yet to have any interaction on the social network. In this paper we present a particular type of conversational recommendation approach, critiquing-based recommendation, to solve the cold start problem. We present a critiquing-based recommendation system, called CSFinder, to recommend users for a new user to follow. A traditional critiquing-based recommendation system allows a user to critique a feature of a recommended item at a time and gradually leads the user to the target recommendation. However this may require a lengthy recommendation session. CSFinder aims to reduce the session length by taking a case-based reasoning approach. It selects relevant recommendation sessions of past users that match the recommendation session of the current user to short-cut the current recommendation session. It selects relevant recommendation sessions from a case base that contains the successful recommendation sessions of past users. A past recommendation session can be selected if it contains recommended items and critiques that sufficiently overlap with the ones in the current session. Our experimental results show that CSFinder has significantly shorter sessions than the ones of an Incremental Critiquing system, which is a baseline critiquing-based recommendation system.

Revising its beliefs when receiving new information is an important ability of any intelligent system. However, in realistic settings the new input is not always certain. A compelling way of dealing with uncertain input in an agent-based setting is to treat it as unreliable input, which may strengthen or weaken the beliefs of the agent. Recent work focused on the postulates associated with this form of belief change and on finding semantical operators that satisfy these postulates. In this paper we propose a new syntactic approach for this form of belief change and show that it agrees with the semantical definition. This makes it feasible to develop complex agent systems capable of efficiently dealing with unreliable input in a semantically meaningful way. Additionally, we show that imposing restrictions on the input and the beliefs that are entailed allows us to devise a tractable approach suitable for resource-bounded agents or agents where reactive ness is of paramount importance.

When an agent wants to fulfill its desires about the world, the agent usually has multiple plans to choose from and these plans have different pre-conditions and additional effects in addition to achieving its goals. Therefore, for further reasoning and interaction with the world, a plan selection strategy (usually based on plan cost estimation) is mandatory for an autonomous agent. This demand becomes even more critical when uncertainty on the observation of the world is taken into account, since in this case, we consider not only the costs of different plans, but also their chances of success estimated according to the agent's beliefs. In addition, when multiple goals are considered together, different plans achieving the goals can be conflicting on their preconditions (contexts) or the required resources. Hence a plan selection strategy should be able to choose a subset of plans that fulfills the maximum number of goals while maintaining context consistency and resource-tolerance among the chosen plans. To address the above two issues, in this paper we first propose several principles that a plan selection strategy should satisfy, and then we present selection strategies that stem from the principles, depending on whether a plan cost is taken into account. In addition, we also show that our selection strategy can partially recover intention revision.

This paper presents a multi-agent system approach to address the difficulties encountered in traditional SCADA systems deployed in critical environments such as electrical power generation, transmission and distribution.The approach models uncertainty and combines multiple sources of uncertain information to deliver robust plan selection.We examine the approach in the context of a simplified power supply/demand scenario using a residential grid connected solar system and consider the challenges of modelling and reasoning with uncertain sensor information in this environment.We discuss examples of plans and actions required for sensing, establish and discuss the effect of uncertainty on such systems and investigate different uncertainty theories and how they can fuse uncertain information from multiple sources for effective decision making in such a complex system.

Web databases are now pervasive. Such a database can be accessed via its query interface (usually HTML query form) only. Extracting Web query interfaces is a critical step in data integration across multiple Web databases, which creates a formal representation of a query form by extracting a set of query conditions in it. This paper presents a novel approach to extracting Web query interfaces. In this approach, a generic set of query condition rules are created to define query conditions that are semantically equivalent to SQL search conditions. Query condition rules represent the semantic roles that labels and form elements play in query conditions, and how they are hierarchically grouped into constructs of query conditions. To group labels and form elements in a query form, we explore both their structural proximity in the hierarchy of structures in the query form, which is captured by a tree of nested tags in the HTML codes of the form, and their semantic similarity, which is captured by various short texts used in labels, form elements and their properties. We have implemented the proposed approach and our experimental results show that the approach is highly effective.

Many algorithms for query rewriting using views have been proposed. Most of them are used for rewriting conjunctive queries with conjunctive views only. Only one inverse rule-based rewriting algorithm has been presented to deal with query rewriting with disjunctive views, in which a set of disjunctive inverse rules are created and used to generate the query rewritings. However, there has no been bucket-based algorithm for this issue. In this paper we apply the ideas of the previous bucket-based algorithms to deal with query rewriting using views in the presence of disjunctions in view definitions. We create a set of buckets over either a conjunctive view or a disjunctive view. Specially, when a bucket is over a disjunctive view, we present a method to remove the tuples that are in the view but not required by a given query. The rewritings obtained by our algorithm are contained in the original query.

User traversals on hyperlinks between Web pages can reveal semantic relationships between these pages. We use user traversals on hyperlinks as weights to measure semantic relationships between Web pages. On the basis of these weights, we propose a novel method to put Web pages on a Web site onto different conceptual levels in a link hierarchy. We develop a clustering algorithm called PageCluster, which clusters conceptually-related pages on each conceptual level of the link hierarchy based on their in-link and out-link similarities. Clusters are then used to construct a conceptual link hierarchy, which is visualized in a prototype called Online Navigation Explorer (ONE) for adaptive Web site navigation. Our experiments show that our method can put Web pages onto conceptual levels of a link hierarchy more accurately than both the breadth-first search method and the shortest-weighted-path method, and PageCluster can cluster conceptually-related pages more accurately than the bibliographic analysis method. Our user study also shows that the conceptual link hierarchy visualized in ONE can help users find information more effectively and efficiently as the task of finding information becomes less specific and involves more Web pages on multiple conceptual levels.

In the environment of data integration over the Internet, three major factors affect the cost of a query: network congestion situation, server contention states (workload), and data/query complexity. We concentrate on system contention states. For a remote data source, we first determine the total number of contention states of the system through applying clustering techniques to the costs of sample queries. We then develop a set of cost formulae for each of the contention states using a multiple regression process. Finally, we estimate the system's current contention state when a query is issued and using either a time slides method or a statistical method depending on the information we have about the system. Our method can accurately predict the system contention state so that the effect of the contention states on the cost of queries can be estimated precisely.

Use of the Dempster-Shafer (D-S) theory of evidence to deal with uncertainty in knowledge-based systems has been widely addressed. Several AI implementations have been undertaken based on the D-S theory of evidence or the extended theory. But the representation of uncertain relationships between evidence and hypothesis groups (heuristic knowledge) is still a major problem. This paper presents an approach to representing such knowledge, in which Yen’s probabilistic multi-set mappings have been extended to evidential mappings, and Shafer’s partition technique is used to get the mass function in a complex evidence space. Then, a new graphic method for describing the knowledge is introduced which is an extension of the graphic model by Lowrance et al. Finally, an extended framework for evidential reasoning systems is specified.