As intelligent systems advance rapidly, human-robot collaboration is becoming increasingly important. Ensuring that the intelligent agent's behaviors match human intentions and value preferences is crucial for effective collaboration, which is termed the value alignment problem. Within the Reinforcement Learning (RL) paradigm, value alignment typically relies on pre-designed reward functions, and Cooperative Inverse Reinforcement Learning (CIRL) is often used to model value alignment as a human-robot game. However, existing works often assume that human is perfectly rational, and can fully obtain robot’s belief on human’s preference. To address this limitation, we propose a Particle Filter-based Hierarchical Dynamic Programming algorithm (PFHDP). By modeling the robot's belief state, this algorithm ensures the correct updates of human's estimate of the robot's belief. This allows human to adopt more targeted pedagogical behaviors to guide the robot based on her understanding of the robot's current belief, achieving belief alignment between human and robot and thereby promoting value alignment more effectively. Furthermore, we run experiments to evaluate the proposed method in two cooperative scenarios against some typical benchmark approaches. The experimental results show that our method can strengthen the alignment of belief states between human and robot, leading to enhanced value alignment.

With the widespread adoption of mobile smart devices and the Internet, a new paradigm known as Mobile Crowdsourcing (MCS) has emerged, which can provide efficient execution of large-scale tasks. However, it faces challenges due to an insufficient number of crowdsourcing participants. Existing works primarily focus on increasing participants’ willingness to engage, without addressing the issue of a limited user pool. In this paper, we incentivize users to diffuse the task within their social networks through financial rewards in the task duration, thereby recruiting additional social neighbors, expanding the user pool, and maximizing task completion. Specifically, we propose a Dynamic Reward Update and User Selection under the Diffusion (DRUUSD) algorithm for social network diffusion scenarios, given a budget constraint. The algorithm dynamically updates the task diffusion reward and task execution reward based on the current recruitment, while selecting users with the highest utility whose activated social neighbors can contribute more to task completion to diffuse tasks. Through multiple time steps of user selection and reward updates, the algorithm aims to recruit more participants and maximize task completion. We run extensive experiments to evaluate the proposed algorithm against four benchmark methods. The experimental results show that under the same total budget constraint, our algorithm can recruit more mobile users and achieve higher task completion compared to other benchmark algorithms.

With the increasing prominence of smart mobile devices, mobile crowdsourcing (MCS) has become an innovative distributed computing paradigm. However, it faces the challenge of users being less willing to participate. Social networks can help diffuse tasks, thereby recruiting more potential users to participate in crowdsourcing tasks. Therefore, we exploit social networks to recruit users by adopting the independent cascade model for diffusing crowdsourcing tasks to increase the number of participants. Considering the time-sensitive crowdsourcing tasks and the delays in the diffusion process, we model the interaction between registered users and the platform as a reverse auction model. Furthermore, the participating users may strategically provide untruthful information to make more profits. Therefore, we design a reverse auction-based incentive mechanism to incentivize users to diffuse tasks while preventing strategic behavior about reporting diffusion costs. We prove that the proposed mechanism can satisfy the desirable properties of incentive compatibility, individual rationality, and computational efficiency. We evaluate the proposed mechanism against four benchmark strategies on real and synthetic datasets. The experimental results show that our mechanism can recruit more mobile users and achieve higher task completion.

Mobile Edge Computing (MEC) is a technology that shows great promise in enhancing the computational power of smart devices (SDs) in the Internet of Things (IoT). However, the fixed location and limited coverage of MEC servers constrain their performance. To overcome this issue, this paper explores a multiple unmanned aerial vehicle (UAV) assisted MEC system. The proposed system considers a scenario where multiple UAVs work together to provide computing services while dynamically adjusting their frequency based on the task size, under the constraint of limited energy. This paper aims to maximize computation bits, SDs’ fairness, and UAVs’ load balancing in multi-UAV assisted MEC system by jointly optimizing the trajectory and frequency. To address this challenge, we model it as a Partially Observable Markov Decision Process and propose a joint optimization strategy based on multi-agent deep reinforcement learning. The effectiveness of the proposed strategy is evaluated on both synthetic and realistic datasets. The results demonstrate that our strategy outperforms other benchmark strategies.

Task offloading in cognitive radio-enabled Internet of Vehicles (CIoV) networks faces critical challenges due to the coupling of vehicular mobility, spectrum dynamics induced by primary users (PUs), and delay-sensitive service constraints. Existing approaches often decouple task offloading, spectrum access, and resource allocation, resulting in poor adaptability under dynamic and dense traffic conditions. To address these limitations, this paper proposes a Robust Parallel Benders Decomposition (RPBD) framework for mobility-aware task offloading in CIoV networks with bidirectional traffic and stochastic spectrum availability. The proposed framework introduces three key innovations: (1) a parallel decomposition strategy that separates discrete offloading decisions from continuous resource allocation, enabling scalable optimization; (2) a staleness-controlled cut generation mechanism that ensures solution quality under asynchronous updates; and (3) an adaptive parameter adjustment strategy that dynamically accommodates vehicular mobility and PU-induced spectrum variability. The formulation incorporates bidirectional mobility through service time windows and models spectrum availability using Markov-modulated channel states. Theoretical analysis guarantees convergence through convex subproblem structures and martingale-bounded parameter evolution. Extensive simulation results demonstrate that RPBD reduces task latency by 62.7 % and outage probability by 41.9 % compared to conventional Benders decomposition. Moreover, it maintains latency below 300 ms under 60 km/h mobility and achieves 83.4 % spectrum utilization efficiency in urban CIoV scenarios.

In recent years, deep learning has advanced the MIDI domain, solidifying music generation as a key application of artificial intelligence. However, most research focuses on Western music, facing challenges in generating Chinese traditional melodies, particularly in capturing modal characteristics and emotional expression. To address this, we propose the Dual-Feature Modeling Module, which integrates the long-range modeling of the Mamba Block with the global structure capturing of the Transformer Block. Additionally, we introduce the Bidirectional Mamba Fusion Layer, which integrates local details and global structures through bidirectional scanning, enhancing sequence modeling. Building on this, we propose the REMI-M representation to better capture and generate modal information in melodies. To support this, we developed FolkDB, a high-quality Chinese traditional music dataset covering over 11 hours of music. Experimental results show our architecture excels in generating melodies with Chinese traditional music characteristics, offering a new solution for music generation.

Nowadays, Mobile Edge Computing (MEC) has been widely deployed to enhance the computational capabilities of mobile devices. However, the geographic location of MEC servers is usually fixed. In order to provide flexible edge computing services, some works have considered integrating Autonomous aerial vehicles (AAVs) into MEC networks. In the context of AAV-assisted edge computing, there usually exist multiple AAVs and users, and each AAV may aim to maximize its profit by providing computing services, while users will decide which AAVs to utilize based on their preferences. In this context, how AAVs and users effectively plan their trajectories becomes particularly important as it will affect the profitability of AAVs and the user experience. Since the trajectories of AAVs and users are affected by each other, we model the trajectories of AAVs and users as a Stackelberg game, and then design trajectory planning strategies for users and AAVs based on Independent Proximal Policy Optimization (IPPO) and Proximal Policy Optimization (PPO) respectively, aiming to maximize AAVs’ profits while ensuring user acceptance of AAV services. Finally, we evaluate the proposed trajectory planning strategy against three typical benchmark strategies using synthetic and realistic datasets. The experimental results demonstrate that our strategy can outperform benchmark strategies in terms of AAV profit while guaranteeing users’ service experience.

In multi-agent systems, how to cooperate with each other to jointly confront opponent agents is a very important issue (i.e., multi-agent cooperative confrontation problem). In recent years, multi-agent reinforcement learning has developed rapidly and become a general paradigm for solving multi-agent decision-making problems, so it has naturally been used to solve multi-agent cooperative confrontation problems. However, existing works usually focus on the cooperation among multiple agents, making it difficult for agents to adapt to highly dynamic adversarial environments. To address this issue, in this paper, we propose a multi-agent reinforcement learning algorithm embedded with opponent modeling (MARLeOM). The opponent modeling module constructs multi-level opponent models using the environment model and recursive reasoning, and then mixes multi-level opponent models to enhance the representational capability. The multi-agent reinforcement learning part adopts CTDE mechanism and Actor-Critic framework. In the centralized training phase, Critic can obtain the observation and action information of all agents to guide the learning of Actor; in the decentralized execution phase, Actor makes decisions based on local information, which includes its own observation and the predicted opponent action based on the opponent’s observation using the opponent modeling module. The combination of opponent modeling and multi-agent reinforcement learning enables agents to learn optimal cooperative confrontation methods. Empirical experiments on multiple cooperative adversarial tasks demonstrate that MARLeOM can achieve more effective adaptation and better performance than baseline methods.

Mobile-edge computing has been widely used in the Internet of Things (IoT) field. In mobile-edge computing, users’ tasks can be executed on local devices or offloaded to edge servers for processing. However, in the multiedge server collaboration scenario, service coverage of multiple edge servers may overlap with each other. Therefore, users in the overlapped service coverage areas need to determine which server to be offloaded. Inefficient task offloading strategies may result in unbalanced workloads of the edge servers, which causes negative impacts on task completion latency and the total number of served users. To address this problem, we propose a task offloading and resource allocation strategy in a multiedge server collaboration scenario in this article. In more detail, we consider that task demands arrive dynamically and tasks can be processed locally or offloaded to edge servers. We then model the task offloading and resource allocation problem as a partially observable Markov game (POMG) and propose a task offloading and resource allocation strategy based on a reinforcement learning algorithm I-PDQN to ensure that the task latency requirements are met while maximizing the number of served users and minimizing the average task energy consumption. Finally, we evaluate the performance of the strategy proposed in this article against some typical benchmark strategies under different system parameters through experiments. The experimental results show that the I-PDQN-based task offloading and resource allocation strategy can outperform benchmark approaches.

Ride-sharing contributes significantly to lowering trip expenses, easing traffic congestion and decreasing air pollution. However, current order pairing approaches in ride-sharing usually focus on minimizing total trip distances or maximizing platform profits, overlooking the drivers’ desire for increased earnings. As a result, drivers might provide dishonest information to gain higher profits, leading to inefficient order pairing for the ride-sharing platform and potential losses for both the platform and drivers. In this paper, we address this challenging issue by developing efficient order pairing mechanisms that maximize the social welfare of the platform and drivers. Specifically, we introduce two truthful auction-based order pairing mechanisms, SWMOM-VCG and SWMOM-GM, where drivers bid on platform-published orders to complete them and earn profits. We provide theoretical proof that both mechanisms fulfill the criteria of individual rationality, profitability, truthfulness and so on. Using real taxi order data from New York City, we assess the performance of both mechanisms and show that they achieve greater social welfare compared to existing methods. Additionally, we find that SWMOM-GM requires less computation time than SWMOM-VCG for order pairing, with only a minor reduction in social welfare.

As a new computing paradigm, mobile edge computing can meet users’ computing demands with low latency. In reality, multiple edge servers with different computing capabilities are usually deployed in a distributed manner, which means that computing tasks cannot be offloaded by a centralized manager. In such a situation, how to achieve load balancing among multiple edge servers is a challenging problem. In the edge computing, edge servers usually set prices for the provided service, which will affect users’ offloading costs and thus can have impacts on users’ offloading decisions. Therefore, in this paper, we design a dynamic pricing strategy for edge servers based on multi-agent reinforcement learning, which can affect users’ offloading costs and thus motivate users to make reasonable offloading decisions in order to improve the load balancing of the entire system. Furthermore, we run extensive experiments to evaluate our dynamic pricing strategy against four benchmark strategies. The experimental results show that our strategy can effectively improve the long-term load balancing in different user-demanding environments.

For online ride-hailing platforms, choosing the right time to match idle vehicles with passengers is one of the most important factors affecting the platform's profit.On one hand, vehicles and passengers arrive dynamically, and an appropriate delayed matching may generate a highly efficient matching result with more values.On the other hand, different regions may have different states of supply (vehicles) and demand (passengers), and the matching time should be different.At this moment, we need an efficient matching time strategy that takes into account matching time and regional differences to maximize the platform's long-term profit.In this paper, we propose a dynamic matching time algorithm based on multi-agent reinforcement learning, which is called Multi-Region Differentiated Matching Decision.Firstly, we describe the order matching process and then model it as a decentralized partially observable Markov decision process (Dec-POMDP).Secondly, considering that there are regional differences in supply and demand, we divide the overall area based on historical data and propose an algorithm based on multi-agent reinforcement learning to realize multiregion differentiated dynamic matching.Finally, we conduct extensive experiments to evaluate our matching algorithm against benchmark algorithms in a real-world dataset.The experimental results show that our algorithm can outperform benchmark algorithms.

The rapid development of mobile devices applications has put tremendous pressure on edge nodes with limited computing capabilities, which may cause poor user experience. To solve this problem, collaborative cloud-edge computing is proposed. In the cloud-edge computing, an edge node with limited local resources can rent more resources from a cloud node. Since cloud service providers offer a variety of pricing modes for users' different computing demands, edge node needs to select appropriate pricing mode of cloud service and allocates resources between the cloud node and the edge node. It is a sequential decision problem. In this paper, we model it as a Parameterized Action Markov Decision Process, and propose a resource allocation algorithm Cost Efficient Resource Allocation under collaborative Cloud-Edge (CERACE) based on the deep reinforcement learning algorithm Parametrized Deep Q-learning (P-DQN). We evaluate CERACE against three typical resource allocation algorithms Edge First + On Demand (E+O), Edge + Random (E+R) and Random + Random (R+R) based on synthetic data and real data of Google dataset. The experimental results show that CERACE can effectively reduce the long-term operation cost of collaborative cloud-side computing in various demanding settings. Our analysis can provide some useful insights for enterprises to design the resource allocation strategy in the collaborative cloud-side computing system.

Viaduct structures have been built to ease traffic congestion in large-sized cities, especially in many large-medium cities in China. In Internet of Vehicles (IoV) networks, inter-level links in urban viaduct scenarios are established between two vehicles on different levels with various heights. The bridge surface of the viaduct acts as an obstacle in the propagation paths of inter-level links and may cause a degradation of the link quality. However, inter-level links may bring more commu-nication opportunities when the traffic is sparse or no intra-level link can be found. Compared with traditional three-dimensional wireless networks, inter-level links for viaduct shadowing sce-narios reveal unique characteristics which are not addressed by previous studies. This paper aims to estimate the unique link connectivity characteristics and figure out whether inter-level links can enhance opportunistic routing performance. To this end, we first build an accurate hybrid link availability and reliability model by considering the unique structure of viaducts. Second, we design a new metric over opportunistic routing combined with our link model. Simulation results show that taking advantage of inter-level links can improve the performance of opportunistic routing.

As an economical, low-carbon and convenient travel model, bike-sharing has become common in many cities around the world. However, the daily usage of shared bikes results in the dispatching problem, i.e., dispatching bikes to the specific destinations to satisfy riding demands. The bike-sharing platform can hire riders as workers and pay to incentivize them to accomplish the dispatching tasks. However, there exist multiple workers competing for the dispatching tasks, and they may strategically report their task accomplishing costs (which are usually private information only known by themselves) in order to make more profits, which may result in inefficient task dispatching results. In this paper, we first design a dispatching algorithm named GDY-MAX to allocate tasks to workers, which can achieve good performance . However, it cannot prevent workers strategically misreporting their task accomplishing costs. Regarding this issue, we further design a strategy proof mechanism under the budget constraint, which consists of a task dispatching algorithm and a worker pricing algorithm. We theoretically prove that our mechanism can satisfy incentive compatibility, individual rationality, budget constraint and a constant approximation ratio. Furthermore, we run extensive experiments to evaluate our mechanism based on a Mobike dataset. The results show that the performance of the proposed strategy proof mechanism and GDY-MAX is similar to the optimal algorithm in terms of the coverage ratio of accomplished task regions and the sum of task region value, and our mechanism has better performance than the uniform algorithm in terms of the total payment and the unit cost value.

As a green and low-carbon transportation way, bike-sharing provides lots of convenience in the daily life. However, how to dispatch bikes efficiently is a key issue in such a system. The bike-sharing platform can hire workers and pay to incentivize them to accomplish the dispatching tasks. However, there exist multiple workers competing for the dispatching tasks, and they may strategically report their task accomplishing costs (private information known by themselves) in order to make more profits, which may result in inefficient task dispatching. In this paper, we first design a dispatching algorithm named GDY-MAX to allocate tasks to workers. Furthermore, we design a strategy proof mechanism under the budget constraint to allocate tasks and determine the payments to workers. We theoretically prove that our mechanism can satisfy the properties of incentive compatibility, individual rationality and budget balance. Furthermore we run extensive experiments to evaluate our mechanism based on a Mobike dataset. The results show that our approaches can make better performance than benchmark approaches.

In the cloud market, there exist multiple cloud providers competing against each other in order to attract cloud users and make profits. Each provider needs an appropriate pricing strategy. In this paper, we analyze how a cloud provider sets the price effectively when competing against other cloud providers. The price charged by the cloud provider is affected by many factors such as the prices of its opponents, the price set in the previous round, the choice of cloud users and the marginal cost of the cloud provider. Therefore, we model this problem as a Partially Observable Markov Game where cloud providers compete against each other in the evolutionary market. In more detail, in this game, the amount of cloud users is increasing before it eventually becomes saturated and the marginal value of users is also changing. Then we use a gradient-based multi-agent reinforcement learning algorithm to generate the pricing strategy for the cloud service provider. Finally, we evaluate our pricing strategy against other typical pricing strategies by conducting extensive experiments. The experimental results show that our pricing strategy can not only adapt the price according to the opponents' prices, but also adapt the price according to the changes of the cloud users' valuations on the cloud resource, and therefore it can outperform other pricing strategies. Furthermore, we also find that when training the pricing strategy against the pricing strategy generated by our algorithm, our pricing strategy can still win in terms of the long-term profits. The experimental results can provide useful insights for designing practical pricing strategies.

Nowadays, cloud brokers play an important role for allocating resources in the cloud computing market, which mediate between cloud users and service providers by buying a limited capacity from the providers and subleasing them to the users to make profits. However, the user demands are usually stochastic and the resource capacity bought from cloud providers is limited. Therefore, in order to maximize the profits, the broker needs an effective resource allocation algorithm to decide whether satisfying the demands of arriving users or not, i.e. need to allocate the resource to a valuable user. In this paper, we propose a resource allocation algorithm named Q-DP, which is based on reinforcement learning and dynamic programming, for the broker to maximize the profits. First, we consider all arriving users' demands at each stage as a bundle, and model the process of the broker allocating resources to all arriving users as a Markov Decision Process. We then use the Q-learning algorithm to determine how much resources will be allocated to the bundle of users arriving at the current stage. Next, we use dynamic programming to decide which cloud user will obtain the resources. Finally, we run experiments in the artificial dataset and realistic dataset respectively to evaluate our resource allocation algorithm against other typical resource allocation algorithms, and show that our algorithm can beat other algorithms, especially in the setting of the broker having extremely limited resources.

Ride-sharing has played an important role in reducing travel costs and global pollution. However, existing works of online matching passengers' orders with vehicles usually aims to minimize the total travel distance of drivers or maximize the profit made by the platform running the ride-sharing service. They ignore the fact that vehicle drivers are usually selfish and heterogeneous, and intend to maximize their own payoffs. In this paper, we intend to solve this online matching issue with the aim of maximizing the social welfare of the platform and vehicles. Specifically, we propose two incentive order matching mechanisms based reverse auction (i.e. MSWR-VCG and MSWR-GM), where the vehicle drivers bid for the orders published by the platform to accomplish the order while making profits. We theoretically prove the related properties of our mechanisms, such as truthfulness, individual rationality, budget-balance, etc. We then evaluate the performance of the mechanism based on the real order data of taxis in New York City and demonstrate that our mechanisms can achieve higher social welfare than the state-of-the-art method which is adopted by industry. Furthermore, we find MSWR-VCG can achieve higher payoff for drivers than MSWR-GM and MSWR-GM can balance the effectiveness of social welfare and computational efficiency well.

This study presents a novel approach for assessing and prioritising relative weights of the factors in order to reveal the level of each factors' contribution to the usability index system in the context of library websites. The inputs of experts in the related field were used to probe weights of seven key dimensions and 20 measuring items of library website usability. These factors covering both dimensions and measuring items were identified and validated from the authors' previous study. The study employed fuzzy theories along with the extent analysis fuzzy analytic hierarchy process (FAHP) method with the expectation of scrutinising the relative importance of the criteria of interest while avoiding uncertainty, ambiguity, loss of data, and difficulties in assessment cycle. In order to maintain uniformity in expert consensus, triangular fuzzy numbers were used instead of linguistic values. The research findings revealed that the satisfaction was the most important dimension, while accessibility was considered the least important. In addition, the top 7 of the 20 measurement items investigated accounted for 49.37% of importance. The results indicate that more attention is needed in ensuring those websites are comfortable and fulfilled with clear information required for the users, while less time and effort can be given to readability and compatible capability considerations.

In line with growing pace of advancement of internet technology, the World Wide Web dramatically changed the traditional ways of information collecting, evaluating, storing and disseminating strategies used by information centers together with saving money and time, and created new opportunities and achievements for promoting social and cultural interactions with updated information. Therefore, accordingly to the fast changing environment of the technology, organizations need to develop instructive and well content websites as well as continuously monitor and update their websites performance up-to-date. This research, therefore, investigates the issues of web usability and proposes a fuzzy-based framework for measuring and evaluating the usability of websites particularly library websites. The findings from the usability criteria elicitation, expert reviews, survey and statistical analysis resulted in a comprehensive list of seven web usability dimensions along with 20 measuring items which formed the basis of the evaluation framework called UNSCALE. Further exploration was conducted to elicit weights of the key dimensions and measuring items in order to reveal the level of each dimension and measuring items' contribution to the usability index system through the extent analysis fuzzy analytic hierarchy process method. The framework was developed including with eight cyclical steps for measuring and evaluating web usability by employing the fuzzy comprehensive evaluation method. It can be used to evaluate the overall usability score and also scores regarding each evaluation dimension of any types of websites but is particularly suitable for library websites. The model was tested for its applicability and practicality on a university library website in Sri Lanka, a developing country with a fast growth in terms of internet access.

In this paper, we analyse how multiple competing cloud platforms set effective service prices between Web service providers and consumers. We propose a novel economic framework to model this problem. Cloud platforms run double auction mechanisms, where Web service is commodity traded by service providers (sellers) and service consumers (buyers). Multiple cloud platforms compete against each other to attract service providers and consumers. Specifically, we use game theory to analyse the pricing policies of competing cloud platforms, where service providers and consumers can choose to participate in any of the platforms, and bid or ask for the Web service. The platform selection and bidding strategies of service providers and consumers are affected by the pricing policies and vice versa, and so we propose a co-learning algorithm based on fictitious play to analyse this problem. In more detail, we investigate a setting with two competing cloud platforms who can adopt either equilibrium k pricing policy or discriminatory k pricing policy. We find that, when both cloud platforms use the same type of pricing policy, they can co-exist in equilibrium, and they have an extreme bias to service providers or consumers when setting k. When both platforms adopt different types of policies, we find that all service providers and consumers converge to the discriminatory k pricing policy and so the two competing platforms can no longer co-exist.

In this paper, we analyse how traders select marketplaces and bid in a setting with multiple competing marketplaces. Specifically, we use a fictitious play algorithm to analyse the traders' equilibrium strategies for market selection and bidding when their types are continuous. To achieve this, we first analyse traders' equilibrium bidding strategies in a single marketplace and find that they shade their offers in equilibrium and the degree to which they do this depends on the amount and types of fees that are charged by the marketplace. Building on this, we then analyse equilibrium strategies for traders in competing marketplaces in two particular cases. In the first, we assume that traders can only select one marketplace at a time. For this, we show that, in equilibrium, all traders who choose one of the marketplaces eventually converge to the same one. In the second case, we allow buyers to participate in multiple marketplaces at a time, while sellers can only select one marketplace. For this, we show that sellers eventually distribute in different marketplaces in equilibrium and that buyers shade less and sellers shade more in the equilibrium bidding strategy (since sellers have more market power than buyers).

In this paper we describe the IAMwildCAT agent, designed for the TAC Market Design game which is part of the International Trading Agent Competition. The objective of an agent in this competition is to effectively manage and operate a market that attracts traders to compete for resources in it. This market, in turn, competes against markets operated by other competition entrants and the aim is to maximise the market and profit share of the agent, as well as its transaction success rate. To do this, the agent needs to continually monitor and adapt, in response to the competing marketplaces, the rules it uses to accept offers, clear the market, price the transactions and charge the traders. Given this context, this paper details IAMwildCAT's strategic behaviour and describes the wide techniques we developed to operationalise this. Finally, we empirically analyse our agent in different environments, including the 2007 competition where it ranked first.