Reinforcement learning (RL) can automatically learn a better policy through a trial-and-error paradigm and has been adopted to revolutionize and optimize traditional traffic signal control systems that are usually based on handcrafted methods. However, most existing RL-based models are either based on a single scenario or multiple independent scenarios, where each scenario has a separate simulation environment with predefined road network topology and traffic signal settings. These models implement training and testing in the same scenario, thus being strictly tied up with the specific setting and sacrificing model generalization heavily. While a few recent models could be trained by multiple scenarios, they require a huge amount of manual labor to label the intersection structure, hindering the model’s generalization. In this work, we aim at ageneralframework that could eliminate heavy labeling and model a variety of scenariossimultaneously. To this end, we propose a general Scenario-Agnostic (GESA) reinforcement learning framework for traffic signal control with: (1) A general plug-in module to map all different intersections into a unified structure, freeing us from the heavy manual labor to specify the structure of intersections; (2) A unified state and action space design to keep the model input and output consistently structured; (3) A large-scale co-training with multiple scenarios, leading to a generic traffic signal control algorithm. GESA can automatically handle various structured intersections from various cities without human labeling, and it co-trains a generalist agent to control traffic signals for multiple cities together, which also demonstrates superior transferability in zero-shot settings. In experiments, we demonstrate our algorithm as the first one that can be co-trained with seven different scenarios without manual annotation and gets13.27%higher rewards than baselines. When dealing with a new scenario, our model can still achieve9.39%higher rewards. The code, scenarios, and demos are available https://github.com/bonaldli/GESA.

This paper proposes to learn Multi-task, Multi-modal Direct Acyclic Graphs (MM-DAGs), which are commonly observed in complex systems, e.g., traffic, manufacturing, and weather systems, whose variables are multi-modal with scalars, vectors, and functions. This paper takes the traffic congestion analysis as a concrete case, where a traffic intersection is usually regarded as a DAG. In a road network of multiple intersections, different intersections can only have someoverlapping and distinct variables observed. For example, a signalized intersection has traffic light-related variables, whereas unsignalized ones do not. This encourages the multi-task design: with each DAG as a task, the MM-DAG tries to learn the multiple DAGs jointly so that their consensus and consistency are maximized. To this end, we innovatively propose a multi-modal regression for linear causal relationship description of different variables. Then we develop a novel Causality Difference (CD) measure and its differentiable approximator. Compared with existing SOTA measures, CD can penalize the causal structural difference among DAGs with distinct nodes and can better consider the uncertainty of causal orders. We rigidly prove our design's topological interpretation and consistency properties. We conduct thorough simulations and one case study to show the effectiveness of our MM-DAG. The code is available under https://github.com/Lantian72/MM-DAG.

To comply with sustainability goals, many companies buy green energy to serve their energy demand. This is typically done by engaging in bilateral power purchase agreements (PPA) with renewable energy producers (REP). A PPA can be flexibly structured, but the core principle is that a buyer (company) agrees to buy future energy production of a seller (REP) at an agreed-upon fixed price. PPAs are financially attractive for sellers, providing price certainty, unlike trading in electricity markets. However, PPAs can bring quantity uncertainty for buyers due to the uncertainty of future green energy delivery. This uncertainty in the long-term endangers sustainability targets, and in the short-term complicates reliable and cost-efficient demand matching. Thus, multiple strategies have been used in PPAs to encourage sellers to provide accurate and good-faith predictions of their short-term and longer-term future production. Yet, it has been shown that REPs can have incentives to misreport predicted values. This has discouraged some companies from engaging in PPAs. In this paper, we first investigate how PPA structure and pricing can incentivize REPs to provide more reliable predictions. This shifts the risk of production uncertainty to REPs, increasing the chance that REPs adopt batteries. We further study how having batteries for REPs affects their own revenue as well as the reliability of their energy predictions for buyers. We use analytical and simulation approaches to propose a decision tree for a win-win PPA structure, which improves reliability for buyers while maintaining profitability for REPs.

Emerging domains such as smart electric grids require decisions to be made autonomously, based on the observed behaviors of large numbers of connected consumers. Existing approaches either lack the flexibility to capture nuanced, individualized preference profiles, or scale poorly with the size of the dataset. We propose a preference model that combines flexible Bayesian nonparametric priors—providing state-of-the-art predictive power—with well-justified structural assumptions that allow a scalable implementation. The Gaussian process scalable preference model via Kronecker factorization ( GaSPK ) model provides accurate choice predictions and principled uncertainty estimates as input to decision-making tasks. In consumer choice settings where alternatives are described by few key attributes, inference in our model is highly efficient and scalable to tens of thousands of choices.

Electric vehicles will play a crucial role in balancing the future electrical grid, which is complicated by many intermittent renewable energy sources. We developed an algorithm that determines for a fleet of electric vehicles, which EV at what price and location to commit to the operating reserve market to either absorb excess capacity or provide electricity during shortages (vehicle-2-grid). The algorithm takes the value of immobility into account by using carsharing fees as a reference point. A virtual power plant autonomously replaces cars that are committed to the operating reserves and are then rented out, with other idle cars to pool the risks of uncertainty. We validate our model with data from a free float carsharing fleet of 500 electric vehicles. An analysis of expected future developments (2015, 2018, and 2022) in operating reserve demand and battery costs yields that the gross profits for a carsharing operator increase between 7-12% with a negligible decrease in car availability (<0.01%).

The growing Electric Vehicles' (EVs) popularity among commuters creates new challenges for the smart grid. The most important of them is the uncoordinated EV charging that substantially increases the energy demand peaks, putting the smart grid under constant strain. In order to cope with these peaks the grid needs extra infrastructure, a costly solution. We propose an Adaptive Management of EV Storage (AMEVS) algorithm, implemented through a learning agent that acts on behalf of individual EV owners and schedules EV charging over a weekly horizon. It accounts for individual preferences so that mobility service is not violated but also individual benefit is maximized. We observe that it reshapes the energy demand making it less volatile so that fewer resources are needed to cover peaks. It assumes Vehicle-to-Grid discharging when the customer has excess capacity. Our agent uses Reinforcement Learning trained on real world data to learn individual household consumption behavior and to schedule EV charging. Unlike previous work, AMEVS is a fully distributed approach. We show that AMEVS achieves significant reshaping of the energy demand curve and peak reduction, which is correlated with customer preferences regarding perceived utility of energy availability. Additionally, we show that the average and peak energy prices are reduced as a result of smarter energy use.

Sustainable energy systems of the future will need more than efficient, clean, and low-cost energy sources. They will also need efficient price signals that motivate sustainable energy consumption behaviors and a tight real-time alignment of energy demand with supply from renewable and traditional sources. The Power Trading Agent Competition (Power TAC) is a rich, competitive, open-source simulation platform for future retail power markets built on real-world data and state-of-the-art customer models. Its purpose is to help researchers understand the dynamics of customer and retailer decision-making as well as the robustness of proposed market designs. Power TAC invites researchers to develop autonomous electricity broker agents and to pit them against best-in-class strategies in global competitions, the first of which will be held at AAAI 2013. Power TAC competitions provide compelling, actionable information for policy makers and industry leaders. We describe the competition scenario, demonstrate the realism of the Power TAC platform, and analyze key characteristics of successful brokers in one of our 2012 pilot competitions between seven research groups from five different countries.

We present SmartRate, a trust and reputation-based decision framework for Smart Grid, based on the available ratings provided by other customers. This model considers multiple trust factors associated with the broker and the preferences of customers for each of these factors. We define a decision framework for broker selection based on multi-attribute utility function and show how learning customers' rating behaviors helps to increase a decision maker's utility. We evaluate this framework by simulating a market based on real-world data. Our results show that learning the characteristics of a rating population helps to interpret and personalize the ratings, which results in better decision making and an increase in customer satisfaction.

Hedonic modeling is used to measure the product price behavior overall in high-tech markets. In a previous work, we showed the opportunity to extend the simple regression to a state space model evaluating hedonic prices from product prices. We created and tested an online estimation algorithm for those values. In that way, we can study time series of implicit prices for individual components of a range of products. In this paper, we implement and compare the hedonic model forecast performances respect to standard autoregressive models, univariate and multivariate. We find that hedonic values not only give extra information about supply market, but they can improve univariate predictions and in, certain periods, also multivariate ones. We show the correctness of algorithm using online version of it. An agent may predict prices for different products sharing a set of component, by taking into account the structure of production process. An application in a multi-agent supply chain simulation confirms the goodness of algorithm to be implemented in a future framework for online price analysis and prediction.

Future sustainable energy systems will need efficient, clean, low-cost, renewable energy sources, as well as market structures that motivate sustainable behaviors on the part of households and businesses. "Smart grid" components can help consumers manage their consumption only if pricing policies are in place that motivate consumers to install and use these new tools in ways that maximize utilization of renewable energy sources while minimizing dependence on non-renewable energy. Serious market breakdowns, such as the California energy crisis in 2000, have made policy makers wary of setting up new retail energy markets. We present the design of an open, competitive simulation approach that will produce robust research results on the structure and operation of retail power markets as well as on automating market interaction by means of competitively tested and bench-marked electronic agents. These results will yield policy guidance that can significantly reduce the risk of instituting competitive energy markets at the retail level, thereby applying economic motivation to the problem of adjusting our energy production and consumption patterns to the requirements of a sustainable future.

Accurate forecasting of market price developments is essential in achieving superior market performance. Especially in oligopolistic markets for durable consumer products a robust understanding of selling prices is important, as it drives pricing behavior as well as procurement, inventory and production decisions. Moreover, a supply chain perspective is indispensable for pricing forecasts since companies not only compete for product sales but also for limited resources. This paper explores the use of dynamic multivariate hedonics-based pricing models that explicitly model selling prices with the market valuation of constituting parts. The model is applied to TAC SCM, a supply-chain trading agent competition. To find unknown component prices series we apply the Kalman filter technique to smooth and forecast implicit prices using the EM algorithm. Finally, we present results of our analysis to establish the viability of this method.

Business intelligence (BI) takes many different forms, as indicated by the varying definitions of BI that can be found in industry and academia. These different definitions help us understand of what BI issues are important to the main players in the field of BI; users, suppliers and academics. The goal of this research is to discover gaps and trends from the standpoints of BI users, BI suppliers and academics, and to examine their effects on business and academia. Consultants also play an important role since they can be seen as the link between users and suppliers. Two research methods are combined to accomplish this goal. We examine the BI focus of users and suppliers through a survey, and we gain insight to the BI focus of academics, vendor-neutral consultants (typical representatives like Forrester, Gartner and IDC) and vendor- specific consultants (typical representatives like IBM, Information builders, Microsoft, Oracle and SAP) through their publications. Previous studies indicate that similar article analyses often focus on academic research methods only. That means that the results so far often reveal the academic perspective. Unlike these previous studies, the perspective of this research is not limited to academics. Our results provide insight of the BI trends and BI issue ranking of BI users, suppliers, academics, vendors neutral consultants and vendor specific consultants.

Information systems are crucial for effective supply chain management in today's complex supply chains for durable goods. Complex decision making processes on strategic, tactical, and operational level require substantial support in order to contribute to the agility of organizations. Supply chain regimes, i.e., regimes encompassing both the sales and the procurement market in a complex supply chain, provide a way of intuitively and meaningfully characterizing and modeling supply chain market conditions without a need for explicit modeling of individual aspects of the market. This paper makes a first explorative step towards a model incorporating such regimes, while maintaining the dynamics which enable the model to be utilized in the sales process, e.g., for dynamic product pricing. Initial results show that supply chain regimes have feasible characteristics, based on both sales and procurement market indicators. Taking into consideration these regimes enables a more deliberate sales model.

An effective Decision Support System (DSS) should help its users improve decision making in complex, information-rich environments. We present a feature gap analysis that shows that current decision support technologies lack important qualities for a new generation of agile business models that require easy, temporary integration across organisational boundaries. We enumerate these qualities as DSS Desiderata, properties that can contribute both effectiveness and flexibility to users in such environments. To address this gap, we describe a new design approach that enables users to compose decision behaviours from separate, configurable components, and allows dynamic construction of analysis and modelling tools from small, single-purpose evaluator services. The result is what we call an ‘evaluator service network’ that can easily be configured to test hypotheses and analyse the impact of various choices for elements of decision processes. We have implemented and tested this design in an interactive version of the MinneTAC trading agent, an agent designed for the Trading Agent Competition for Supply Chain Management.

We investigate the effects of adding procurement information (component offer prices) to a sales-based economic regime model, which is used for strategic, tactical, and operational decision making in dynamic supply chains. The performance of the regime model is evaluated through experiments with the MinneTAC trading agent, which competes in the TAC SCM game. We find that the new regime model has a similar overall predictive performance as the existing model. Regime switches are predicted more accurately, whereas the prediction accuracy of dominant regimes is slightly worse. However, by adding procurement information, we have enriched the model and we have further opportunities for applications in the procurement market, such as procurement reserve pricing.

In today's complex supply chains, product pricing is a vital, yet non-trivial task. We propose a product pricing approach using adaptive real-time probability of acceptance estimations based on economic regimes. Radial Basis Function Networks are trained to estimate parameters for double-bounded log-logistic distributions assumed to be underlying daily offer prices, using information available real-time. The relation between data and parameters is dynamically modeled using economic regimes (characterizing market conditions) and error terms (accounting for customer feedback). Given the parametric approximations of price distributions, acceptance probabilities are estimated using a closed-form mathematical expression, which is used to determine the price yielding a desired quota. The approach is implemented in the MinneTAC agent and tested against a price-following product pricing method in the TAC SCM game. Performance significantly improves; more customer orders are obtained against higher prices and profits more than double.

The world-wide-web (WWW) today consists of distinct, isolated islands of data and metadata. In the near future we expect the availability of a critical mass of data and metadata for use by intelligent agents that act on behalf of human users. These agents would identify, propose and capture new opportunities to assist human users in satisfying their goals, by traversing and acting on this semantically rich and abundant information. We envision a new class of agents, their networks and their communities that exist for the sole purpose of serving as their human "master's" Advocates - Advocate Agents. Advocate Agents learn a human's goals and preferences, collaborate with other agents, mine semantic content, identify new opportunities for action, propose them and finally transact them, while always keeping the human "in-the-loop." This paper discusses this class of distributed, intelligent, Advocate Agents, their potential uses, and proposed architectures and techniques that provide a conceptual framework for these networked agent societies to collaborate in the achievement of their human user's goals.

We present a semi-parametric model that describes pricing behaviors in a market environment, and we show how that model can be used to guide resource allocation and pricing decisions in an autonomous trading agent. We validate our model by presenting experimental results obtained in the Trading Agent Competition for Supply Chain Management.