Differential Architecture Search (DARTS) has become a prominent technique for neural architecture search in recent years. Despite its merits, the issue of discretization discrepancy within DARTS still necessitates further exploration, as it can degrade in performance. In this paper, we introduce a novel algorithm termed Continuous–Discrete Alignment Optimization (DARTS-CDAO), designed to address the discretization discrepancy and thereby enhance the robustness and generalization capabilities of the discovered neural architectures. Our proposed DARTS-CDAO algorithm seamlessly integrates the discretization process into the training phase of the architecture parameters, thereby bolstering the search algorithm’s adaptability to the inherent discretization processes. Specifically, our methodology commences by formalizing the process of architecture parameter discretization. Subsequently, we introduce a coarse gradient weighting algorithm that is employed to update the architecture parameters, effectively minimizing the divergence between the representation of continuous and discrete parameters. Rigorous theoretical analysis, coupled with extensive experimental outcomes, substantiates that our proposed approach can elevate the performance of the searched models. Notably, this enhancement is achieved without incurring additional search time, rendering DARTS more robust and endowed with a heightened capacity for generalization.

This paper introduces a new approach for promoting diversity of behavior in complex multi-agent environments that pose three challenges: 1) competition or collaboration among agents of diverse types, 2) the need for complex multi-agent coordination, which makes it challenging to achieve risky cooperation strategies, and 3) a large number of agents in the environment, leading to increased complexity when considering agent-to-agent relationships. To address the first two challenges, we leverage Reward Randomization in combination with Bayesian Optimization to train agents to exhibit diverse strategic behaviors, thereby mitigating the issue of risky cooperation. To address the challenge of learning in a large number of agents, we utilize MAPPO with parameter sharing to enhance learning efficiency. Experimental results demonstrate that within this multi-agent environment, agents can effectively learn multiple visually distinct behaviors, and the incorporation of these two techniques significantly improves agents' performance.

Despite the success of recent reinforcement learning (RL) in traffic signal control which has shown to outperform the conventional control methods, current RL-based methods require large amounts of samples to learn and lack the generalization ability to a new environment. In order to solve these problems, we propose a new context-based meta-RL model that disentangles task inference and control, which improves the meta-training efficiency and accelerates the learning process in a new environment. Moreover, the Graph Attention Network is employed to achieve effective cooperation between intersections. The experiments show that our method not only improves the traffic control efficiency but also converges faster and performs more stably, compared with traditional, RL-based, and meta-RL-based traffic control methods.

With the fast development of autonomous vehicle technologies, the vehicle fleet will be made up of a mixture of human-driven vehicles and autonomous vehicles (AVs) in the coming 20-30 years. To efficiently utilize abundant data to deal with the mixed-autonomy traffic control problem, this paper formulates and approaches the problem using a deep reinforcement learning framework (DRL). DRL is a promising data-driven approach for traffic signal control and AVs control in a large-scale grid. However, traffic control based DRL is quite challenging since the complexity of control and the large search space of the policy. To deal with these issues, we propose a hierarchical joint control framework based on prior knowledge. Specifically, traffic signals at intersections and AVs are controlled by their local controllers, set according to well-adjusted policies; while the coordination of the traffic signals at intersections and the coordination among AVs are determined by two master controllers, respectively. Thus, the control of the whole grid is handled by two master controllers. In this way, the dimension of the action space is greatly decreased and the control operates much smoother. We verify our method by implementing a series of experiments in SUMO. The numerical experiments demonstrate the potential of the mixed-autonomy traffic control, compared with traditional traffic signal systems without AVs. We also demonstrate that our method is easy to train and operates robustly.

Hyperparameter optimization is a key part of AutoML. In recent years, there have been successful hyperparameter optimization algorithms. However, these methods still face several challenges, such as high cost of evaluating large models or large datasets. In this paper, we introduce a new deep reinforcement learning architecture with model-based acceleration to optimize hyperparameters for any machine learning model. In this method, an agent constructed by a Long Short-Term Memory Network aims at maximizing the expected accuracy of a machine learning model on a validation set. To speed up training, we employ a model to predict the accuracy on a validation set instead of evaluating a machine learning model. To effectively train the agent and the predictive model, Real-Predictive-Real training process is proposed. Besides, to reduce the variance, we propose a bootstrap pool to guide the exploration in the search space. The experiment was carried out by optimizing hyperparameters of two widely used machine learning models: Random Forests and XGBoost. Experimental results show that the proposed method outperforms random search, Bayesian optimization, and Tree-structured Parzen Estimator in terms of accuracy, time efficiency and stability.

This paper investigates high frequency currency trading with neural networks trained via Reinforcement Learning. A neural network-based agent is proposed to learn the temporal pattern in data and automatically trades according to the current market condition and the historical data. We propose two techniques: action shaping and advantage function shaping, to improve the total profit. The action shaping is used to avoid the agent outputting illegal actions since we assume that the agent trades fixed position sizes in a single security. The advantage function shaping is proposed to increase the probability of actions that lead to more profit. The proposed system has been back-tested on the currency market. The results demonstrate that our method performs well in most conditions.

We introduce a new deep reinforcement learning architecture - RPR-BP to optimize hyperparameter for any machine learning model on a given data set. In this method, an agent constructed by a Long Short-Term Memory Network aims at maximizing the expected accuracy of a machine learning model on a validation set. At each iteration, it selects a set of hyperparameters and uses the accuracy of the model on the validation set as the reward signal to update its internal parameters. After multiple iterations, the agent learns how to improve its decisions. However, the computation of the reward requires significant time and leads to low sample efficiency. To speed up training, we employ a neural network to predict the reward. The training process for the agent and the prediction network is divided into three phases: Real-Predictive-Real (RPR). First, the agent and the prediction network are trained by the real experience; then, the agent is trained by the reward generated from the prediction network; finally, the agent is trained again by the real experience. In this way, we can speed up training and make the agent achieve a high accuracy. Besides, to reduce the variance, we propose a Bootstrap Pool (BP) to guide the exploration in the search space. The experiment was carried out by optimizing hyperparameters of two widely used machine learning models: Random Forest and XGBoost. Experimental results show that the proposed method outperforms random search, Bayesian optimization and Tree-structured Parzen Estimator in terms of accuracy, time efficiency and stability.

With the development of computer science technology and artificial intelligence, quantitative trading attracts more investors due to its efficiency and stable performance. In this paper, we explore the potential of deep reinforcement learning in quantitative trading. A LSTM-based agent is proposed to learn the temporal pattern in data and automatically trades according to the current market condition and the historical data. The input to the agent is the raw financial data and the output of the agent is decision of trading. The goal of the agent is to maximize the ultimate profit. Besides, to reduce the influence of noise in the market and to improve the performance of the agent, we use several technical indicators as an extra input. The proposed system has been back-tested on the stock market. The results demonstrate that our method performs well in most conditions.

Location is an important context on which a broad range of context-aware applications can be built. Most previous approaches require the targets to carry electronic devices, while device-free passive localization is in need on many occasions. This paper proposes a device-free passive localization algorithm based on WiFi Channel State Information(CSI) and Support Vector Machines(SVM). In a physical space covered with WiFi signals, movements of targets may cause observable alteration of CSI. By establishing the nonlinear relationship between CSI fingerprints and target locations through SVM regression, the proposed algorithm is able to estimate the target locations according to the corresponding CSI fingerprints. The algorithm applies Density-Based Spatial Clustering of Applications with Noise(DBSCAN) to reduce the noise in CSI fingerprints, and applies Principal Component Analysis(PCA) to extract the most useful features and reduce the dimension of CSI fingerprints. Evaluations achieved the mean localization error distance of 1.22m, outperforming the Received Signal Strength Indication(RSSI) approach by 43.0%, and outperforming SVM classification and Naive Bayesian by 39.9% and 41.9%.

In this paper, we propose new controls for a simple intersection based on new information and communication system for intelligent vehicles. These controls are performed to take into account all vehicle arrivals individually. Hence, we consider that there is no traffic light planned by the city and vehicles negotiate their time of access between them or through an intelligent device embedded in the intersection. The intersection is modelled as a resource shared between vehicles of roads. Intersection control becomes to determine the best access order to the intersection for all vehicles which are approaching it. The objective is to evacuate all vehicles as soon as possible. This paper shows that negotiation between vehicles by means of well adapted approaches is efficient to improve traffic control at a simple intersection.