Upper Confidence Bound (UCB) methods are one of the most effective methods in dealing with the exploration-exploitation trade-off in online decision-making problems. The confidence bounds utilized in UCB methods tend to be constructed based on concentration equalities which are usually dependent on a parameter of scale (e.g. a bound on the payoffs, a variance, or a subgaussian parameter) that must be known in advance. The necessity of knowing a scale parameter a priori and the fact that the confidence bounds only use the tail information can deteriorate the performance of the UCB methods. Here we propose a data-dependent UCB algorithm called MARS (Maximum Average Randomly Sampled) in a non-parametric setup for multi-armed bandits with symmetric rewards. The algorithm does not depend on any scaling, and the data-dependent upper confidence bound is constructed based on the maximum average of randomly sampled rewards inspired by the work of Hartigan in the 1960s and 70s. A regret bound for the multi-armed bandit problem is derived under the same assumptions as for the $\psi$-UCB method without incorporating any correction factors. The method is illustrated and compared with baseline algorithms in numerical experiments.

Irrigation networks of open-water channels are used throughout the world to support agricultural activity. By and large, these networks are managed in open loop. To achieve closed-loop water distribution management, it is necessary to augment these civil engineering systems with an appropriate information infrastructure-sensors, actuators, information processing, and communication resources. Recent pilot projects in Australia demonstrate the significant potential of closed-loop management, which can yield a significant improvement in the quality of service, while achieving improved water distribution efficiency. This paper focuses on the modelling and closed-loop control of open-water channels from the perspective of large-scale irrigation network management. Several feedback information structures are discussed and the key design tradeoffs identified