Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples – the many-shot regime. Going from few-shot to many-shot, we observe significant performance gains across a wide variety of generative and discriminative tasks. While promising, many-shot ICL can be bottlenecked by the available amount of human-generated outputs. To mitigate this limitation, we explore two new settings: (1) "Reinforced ICL" that uses model-generated chain-of-thought rationales in place of human rationales, and (2) "Unsupervised ICL" where we remove rationales from the prompt altogether, and prompts the model only with domain-specific inputs. We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. We demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases, can learn high-dimensional functions with numerical inputs, and performs comparably to supervised fine-tuning. Finally, we reveal the limitations of next-token prediction loss as an indicator of downstream ICL performance.

In this paper we investigate the properties of representations learned by deep reinforcement learning systems. Much of the early work on representations for reinforcement learning focused on designing fixed-basis architectures to achieve properties thought to be desirable, such as orthogonality and sparsity. In contrast, the idea behind deep reinforcement learning methods is that the agent designer should not encode representational properties, but rather that the data stream should determine the properties of the representation—good representations emerge under appropriate training schemes. In this paper we bring these two perspectives together, empirically investigating the properties of representations that support transfer in reinforcement learning. We introduce and measure six representational properties over more than 25,000 agent-task settings. We consider Deep Q-learning agents with different auxiliary losses in a pixel-based navigation environment, with source and transfer tasks corresponding to different goal locations. We develop a method to better understand why some representations work better for transfer, through a systematic approach varying task similarity and measuring and correlating representation properties with transfer performance. We demonstrate the generality of the methodology by investigating representations learned by a Rainbow agent that successfully transfers across Atari 2600 game modes.

State construction is important for learning in partially observable environments. A general purpose strategy for state construction is to learn the state update using a Recurrent Neural Network (RNN), which updates the internal state using the current internal state and the most recent observation. This internal state provides a summary of the observed sequence, to facilitate accurate predictions and decision-making. At the same time, specifying and training RNNs is notoriously tricky, particularly as the common strategy to approximate gradients back in time, called truncated Back-prop Through Time (BPTT), can be sensitive to the truncation window. Further, domain-expertise—which can usually help constrain the function class and so improve trainability—can be difficult to incorporate into complex recurrent units used within RNNs. In this work, we explore how to use multi-step predictions to constrain the RNN and incorporate prior knowledge. In particular, we revisit the idea of using predictions to construct state and ask: does constraining (parts of) the state to consist of predictions about the future improve RNN trainability? We formulate a novel RNN architecture, called a General Value Function Network (GVFN), where each internal state component corresponds to a prediction about the future represented as a value function. We first provide an objective for optimizing GVFNs, and derive several algorithms to optimize this objective. We then show that GVFNs are more robust to the truncation level, in many cases only requiring one-step gradient updates.

In model-based reinforcement learning, planning with an imperfect model of the environment has the potential to harm learning progress. But even when a model is imperfect, it may still contain information that is useful for planning. In this paper, we investigate the idea of using an imperfect model selectively. The agent should plan in parts of the state space where the model would be helpful but refrain from using the model where it would be harmful. An effective selective planning mechanism requires estimating predictive uncertainty, which arises out of aleatoric uncertainty, parameter uncertainty, and model inadequacy, among other sources. Prior work has focused on parameter uncertainty for selective planning. In this work, we emphasize the importance of model inadequacy. We show that heteroscedastic regression can signal predictive uncertainty arising from model inadequacy that is complementary to that which is detected by methods designed for parameter uncertainty, indicating that considering both parameter uncertainty and model inadequacy may be a more promising direction for effective selective planning than either in isolation.

A new parameter estimation algorithm, MERLIN, is presented for accurate and robust multiexponential relaxometry using magnetic resonance imaging, a tool that can provide valuable insight into the tissue microstructure of the brain. Multi-exponential relaxometry is used to analyze the myelin water fraction and can help to detect related diseases. However, the underlying problem is ill-conditioned, and as such, is extremely sensitive to noise and measurement imperfections, which can lead to less precise and more biased parameter estimates. MERLIN is a fully automated, multi-voxel approach that incorporates state-of-the-art ℓ1-regularization to enforce sparsity and spatial consistency of the estimated distributions. The proposed method is validated in simulations and in vivo experiments, using a multi-echo gradient-echo (MEGE) sequence at 3 T. MERLIN is compared to the conventional single-voxel ℓ2-regularized NNLS (rNNLS) and a multivoxel extension with spatial priors (rNNLS + SP), where it consistently showed lower root mean squared errors of up to 70 percent for all parameters of interest in these simulations.

In this paper, we proposed a home energy management (HEM) scheme for minimization in electricity bills and reduction in peak load. This can be achieved by scheduling the usage timings of appliances (APP) for shifting load from peak hours (PHs) to OFF-peak hours (OPHs). In this study we proposed a technique which is hybrid of two bio-inspired optimization techniques chicken swarm optimization (CSO) and bacterial foraging optimization (BFA). Simulation results shows that proposed hybrid technique reduces the cost and load peaks by shifting load from PHs to OPHs and reduction in load peaks.

In this paper, our goal is to minimize the electricity cost, electricity consumption at minimum user discomfort while considering the peak electricity consumption. Electricity consumption may not be the same in residential, commercial and industrial areas. It may vary from each and every area. It is a challenging task to maintain the balance between the conflicting objectives: electricity consumption and user comfort. To meet the rising electricity demand in residential area, scheduleable devices can be equally distributed to the available time slots on the basis of average power consumption. The main objective is to minimize the electricity usage during the electricity peak hours by distributing the electricity load during the off-peak hours. In this regard, Genetic Algorithm (GA), Pigeon Inspired Optimization (PIO) and our proposed hybridization of GA and PIO (HGP) in Demand Side Management(DSM) are applied for residential load management to optimize the fitness function. GA, PIO and HGP are evaluated on the basis of real time pricing scheme (RTP) for single home with three different operational time interval (OTI) and for multiple homes with a single OTI. Simulations results shows that GA, PIO and HGP are able to minimize electricity bill and electricity consumption while minimizing the user discomfort. The performance of HGP is better than GA, PIO with respect to PAR, electricity load and electricity cost for both single home and multiple homes scenario. The feasible region between electricity cost and electricity consumption is also represented. Moreover, the desired trade-off between electricity cost and user comfort is also achieved in both techniques.

Home Energy Management Systems (HEMS) have been widely used for energy management in smart homes. Energy management in a smart home is a challenging task, which require efficient scheduling of appliances. The main focus of HEMS is to schedule the operation of appliances in such a way that it gives us optimized performance in terms of Peak to Average Ratio (PAR), Electric Cost (EC) minimization, execution time and User Comfort (UC). The Time of Use (ToU) pricing scheme is used in this paper. We used Genetic Algorithm (GA), Biogeography-based optimization (BBO) and our proposed hybrid Genetic Biogeography-based Optimization (GBBO), techniques to schedule appliances in single home and for multiple homes. Simulations are carried out using eight different appliances. The results show that GA and GBBO execute better in case of PAR reduction and EC minimization. GBBO outperforms in terms of user comfort. We calculated the UC in terms of waiting time.

A new reconstruction method, coined MIRAGE, is presented for accurate, fast, and robust parameter mapping of multiple-echo gradient-echo (MEGE) imaging, the basis sequence of novel quantitative magnetic resonance imaging techniques such as water content and susceptibility mapping. Assuming that the temporal signal can be modeled as a sum of damped complex exponentials, MIRAGE performs model-based reconstruction of undersampled data by minimizing the rank of local Hankel matrices. It further incorporates multi-channel information and spatial prior knowledge. Finally, the parameter maps are estimated using nonlinear regression. Simulations and retrospective undersampling of phantom and in vivo data affirm robustness, e.g., to strong inhomogeneity of the static magnetic field and partial volume effects. MIRAGE is compared with a state-of-the-art compressed sensing method, -ESPIRiT. Parameter maps estimated from reconstructed data using MIRAGE are shown to be accurate, with the mean absolute error reduced by up to 50% for in vivo results. The proposed method has the potential to improve the diagnostic utility of quantitative imaging techniques that rely on MEGE data.

The radio interface is the bottleneck of most wireless communication systems. All techniques used to enhance this interface require certain knowledge of the radio environment. This knowledge usually concerns either short-term, or long-term models and statistical characteristics of the environment, depending on the technique. Short-term models are suitable especially for signal processing applications, whereas long-term models are mostly used by radio resource management techniques. Most of the existing work is either focusing on outdoor propagation or short-term models. In this paper we present first results on developing long-term indoor propagation models based on extensive measurements. The results show that fast fading in indoor propagation models cannot be modeled with one distribution alone and thus dynamic models are required. Moreover, slow fading and its characteristics change very often and cannot be modeled with fixed distribution over time.

E-learning can be loosely defined as a wide set of applications and processes, which uses available electronic media (and tools) to deliver vocational education and training. With its increasing recognition as an ubiquitous mode of instruction and interaction in the academic as well as corporate world, the need for a scaleable and realistic model is becoming important. In this paper, we introduce SELF; a semantic grid-based e-learning framework. SELF aims to identify the key-enablers in a practical grid-based e-learning environment and to minimize technological reworking by proposing a well-defined interaction plan among currently available tools and technologies. We define a dichotomy with e-learning specific application layers on top and semantic grid-based support layers underneath. We also map the latest open and freeware technologies with various components in SELF.