KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes. We pose a tokenwise RL objective and propose a modular solver for it, called *controlled decoding (CD)*. CD exerts control through a separate *prefix scorer* module, which is trained to learn a value function for the reward. The prefix scorer is used at inference time to control the generation from a frozen base model, provably sampling from a solution to the RL objective. We empirically demonstrate that CD is effective as a control mechanism on popular benchmarks. We also show that prefix scorers for multiple rewards may be combined at inference time, effectively solving a multi-objective RL problem with no additional training. We show that the benefits of applying CD transfer to an unseen base model with no further tuning as well. Finally, we show that CD can be applied in a blockwise decoding fashion at inference-time, essentially bridging the gap between the popular best-of-$K$ strategy and tokenwise control through reinforcement learning. This makes CD a promising approach for alignment of language models.

Modern online content-sharing platforms host billions of items like music, videos, and products uploaded by various providers for users to discover items of their interests. To satisfy the information needs, the task of effective item retrieval (or item search ranking) given user search queries has become one of the most fundamental problems to online content-sharing platforms. Moreover, the same query can represent different search intents for different users, so personalization is also essential for providing more satisfactory search results. Different from other similar research tasks, such as ad-hoc retrieval and product retrieval with copious words and reviews, items in content-sharing platforms usually lack sufficient descriptive information and related meta-data as features. In this paper, we propose the end-to-end deep attentive model (EDAM) to deal with personalized item retrieval for online content-sharing platforms using only discrete personal item history and queries. Each discrete item in the personal item history of a user and its content provider are first mapped to embedding vectors as continuous representations. A query-aware attention mechanism is then applied to identify the relevant contexts in the user history and construct the overall personal representation for a given query. Finally, an extreme multi-class softmax classifier aggregates the representations of both query and personal item history to provide personalized search results. We conduct extensive experiments on a large-scale real-world dataset with hundreds of million users from a large video media platform at Google. The experimental results demonstrate that our proposed approach significantly outperforms several competitive baseline methods. It is also worth mentioning that this work utilizes a massive dataset from a real-world commercial content-sharing platform for personalized item retrieval to provide more insightful analysis from the industrial aspects.

TV Whitespaces, recently opened up by the Federal Communications Commission (FCC) for unlicensed use, are seen as a potential cellular offload and/or standalone mechanism, especially in dense metros where the demand for throughput is high. In this paper, we use real data collected from whitespaces databases to empirically demonstrate features unique to whitespaces-power-spectrum tradeoff and spatial variation in spectrum availability. From this study, we conclude the need for whitespaces-specific adaptations to cellular networks so as to be able to extract maximum throughput and guarantee reliability. To tackle the effects of the power-spectrum tradeoff, we propose a novel base-station design that specifically uses low-power transmitters as a means to maximize throughput. This design co-locates and networks together many low-powered mode-I devices to act as a multiple-antenna array. We estimate the size of the array required to meet typical rate targets, and show that the array design significantly outperforms traditional designs in terms of throughput for a given cost. We then turn our attention to spatial variability and study its impact on the problem of locating base stations in a whitespaces network. Here, we propose spectrum-aware placement algorithms for whitespaces, which account for this spatial variability along with key parameters like user density. We show that such algorithms clearly outperform traditional placement algorithms and improve network coverage in this band.

Sub-carrier grouping is a popular feedback reduction approach for orthogonal-frequency-division multiple-access (OFDMA) systems that has been adopted into fourth-generation standards such as 3GPP Long Term Evolution (LTE). Feedback reduction is motivated by the fact that the bandwidth expenditure in acquiring full information in a downlink OFDMA system scales as the product of the number of users and the number of OFDMA bands. As this is infeasible in most systems, sub-carrier grouping calls for users to report a single channel state value per predesignated group of OFDM bands. Such an approach would reduce the amount of feedback by a factor that is equal to the size of the group, albeit at a loss in throughput. In this paper, we propose a throughput-optimal joint sub-carrier grouping and data scheduling policy that makes decisions based on queue-lengths and channel states in each time slot. The feedback allocation or sub-carrier grouping policy, inspired by the current approach in LTE, operates under a total feedback budget and must periodically decide a sub-carrier grouping size for each user that obeys this resource constraint. However, as we show, the optimal allocation algorithm has complexity that, in general, scales exponentially in the number of users. Thus, we turn our attention to the important issue of computational efficiency and propose a greedy algorithm that allocates full feedback bandwidth to a constant-sized subset of users based on the network state. We evaluate the performance of this simple approach through extensive numerical experiments. We show that under asymmetric arrival rate settings, our greedy algorithm is within 10% of the optimal (throughput-wise) when consuming only 25% of the full feedback bandwidth while paying only a logarithmic (in the number of users) price in control overhead.

TV Whitespaces, recently opened up by the FCC for unlicensed use by wireless devices, are seen as a potential cellular offload solution, especially in dense metros. However, under the new database-driven guidelines, there are typically very few whitespace bands available in such dense metros to a high-powered fixed device, which plays the role of a cellular base station in whitespaces. To address the lack of degrees-of-freedom (DoF) with this traditional architecture of one high-powered serving device, we propose a novel base station design that co-locates and networks together many low-powered devices to act as a multiple-antenna array. Lower-powered whitespace devices have access to more spectral DoF, a property that is unique to whitespaces. In the first part of the paper, we solve an array design problem where we estimate the size of the array required to meet long-term (worst-case) throughput targets. Using extensive simulations, we show that by effectively exploiting both spatial and spectral DoF, the array design outperforms the traditional design in most network conditions. Specifically, the proposed design can support throughputs of the order of a WiMAX cell running applications such as high-definition television. In the second part of the paper, we turn our attention to the operational aspects of such a design. Recognizing that the proposed array can potentially contain hundreds of elements, we propose a dynamic ON-OFF power control algorithm that operates in conjunction with the MaxWeight data scheduling algorithm and responds to the current network state - queues and channels - of the system, thus making the system power-efficient.

We derive new bounds on the periodic (cyclic) total squared correlation (PTSC) of binary antipodal signature sets for any number of signatures K and any signature length L. Optimal designs that achieve the new bounds are then developed for several (K,L) cases. As an example, it is seen that complete (K = L + 2) Gold sets are PTSC optimal, but not, necessarily, Gold subsets of K <; L + 2 signatures. In contrast, arguably against common expectation, the widely used Kasami sets are not PTSC optimal in general. The optimal sets provided herein are in this sense better suited for asynchronous and/or multipath code-division multiplexing applications.

We derive new bounds on the aperiodic total squared correlation (ATSC) of binary antipodal signature sets for any number of signatures K and any signature length L. We then present optimal designs that achieve the new bounds for several (K,L) cases. As interesting -arguably- side results, we show that individual maximal merit factor sequences (for example Barker sequences) are single-user ATSC-optimal, while neither the familiar Gold nor the Kasami set designs are ATSC-optimal in general. The ATSC-optimal signature set designs provided in this work are in this sense better suited for asynchronous and/or multipath code-division multiplexing applications.

This paper investigates the impact of limited feedback on the uplink of a wireless network with multiple access points. The network adopts universal frequency re-use with each access point serving a set of users using orthogonal signalling. All wireless nodes are multiple-input-multiple-output-capable. Block Diagonalization, a popular technique for interference cancellation, is performed at each transmitter. Interference cancellation on the cell boundary is being considered in future wireless standards such as IEEE 802.16m and 3GPP Long Term Evolution Advanced. In such systems, the precoder is fed back to the mobile by its home base station. It follows that the rate achieved at the home as well as neighbouring access points is a function of the bandwidth of this feedback channel. We study the problem of minimizing the network-wide feedback budget subject to guaranteeing a minimum rate at each access point. We compute the optimal network-wide allocation using dynamic programming on junction trees with complexity that depends on the structure of the underlying interference graph. Unfortunately, when the interference graph forms a grid such as the traditional arrangement of base stations, this complexity becomes exponential in the number of users, N. We propose a greedy allocation algorithm with significantly-reduced complexity and with an approximation guarantee that, provably, scales as N. The constant of proportionality is shown to be a function of the precoder quantizer distortion, cost per feedback bit (for reliable delivery) and the size of the largest neighbourhood of the network.

We derive new bounds on the aperiodic total squared correlation (ATSC) of binary antipodal signature sets for any number of signatures K and any signature length L. We then present optimal designs that achieve the new bounds for several (K, L) cases. As an example, it is seen that the familiar Gold and (small or large) Kasami designs are not ATSC-optimal in general. The optimal signature set designs provided herein are in this sense better suited for asynchronous and/or multipath code-division multiplexing applications.

Recent advances in the area of binary sequences for code-division multiplexing provide us with minimum total-squared-correlation (TSC) optimal signature sets for (almost) all signature lengths L and set sizes K. The sets are scalable as long as K les L (underloaded systems) and non- scalable -in general require signature re-design/re-assignment as users enter or exit- when K > L (overloaded systems). In this paper, we derive new lower bounds on the conditional TSC of overloaded binary signature sets built on fixed full- load TSC-optimal sets. Overloading is allowed to be as high as 100%. Scalable designs that achieve the new bounds are then developed. To evaluate the performance of the proposed designs, we compare the TSC of our constructions to the unconditionally minimum achievable TSC values.