As the final stage of recommender systems, re-ranking presents ordered item lists to users that best match their interests. It plays such a critical role and has become a trending research topic with much attention from both academia and industry. Recent advances of re-ranking are focused on attentive listwise modeling of interactions and mutual influences among items to be re-ranked. However, principles to guide the learning process of a re-ranker, and to measure the quality of the output of the re-ranker, have been always missing. In this paper, we study such principles to learn a good re-ranker. Two principles are proposed, including convergence consistency and adversarial consistency. These two principles can be applied in the learning of a generic re-ranker and improve its performance. We validate such a finding by various baseline methods over different datasets.

Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffers from substantial communication overhead, which can affect training efficiency. Recent efforts have mitigated this issue by quantizing model updates to reduce communication costs. However, most existing methods apply quantization only after local training, introducing quantization errors into the trained parameters and potentially degrading model accuracy. In this paper, we propose Federated Bit Freezing (FedBiF), a novel FL framework that directly learns quantized model parameters during local training. In each communication round, the server first quantizes the model parameters and transmits them to the clients. FedBiF then allows each client to update only a single bit of the multi-bit parameter representation, freezing the remaining bits. This bit-by-bit update strategy reduces each parameter update to one bit while maintaining high precision in parameter representation. Extensive experiments are conducted on five widely used datasets under both IID and Non-IID settings. The results demonstrate that FedBiF not only achieves superior communication compression but also promotes sparsity in the resulting models. Notably, FedBiF attains accuracy comparable to FedAvg, even when using only 1 bit-per-parameter (bpp) for uplink and 3 bpp for downlink communication. The code is available athttps://github.com/Leopold1423/fedbif-tpds25.

In Federated Learning (FL), the data in each client is typically assumed fixed or static. However, data often comes in an incremental manner in real-world applications, where the data domain may increase dynamically. In this work, we study catastrophic forgetting with data heterogeneity in Federated Incremental Learning (FIL) scenarios where edge clients may lack enough storage space to retain full data. We propose to employ a simple, generic frame-work for FIL named Re-Fed, which can coordinate each client to cache important samples for replay. More specifically, when a new task arrives, each client first caches selected previous samples based on their global and local im-portance. Then, the client trains the local model with both the cached samples and the samples from the new task. The-oretically, we analyze the ability of Re-Fed to discover important samples for replay thus alleviating the catastrophic forgetting problem. Moreover, we empirically show that Re-Fed achieves competitive performance compared to state-of-the-art methods.

Federated learning (FL) is a popular way of edge computing that does not compromise user's privacy. Current FL paradigms assume data only resides on the edge, while cloud servers only perform model averaging. However, in real-life situations such as recommender systems, the cloud server usually has abundant features and computation resources. Specifically, the cloud stores historical and interactive features, and the edge stores privacy-sensitive and real-time features. In this paper, our proposed Edge-Cloud Collaborative Knowledge Transfer Framework (ECCT) jointly utilizes the edge-side features and the cloud-side features, enabling bi-directional knowledge transfer between the two by sharing feature embeddings and prediction logits. ECCT consolidates various benefits, including enhancing personalization, enabling model heterogeneity, tolerating training asynchronization, and relieving communication burdens. Extensive experiments on public and industrial datasets demonstrate the effectiveness of ECCT.

Despite that path-based and embedding-based models with knowledge graphs (KGs) achieve better recommendation performance compared with other deep learning based methods, such improvement is limited due to a lack of modeling user's dynamic interest. To address this issue, we explore a principled model to provide semantic understanding of each item in user's historical interest sequence in KGs. Specifically, we propose a multi-granularity dynamic interest sequence learning method, which is based on knowledge-enhanced path mining and interest fluctuation signal discovery, to obtain semantic-enhanced paths. Furthermore, the paths are embedded by the SEP2Vec, and merged through the proposed entropy-aware pooling layer to obtain the user preference representation, which is then used to learn dynamic user interest sequence. Experimental results on two public datasets of movie and music recommendation, and two industrial datasets of personalized local service recommendation in Alipay App have illustrated that the proposed model can achieve significantly better prediction performance compared with other known baselines.

Deep Candidate Generation plays an important role in large-scale recommender systems. It takes user history behaviors as inputs and learns user and item latent embeddings for candidate generation. In the literature, conventional methods suffer from two problems. First, a user has multiple embeddings to reflect various interests, and such number is fixed. However, taking into account different levels of user activeness, a fixed number of interest embeddings is sub-optimal. For example, for less active users, they may need fewer embeddings to represent their interests compared to active users. Second, the negative samples are often generated by strategies with unobserved supervision, and similar items could have different labels. Such a problem is termed as class collision. In this paper, we aim to advance the typical two-tower DNN candidate generation model. Specifically, an Adaptive Interest Selection Layer is designed to learn the number of user embeddings adaptively in an end-to-end way, according to the level of their activeness. Furthermore, we propose a Prototypical Contrastive Learning Module to tackle the class collision problem introduced by negative sampling. Extensive experimental evaluations show that the proposed scheme remarkably outperforms competitive baselines on multiple benchmarks.

In critical environments that require a high accuracy of decisions, utilizing human cognitive strengths and expertise in addition to machine observations is advantageous to improve decision quality and enhance situational awareness. While the current literature on human decision making is primarily based on the paradigm of perfect rationality, humans are subject to decision noise and employ stochastic choice rules. Human decision making under such realistic environments needs to be further studied. In this paper, instead of assuming that a human selects the optimal action with probability one, we employ a bounded rationality choice model where all the actions are candidates for selection, but better options are chosen with higher probabilities. In a Bayesian hypothesis testing framework, we evaluate the individual decision making performance when humans have different degrees of bounded rationality. Furthermore, we analyze the decision fusion rule for a team of two human agents and characterize the asymptotic performance of collaborative decision making as the number of human participants becomes large.

Recently, modeling temporal patterns of user-item interactions have attracted much attention in recommender systems. We argue that existing methods ignore the variety of temporal patterns of user behaviors. We define the subset of user behaviors that are ir- relevant to the target item as noises, which limits the performance of target-related time cycle modeling and affect the recommendation performance. In this paper, we propose Denoising Time Cycle Modeling (DiCycle), a novel approach to denoise user behaviors and select the subset of user behaviors that are highly related to the target item. DiCycle is able to explicitly model diverse time cycle patterns for recommendation. Extensive experiments are conducted on both public benchmarks and a real-world dataset, demonstrating the superior performance of DiCycle over the state-of-the-art recommendation methods.

The first provably efficient algorithm for learning graph neural networks (GNNs) with one hidden layer for node information convolution is provided in this paper. Two types of GNNs are investigated, depending on whether labels are attached to nodes or graphs. A comprehensive framework for designing and analyzing convergence of GNN training algorithms is developed. The algorithm proposed is applicable to a wide range of activation functions including ReLU, Leaky ReLU, Sigmod, Softplus and Swish. It is shown that the proposed algorithm guarantees a linear convergence rate to the underlying true parameters of GNNs. For both types of GNNs, sample complexity in terms of the number of nodes or the number of graphs is characterized. The impact of feature dimension and GNN structure on the convergence rate is also theoretically characterized. Numerical experiments are further provided to validate our theoretical analysis.

The problem of quickest change detection in anonymous heterogeneous sensor networks is studied. The sensors are clustered into$K$groups, and different groups follow different data generating distributions. At some unknown time, an event occurs in the network and changes the data generating distribution of the sensors. The goal is to detect the change as quickly as possible, subject to false alarm constraints. The anonymous setting is studied, where at each time step, the fusion center receives unordered samples without knowing which sensor each sample comes from, and thus does not know its exact distribution. In [1], an optimal algorithm was provided, which however is not computational efficient for large networks. In this paper, a computationally efficient test is proposed and a novel theoretical characterization of its false alarm rate is further developed.

We study the problem of a sensor performing inference tasks based on the utility theory, where the objective is to derive the optimal resource usage amount that maximizes a profit-cost-based utility function. Furthermore, to enable the concept ofsensing as a servicein the context of IoT systems, we present a market-based paradigm, where there is a “buyer” interested in buying the inference result from the sensor. We jointly optimize the resource usage policy and payment negotiation strategy for the sensor so as to maximize the expected profit. Optimal payment negotiation is analyzed in two situations, namely, when the sensor spends a fixed amount of resource, as well as when the sensor could vary the amount of resource consumption to maximize profit. It is shown that in the presence of the buyer, the optimal amount of resource consumption increases and, hence, the inference accuracy improves. Finally, we present some discussions on how energy efficiency affects the behavior of energy consumption in realistic environments. Simulation results are provided to illustrate the performance of our approach.

The problem of quickest change detection (QCD) in anonymous heterogeneous sensor networks is studied. There are n heterogeneous sensors and a fusion center. The sensors are clustered into K groups, and different groups follow different data generating distributions. At some unknown time, an event occurs in the network and changes the data generating distribution of the sensors. The goal is to detect the change as quickly as possible, subject to false alarm constraints. The anonymous setting is studied in this paper, where at each time step, the fusion center receives n unordered samples. The fusion center does not know which sensor each sample comes from, and thus does not know its exact distribution. In this paper, a simple optimality proof is derived for the Mixture Likelihood Ratio Test (MLRT), which was constructed and proved to be optimal for the non-sequential anonymous setting in [1]. For the QCD problem, a mixture CuSum algorithm is constructed in this paper, and is further shown to be optimal under Lorden's criterion [2].

In this paper, we present a statistical mechanics framework to understand the effect of sampling properties of training data on the generalization gap of machine learning (ML) algorithms. We connect the generalization gap to the spatial properties of a sample design characterized by the pair correlation function (PCF). In particular, we express generalization gap in terms of the power spectra of the sample design and that of the function to be learned. Using this framework, we show that space-filling sample designs, such as blue noise and Poisson disk sampling, which optimize spectral properties, outperform random designs in terms of the generalization gap and characterize this gain in a closed-form. Our analysis also sheds light on design principles for constructing optimal task-agnostic sample designs that minimize the generalization gap. We corroborate our findings using regression experiments with neural networks on: a) synthetic functions, and b) a complex scientific simulator for inertial confinement fusion (ICF).

We explore the design of an effective crowdsourcing system for an M -ary classification task. Crowd workers complete simple binary microtasks whose results are aggregated to give the final decision. We consider the scenario where the workers have a reject option so that they are allowed to skip microtasks when they are unable to or choose not to respond to binary microtasks. We present an aggregation approach using a weighted majority voting rule, where each worker's response is assigned an optimized weight to maximize crowd's classification performance.

In this work, we investigate the accelerated proximal gradient method for nonconvex programming (APGnc). The method compares between a usual proximal gradient step and a linear extrapolation step, and accepts the one that has a lower function value to achieve a monotonic decrease. In specific, under a general nonsmooth and nonconvex setting, we provide a rigorous argument to show that the limit points of the sequence generated by APGnc are critical points of the objective function. Then, by exploiting the Kurdyka-Lojasiewicz (KL) property for a broad class of functions, we establish the linear and sub-linear convergence rates of the function value sequence generated by APGnc. We further propose a stochastic variance reduced APGnc (SVRG-APGnc), and establish its linear convergence under a special case of the KL property. We also extend the analysis to the inexact version of these methods and develop an adaptive momentum strategy that improves the numerical performance.

In this paper, an adaptive two-way relay cooperation scheme is studied for multiple-relay cognitive radio networks to improve the performance of secondary transmissions. The power allocation and relay selection schemes are derived to minimize the secondary outage probability where only statistical channel information is needed. Exact closed-form expressions for secondary outage probability are derived under a constraint on the quality of service of primary transmissions in terms of the required primary outage probability. To better understand the impact of primary user interference on secondary transmissions, we further investigate the asymptotic behaviors of the secondary relay network, including power allocation and outage probability, when the primary signal-to-noise ratio goes to infinity. Simulation results are provided to illustrate the performance of the proposed schemes.

A multiple-input single-output cognitive radio downlink network is studied with simultaneous wireless information and power transfer. In this network, a secondary user coexists with multiple primary users and multiple energy harvesting receivers. In order to guarantee secure communication and energy harvesting, the problem of robust secure artificial noise-aided beamforming and power splitting design is investigated under imperfect channel state information (CSI). Specifically, the transmit power minimization problem and the max-min fairness energy harvesting problem are formulated for both the bounded CSI error model and the probabilistic CSI error model. These problems are non-convex and challenging to solve. A 1-D search algorithm is proposed to solve these problems based on S-Procedure under the bounded CSI error model and based on Bernstein-type inequalities under the probabilistic CSI error model. It is shown that the optimal robust secure beamforming can be achieved under the bounded CSI error model, whereas a suboptimal beamforming solution can be obtained under the probabilistic CSI error model. A tradeoff is elucidated between the secrecy rate of the secondary user receiver and the energy harvested by the energy harvesting receivers under a max-min fairness criterion.

As spectrum sensing detects the presence of PU signal, an efficient and reliable spectrum sensing scheme plays a critical role in CR. For this purpose, sequential sensing technique is introduced to reduce the sensing time to the minimum while desirable detection performance is maintained. However the sensing time could still be unacceptably long due to the weak PU signal, especially in non-Gaussian noise. To improve spectrum sensing efficiency, we propose a novel sequential sensing scheme based on suprathreshold stochastic resonance (SSR). We address the theoretical bound to achieve potential performance improvement and give the applicable algorithm of SSR-based sequential sensing scheme. In the scheme, the average sample number (ASN) is reduced in a single sensing node using nonlinear stochastic resonance method. The simulation results show that the proposed scheme significantly outperforms the conventional scheme, especially in low signal-to-noise ratio (SNR) scenario.

To tackle the problem of performance degradation of traditional spectrum sensing technique with energy detection under the circumstances of weak signal and non-Gaussian environment in cognitive radio (CR), a novel spectrum sensing method based on suprathreshold stochastic resonance (SSR) is proposed in this paper. Through the resonance between the primary user (PU) signal and noise by introducing nonlinearity of a parallel of quantizers, the signal-to-noise ratio (SNR) of the received signal can be increased when the constraint we develop is satisfied. To obtain a constant false-alarm rate (CFAR), the detection probability of the proposed method is derived. Theoretical analyses and simulation results show that the detection performance is superior to the conventional energy detection method under low SNR circumstances when the a certain range of non-Gaussian noise is input.