Despite some promising results in federated learning using game-theoretical methods, most existing studies mainly employ a one-level game in either a cooperative or competitive environment, failing to capture the complex dynamics among participants in practice. To address this issue, we propose DualGFL, a novel federated learning framework with a dual-level game in cooperative-competitive environments. DualGFL includes a lower-level hedonic game where clients form coalitions and an upper-level multi-attribute auction game where coalitions bid for training participation. At the lower-level DualGFL, we introduce a new auction-aware utility function and propose a Pareto-optimal partitioning algorithm to find a Pareto-optimal partition based on clients' preference profiles. At the upper-level DualGFL, we formulate a multi-attribute auction game with resource constraints and derive equilibrium bids to maximize coalitions' winning probabilities and profits. A greedy algorithm is proposed to maximize the utility of the central server. Extensive experiments on real-world datasets demonstrate DualGFL's effectiveness in improving both server utility and client utility.

The advent of ubiquitous computing devices in the Internet of Things (IoT) has resulted in an explosion of data. Traditional centralized machine learning models face challenges including limited bandwidth in wireless environments and privacy concerns due to their data aggregation approach. Federated learning addresses these challenges via decentralizing model training across numerous devices, leveraging model updates to enhance privacy and reduce communication overhead. To improve its cost efficiency, current research focuses on minimizing either time or energy costs but rarely both, and does not jointly optimize the parameters of device and training scheduling in the presence of system and data heterogeneity inherent in IoT networks. In our paper, we first introduce a multi-group transmission scheme and propose a comprehensive device scheduling framework, Group Scheduling on Orthogonal Frequency-Division Multiple Access (GS-OFDMA), to address time bottlenecks. Then we formulate a joint optimization problem for device and training scheduling that minimizes the total cost of training while ensuring model convergence. To tackle the resulting mixed integer nonlinear programming problem, we develop an iterative algorithm. Experimental results show that our approach significantly reduces the total cost by at least 35% across various real-world datasets and data distributions in comparison with random participant selection. The proposed GS-OFDMA protocol also exhibits higher time efficiency over other device scheduling schemes.

Federated learning enables decentralized model training across numerous devices without data centralization, leveraging model updates to enhance privacy and reduce communication overhead. Despite its advantages, federated learning systems must be optimized for cost efficiency, considering the limited computational capabilities and battery life of edge devices. Current research often focuses on minimizing either time or energy costs but rarely both, and does not jointly optimize the parameters of device and training scheduling in the presence of system and data heterogeneity. In our paper, we formulate a novel joint optimization problem for device and training scheduling that minimizes the total cost of federated learning while ensuring model convergence. We propose a new device scheduling scheme, Group Scheduling on Orthogonal Frequency-Division Multiple Access (GS-OFDMA), to improve time efficiency and develop an iterative algorithm to tackle the resulting mixed integer nonlinear programming problem. Our experimental results show that our approach significantly reduces the total cost by at least 35 % across different real-world datasets and data distributions in comparison with random participant selection.

Federated learning is a distributed machine learning paradigm that allows multiple edge devices to collaboratively train a shared model without exchanging raw data. However, the training efficiency of federated learning is highly dependent on client selection. Moreover, due to the varying wireless communication environments and various computation latencies among the clients, selecting clients randomly or uniformly may not be optimal for balancing the data diversity and training efficiency. In this article, we formulate a new latency-minimization problem that simultaneously optimizes client selection and training procedures in federated learning, which takes into account the data and latency heterogeneity among the clients. Given the nonconvexity of the problem, we derive a new convergence upper bound for federated learning with probabilistic client selection. To solve the mixed integer nonlinear programming problem, we introduce a hybrid solution that integrates grid search techniques with the polyhedral active set algorithm. Numerical analyses and experiments on real-world data demonstrate that our scheme outperforms the existing ones in terms of overall training latency and achieves up to three times acceleration over random client selection, especially in scenarios with highly heterogeneous data and latencies among the clients.

Previous Sentiment Analysis (SA) studies have demonstrated that exploring sentiment cues from multiple synchronized modalities can effectively improve the SA results. Unfortunately, until now there is no publicly available dataset for multimodal SA of the stock market. Existing datasets for stock market SA only provide textual stock comments, which usually contain words with ambiguous sentiments or even sarcasm words expressing opposite sentiments of literal meaning. To address this issue, we introduce a Fine-grained Multimodal Sentiment Analysis dataset built upon 1,247 Stock Comment videos, called FMSA-SC. It provides both multimodal sentiment annotations for the videos and unimodal sentiment annotations for the textual, visual, and acoustic modalities of the videos. In addition, FMSA-SC also provides fine-grained annotations that align text at the phrase level with visual and acoustic modalities. Furthermore, we present a new fine-grained multimodal multi-task framework as the baseline for multimodal SA on the FMSA-SC. Data and codes are available athttps://github.com/sunlitsong/FMSA-SC-dataset.git.

Temporal event sequences associated with different event types (e.g., location indices, disease types) are observed in various applications such as disaster resilience, criminology, and healthcare. Temporal point processes (TPPs) have been developed to capture the exciting patterns between events and forecast future events quantitatively. Unfortunately, the events with different types often suffer from unknown biased observations in real-world scenarios due to external interference. Accordingly, the temporal point processes learned by conventional maximum likelihood estimation (MLE) from such biased data may be misspecified and may lead to inaccurate predictions. To overcome this issue, we model biased event sequences as modulating TPPs with additional unknown thinning processes. Furthermore, we develop a novel debiased imitation learning framework to learn the modulated TPPs and suppress the negative influences of biased data, which is more robust than conventional MLE. When applying the debiased imitation learning framework, we design a simple but effective reward function based on the historical embedding obtained by the TPP model. Experiments on three real-world datasets demonstrate that our proposed method significantly outperforms existing methods.

Cross-Modal Zero-Shot Hashing (CMZSH) is an important image retrieval technique, e.g., Text Based Image Retrieval. Most of existing CMZSH methods mainly use semantic attributes as guidance to generate hash codes for both the images and texts of seen and unseen categories. However, existing CMZSH methods only focus on learning global attribute vectors and hash codes for images, which mixes up information of complex semantics and background clutters, and thus impedes the retrieval performance. To solve this issue, we propose an Attribute-Guided Multiple Instance Hashing (AG-MIH) network for CMZSH, where each instance represents one image region. Instead of generating global image hash codes, AG-MIH can effectively learn instance-level hash codes based on instance attributes. To improve the attribute learning for instances, AG-MIH can exploia novel 2-D Category-Attribute Relation (CAR) layer, which uses different matching templates to model the relationships between each instance and the attributes for different categories. Under the guidance of semantic attributes, AG-MIH can effectively learn hash codes for each visual instance and texts by a Multi-stream Instance Hashing Refinement (MIHR) procedure. In the MIHR, the pseudo supervisions for the instance-level attributes and hash codes in each stream are from its proceeding stream. Empirical studies on benchmark datasets show that AG-MIH achieves state-of-the-art performance on both cross-modal and single-modal zero-shot image retrieval tasks.

Multivariate Hawkes processes have been widely used in many applications such as crime detection and disaster rescue forecast to model events that exhibit self-exciting properties. One of the biggest challenges is that data collected from real world is usually incomplete, and even biased. The training of a machine learning model using such data can introduce biased predictions. For example, event hotspot predictions using biased data can make the visibility of minority groups (e.g., communities of racial minorities) more apparent. While there have been some explorations in developing Hawkes processes for event data, none of those methods deals with incomplete biased data where events of certain markers (e.g., events reported from racial minorities) may be missing or heavily underrepresented. In this paper, we propose a novel Multivariate Hawkes model to tackle the incomplete biased data challenge. First, we assume that there is possibility that events can be missing between any two observed events and we define a novel likelihood function integrating missing window probabilities. A Markov Chain Monte Carlo (MCMC) sampling framework is used to generate virtual event data probabilistically in missing windows. Second, we propose to incorporate event marker features such as geographic information to regularize the infectivity kernel matrix between markers. In such a way, the MCMC sampler is encouraged to generate more virtual events with markers that are biased. Both observed and virtual events will contribute to the model estimation through maximizing the log-likelihood. We carry on experiments over several real-world datasets, and our model improves prediction accuracy in comparison with the state-of-arts.

In spectrum sharing systems based on spectrum trading, user locations are vital for the efficiency of dynamic channel reuse. However, both primary users (PUs) and secondary users (SUs) undertake the risk of location information leakage: a malicious PU may illegally collect SUs' location information to manipulate market decisions; a malicious SU would threat a PU's operational privacy by inferring the PU's location through seemingly inoffensive queries. To protect both PUs' and SUs' location information in spectrum trading, a bilateral privacy preservation framework is introduced in this paper. A game-theoretic approach based on the Stackelberg model is proposed to achieve the tradeoff between the privacy-preserving level and user utility. With the proposed approach, both PUs and SUs can maximize their utilities while maintaining their location privacy to desired levels. Simulation results demonstrate that the proposed approach can effectively enhance user utility gain and strengthen user privacy guarantee by flexibly adjusting their privacy levels in practice.

In this paper, the problem of privacy-utility tradeoff in a database-driven spectrum sharing system is considered, where both primary users (PUs) and secondary users (SUs) may suffer from location privacy leakage. To protect the location information of both parties, a bilateral privacy preservation mechanism is introduced, in which the privacy-preserving levels are quantified. To tackle the dilemma that a higher privacy level leads to less available spectrum to share and thus reduces the profits of both parties, a game-theoretic approach based on the Stackelberg model is proposed to achieve the tradeoff between the privacy-preserving level and user utility. With the proposed approach, both PUs and SUs can maximize their utilities by adjusting their location privacy to desired levels. Simulation results demonstrate that the proposed approach can effectively increase the utilities for both PUs and SUs in comparison with the privacy preservation mechanism with a fixed privacy level.

Large volumes of human action image data are becoming increasingly available due to the prevalence of surveillance cameras and smart personal devices. While such image data enables important applications such as activity recognition for health and safety enhancement, they often contain sensitive information such as identities that introduce high risks to individual privacy. Existing image privacy-enhancing techniques are either developed at the cost of sacrificing image utility or lack of provable privacy guarantees. We propose a novel human action image generation model that enforces rigorous differential privacy protection. Theoretical analysis is provided to quantify the privacy protection on the training data within the differential privacy framework. Experiments with real-world datasets demonstrate that images generated using our method achieve higher image utilities than baselines given similar degrees of privacy protection.

Many types of event sequence data exhibit triggering and clustering properties in space and time. Point processes are widely used in modeling such event data with applications such as predictive policing and disaster event forecasting. Although current algorithms can achieve significant event prediction accuracy, the historic data or the self-excitation property can introduce biased prediction. For example, hotspots ranked by event hazard rates can make the visibility of a disadvantaged group (e.g., racial minorities or the communities of lower social economic status) more apparent. Existing methods have explored ways to achieve parity between the groups by penalizing the objective function with several group fairness metrics. However, these metrics fail to measure the fairness on every prefix of the ranking. In this paper, we propose a novel list-wise fairness criterion for point processes, which can efficiently evaluate the ranking fairness in event prediction. We also present a strict definition of the unfairness consistency property of a fairness metric and prove that our list-wise fairness criterion satisfies this property. Experiments on several real-world spatial-temporal sequence datasets demonstrate the effectiveness of our list-wise fairness criterion.

Hawkes processes are popular for modeling correlated temporal sequences that exhibit mutual-excitation properties. Existing approaches such as feature-enriched processes or variations of Multivariate Hawkes processes either fail to describe the exact mutual influence between sequences or become computational inhibitive in most real-world applications involving large dimensions. Incorporating additional geometric structure in the form of graphs into Hawkes processes is an effective and efficient way for improving model prediction accuracy. In this paper, we propose the Geometric Hawkes Process (GHP) model to better correlate individual processes, by integrating Hawkes processes and a graph convolutional recurrent neural network. The deep network structure is computational efficient since it requires constant parameters that are independent of the graph size. The experiment results on real-world data show that our framework outperforms recent state-of-art methods.

To enable heterogeneous channel valuations in spatial spectrum reuse, user characteristics involving the supply and demand relationship need to be considered. In this paper, we design a channel transaction mechanism for non-symmetric networks and maximize the social welfare in consideration of multi-level channel valuations of the secondary users (SUs). Specifically, we group the SUs into allowable user crowds (AUCs) through a modified Bron-Kerbosch algorithm. We introduce a Vickrey-Clarke-Groves (VCG) auction, in which the participants are limited to the AUCs. To facilitate the bid formation, we transform the constrained VCG auction to a step-by-step decision process. In each step, the truthful bidding of an AUC is to reveal the accumulated channel valuation of the coalition. Meanwhile, the SUs in a coalition play a coalitional game with transferable utilities. We use the Shapley value to realize fair payoff distribution among the SUs in a coalition. Furthermore, we approach the optimal channel allocation via a greedy algorithm and batch allocation. In our simulation, we compare the low-complexity algorithms and demonstrate the efficiency of the channel transaction mechanism.

Machine learning models especially deep neural network models have shown great potential in making decisions when analyzing clustered or longitudinal data. However, lack of model transparency is a major concern in risk sensitive domains such as social science and medical diagnosis. Despite the early success of explaining machine learning models, there is a lack of explanation methods that can be applied to any predictors on clustered data since most of the existing models assume that all observations are independent of each other. In this paper, we address this deficiency and propose to use a linear mixed model to mimic the local behavior of any complex model on clustered data, which can also improve the fidelity of the explanation method to the complex models. We apply our method to explain several models including a deep neural network model on two tasks including movie recommendation and medical record diagnosis. Experiment results show that our model outperforms the baseline models on several metrics such as fidelity and exactness.

Spatial spectrum reuse enables better utilization of limited spectral resources to achieve higher system throughput. However, improving the system throughput or spectrum efficiency does not necessarily translate to the satisfaction of more secondary users (SUs) according to their demands. To improve user satisfaction, user characteristics involving the supply and demand relationship need to be considered and thus enable heterogeneous channel valuations in spatial spectrum reuse. In this paper, we design a channel transaction mechanism for non-symmetric networks and maximize user satisfaction in consideration of multi-level flexible channel valuations of the SUs. Specifically, we introduce a Vickrey-Clarke-Groves (VCG) auction, in which the participants are limited to the allowable user crowds. To facilitate the bid formation, we transform the constrained VCG auction to a step-by-step decision process. Meanwhile, the SUs in a coalition play a coalitional game with transferable utilities. We use the Shapley value to realize fair payoff distribution among the SUs in a coalition. Furthermore, we approach the optimal channel allocation via finding the longest path in a directed acyclic graph, a greedy algorithm, and batch allocation. In our simulation, we compare the low-complexity algorithms and demonstrate the efficiency of the channel transaction mechanism.

Cooperative relaying is highly affected by the way that the source and relays are incentivized. However, the existing studies have not paid enough attention to the impacts of negotiation power and information availability of the players on the network performance. In this paper, two incentive mechanisms for cooperative relaying are first proposed to explore the influence of negotiation power, wherein the source holds either strong or weak negotiation power. In these two cases, the source posts take-it-or-leave-it offers for the relays and has to negotiate possible deals with the relays, respectively. The relay selection rules and the optimal amounts of relaying service and rewards are derived for each mechanism, respectively. Another two incentive mechanisms are also proposed to explore the influence of information availability, wherein the source has either weakly or strongly incomplete information about the relays. In these two cases, the source acquires the number of relays belonging to each type and the probability of each relay belonging to a certain type, respectively. The relay selection rules and the optimal contract offers are derived for the two mechanisms, respectively. The numerical results are provided to verify the theoretical analyses and demonstrate the effectiveness of the proposed mechanisms.

Cooperative relaying in wireless networks is strongly affected by the way that the source and relays are incentivized. However, existing studies have not paid enough attention to the influence of negotiation power of the involved parties. In this paper, two incentive mechanisms for cooperative relaying are proposed, wherein the source possesses different degrees of negotiation power. One mechanism is for the source with strong negotiation power posting a series of take-it-or- leave-it contract offers for the relays while the relays are not entitled to negotiate the counteroffers, and the other mechanism is for the source with weak negotiation power while the relays confer substantial negotiation power, i.e., are capable of doing business with the source by crafting more profitable deals. The relay selection rules, the optimal amounts of relaying service, and the optimal rewards for the relays, are derived for the proposed mechanisms, respectively. A distributed algorithm is further proposed to iteratively obtain the optimal solution for the second mechanism. A case study is also presented to show the influence of negotiation power on the behaviors of the participants and the efficiency and distribution of profits. Numerical results are provided to verify the theoretical analyses of the proposed mechanisms.

In this paper, a network consisting of a source- destination pair and multiple relays in the presence of a smart jammer who can launch multi- channel jamming attacks is considered, where the direct link between the source and destination does not exist. A game-theoretic framework is proposed to analyze the conflict between the jammer and the legitimate nodes, i.e., the source and the relays, and the cooperation among the legitimate nodes. Specifically, a two-level Stackelberg game is formulated, where the jammer as a leader combats against the legitimate nodes by allocating the jamming powers at the upper level, and the relays as leaders cooperate with the source by selling forwarding powers at the lower level after observing the strategy of the jammer. The Stackelberg equilibrium of the proposed game is derived and analyzed. An algorithm is further designed to obtain the optimal jamming power allocation. Numerical results are provided to verify the theoretical analysis and show the effectiveness of the proposed algorithm.

Hawkes processes have become very popular in modeling multiple recurrent user-item interaction events that exhibit mutual-excitation properties in various domains. Generally, modeling the interaction sequence of each user-item pair as an independent Hawkes process is ineffective since the prediction accuracy of future event occurrences for users and items with few observed interactions is low. On the other hand, multivariate Hawkes processes (MHPs) can be used to handle multi-dimensional random processes where different dimensions are correlated with each other. However, an MHP either fails to describe the correct mutual influence between dimensions or become computational inhibitive in most real-world events involving a large collection of users and items. To tackle this challenge, we propose local low-rank Hawkes processes to model large-scale user-item interactions, which efficiently captures the correlations of Hawkes processes in different dimensions. In addition, we design an efficient convex optimization algorithm to estimate model parameters and present a parallel algorithm to further increase the computation efficiency. Extensive experiments on real-world datasets demonstrate the performance improvements of our model in comparison with the state of the art.

User demographics such as age and gender are very useful in recommender systems for applications such as personalization services and marketing, but may not always be available for individual users. Existing approaches can infer users' private demographics based on ratings, given labeled data from users who share demographics. However, such labeled information is not always available in many e-commerce services, particularly small online retailers and most media sites, for which no user registration is required. We introduce a novel probabilistic matrix factorization model for demographic transfer that enables knowledge transfer from the source domain, in which users' ratings and the corresponding demographics are available, to the target domain, in which we would like to infer unknown user demographics from ratings. Our proposed method is based on two observations: (1) Items from different but related domains may share the same latent factors such as genres and styles, and (2) Users who share similar demographics are likely to prefer similar genres across domains. This approach can align latent factors across domains that share neither common users nor common items, associating user demographics with latent factors in a unified framework. Experiments on cross-domain datasets demonstrate that the proposed method consistently improves demographic classification accuracy over existing methods.

In this paper, the energy efficiency in a cellular network with device-to-device communications is studied. A mixed-integer max-min optimization problem is formulated with both mode selection and resource allocation. Since the optimal solution requires an exhaustive search, a low-complexity decomposition (LCD) method is derived. A fairness-aware mode selection scheme, a subcarrier assignment scheme, and a mode switching scheme are introduced and analyzed. Moreover, a novel power allocation scheme is proposed, which exploits the property of the fractional structure of the energy efficiency optimization problem over multiple subcarriers. The proposed LCD method is scalable and suitable for a large number of users and subcarriers. Simulation results demonstrate that our proposed LCD method achieves satisfactory energy efficiency performance and promotes the fairness among individual users.

In this paper, secure transmission in a heterogeneous network in the presence of multiple eavesdroppers is studied. A game-theoretic framework is proposed to enhance the security of the macro base station (MBS), while guaranteeing user satisfaction of the small base stations (SBSs) by exploiting the cooperation and competition among the entire network. Specifically, a two-level Stackelberg game is formulated, where the MBS employs a set of competing jamming SBSs to jam the eavesdroppers at the top level, and each employed jamming SBS may require offloading service from multiple competing helping SBSs in its cluster at the bottom level if needed. Two levels of user satisfaction are investigated at the bottom level, respectively. One is fixed with the priority given to user satisfaction over the profit and the other is flexible with a balance between user satisfaction and the profit. The Stackelberg equilibrium of the proposed game is derived and analyzed from the economics viewpoint. An iterative algorithm is also proposed to obtain the optimal solutions. Numerical results are provided to verify the theoretical analysis and show the effectiveness of the proposed algorithm.

Deep convolutional neural networks (CNNs) have shown superior performance on the task of single-label image classification. However, the applicability of CNNs to multi-label images still remains an open problem, mainly because of two reasons. First, each image is usually treated as an inseparable entity and represented as one instance, which mixes the visual information corresponding to different labels. Second, the correlations amongst labels are often overlooked. To address these limitations, we propose a deep multi-modal CNN for multi-instance multi-label image classification, called MMCNN-MIML. By combining CNNs with multi-instance multi-label (MIML) learning, our model represents each image as a bag of instances for image classification and inherits the merits of both CNNs and MIML. In particular, MMCNN-MIML has three main appealing properties: 1) it can automatically generate instance representations for MIML by exploiting the architecture of CNNs; 2) it takes advantage of the label correlations by grouping labels in its later layers; and 3) it incorporates the textual context of label groups to generate multi-modal instances, which are effective in discriminating visually similar objects belonging to different groups. Empirical studies on several benchmark multi-label image data sets show that MMCNN-MIML significantly outperforms the state-of-the-art baselines on multi-label image classification tasks.

In this paper, secure transmission in a two-tier heterogeneous network, consisting of a macrocell and a set of small cells, is considered, where an eavesdropper attempts to wiretap legitimate macrocell users. A game-theoretic framework is proposed to enhance the security of the macrocell while guaranteeing user satisfaction of the small cells. Specifically, a two-level Stackelberg game is formulated, where the macro base station as a follower employs multiple small base stations (SBSs) to jam the eavesdropper at the top level, and each employed SBS as a leader requires offloading service from multiple helping SBSs in its cluster at the bottom level if needed. Two types of objectives are investigated at the bottom level, one with the priority given to user satisfaction over the leader's profit, and the other to balance between user satisfaction and the leader's profit, respectively. The Stackelberg equilibrium of the proposed game is also derived. Numerical results verify the analysis and show that employing the existing SBSs is a promising approach to enhancing security while satisfying the user demands of the small cells.

Spatial spectrum reuse significantly enhances spectrum utilization but requires delicate design to avoid co-channel interference. Instead of focusing solely on spectrum efficiency, we consider maximizing social welfare via on-demand channel allocation in this paper. We design a spectrum reuse mechanism for non-symmetric networks, in which the optimal channel allocation that maximizes social welfare is the result of an appropriate bidding method of secondary users (SUs) in the constrained Vickrey-Clarke-Groves (VCG) auction. To simplify the constrained VCG auction, we group the SUs into interference-free maximal independent groups (MIGs) using a modified Bron-Kerbosch algorithm. We introduce the VCG auction for MIGs, in which truthful bidding is the optimal strategy for the MIGs. We build a decision process such that the MIGs as representatives of the SUs can update their channel evaluations in each step and submit truthful bids. Furthermore, we approximate and simplify the optimal channel allocation with a greedy algorithm and Dijkstra's algorithm. In our simulation, we compare the proposed methods and demonstrate that our on- demand channel allocation increases social welfare.

Variations of deep neural networks such as convolutional neural network (CNN) have been successfully applied to image denoising. The goal is to automatically learn a mapping from a noisy image to a clean image given training data consisting of pairs of noisy and clean images. Most existing CNN models for image denoising have many layers. In such cases, the models involve a large amount of parameters and are computationally expensive to train. In this paper, we develop a dilated residual CNN for Gaussian image denoising. Compared with the recently proposed residual denoiser, our method can achieve comparable performance with less computational cost. Specifically, we enlarge receptive field by adopting dilated convolution in residual network, and the dilation factor is set to a certain value. We utilize appropriate zero padding to make the dimension of the output the same as the input. It has been proven that the expansion of receptive field can boost the CNN performance in image classification, and we further demonstrate that it can also lead to competitive performance for denoising problem. Moreover, we present a formula to calculate receptive field size when dilated convolution is incorporated. Thus, the change of receptive field can be interpreted mathematically. To validate the efficacy of our approach, we conduct extensive experiments for both gray and color image denoising with specific or randomized noise levels. Both of the quantitative measurements and the visual results of denoising are promising comparing with state-of-the-art baselines.

With the growing number of wireless and mobile devices ingrained into our daily lives, more and more people are interacting with online services that adopt recommender systems to suggest movies, news and points of interest. The private demographics of users such as age and gender in online recommender systems are very useful for many applications such as personalized ads, social study and marketing. However, users do not always provide details in their online profiles due to privacy concern. Most existing approaches can infer user private attributes based on sufficient interaction history but could fail for new users with few ratings. In this paper, we present a novel preference elicitation method, with which a recommender system asks cold-start users to rate selected items adaptively and infer the demographics rapidly via a few interactions. Specifically, latent user profiles are learned across the tasks of demographic inference and rating prediction simultaneously, which enables knowledge transfer through the two related tasks and improves the prediction accuracy for both tasks. The proposed method can also facilitate the understanding of the tradeoff between user privacy and the utility of personalization. Experimental results on real-world datasets demonstrate the performance of the proposed method in terms of the accuracy of both demographics inference and rating prediction.

The Domain Name System (DNS), which does not encrypt domain names such as "bank.us" and "dentalcare.com", commonly accurately reflects the specific network services. Therefore, DNS-based behavioral analysis is extremely attractive for many applications such as forensics investigation and online advertisement. Traditionally, a user can be trivially and uniquely identified by the device's IP address if it is static (i.e., a desktop or a laptop). As more and more wireless and mobile devices are deeply ingrained in our lives and the dynamic IP address such as DHCP has been widely applied, it becomes almost impossible to use one IP address to identify a unique user. In this paper, we propose a new tracking method to identify individual users by the way they query DNS regardless of dynamic changing IP addresses and various types of devices. The method is applicable based on two observations. First, even though users may update IP addresses dynamically during different sessions, their query patterns can be stable across these sessions. Secondly, domain name look ups in sessions are different from users to users according to their personal behaviors. Specifically, we propose the constrained Dirichlet multinomial mixture (CDMM) clustering model to cluster DNS queries of different sessions into groups, each of which is considered being generated by a unique user. Compared with traditional supervised and unsupervised models, our model does not acquire any labeled user information that is very hard to obtain in real networks or the specification of the number of clusters, and meanwhile enforces the maximum number of session data in each cluster, which fits the DNS tracking problem nicely. Experimental results on DNS queries collected from real networks demonstrate that our method accomplishes a high clustering accuracy and outperforms the existing methods.

In the competitive environment of the internet, retaining and growing one's user base is of major concern to most web services. Furthermore, the economic model of many web services is allowing free access to most content, and generating revenue through advertising. This unique model requires securing user time on a site rather than the purchase of good which makes it crucially important to create new kinds of metrics and solutions for growth and retention efforts for web services. In this work, we address this problem by proposing a new retention metric for web services by concentrating on the rate of user return. We further apply predictive analysis to the proposed retention metric on a service, as a means for characterizing lost customers. Finally, we set up a simple yet effective framework to evaluate a multitude of factors that contribute to user return. Specifically, we define the problem of return time prediction for free web services. Our solution is based on the Cox's proportional hazard model from survival analysis. The hazard based approach offers several benefits including the ability to work with censored data, to model the dynamics in user return rates, and to easily incorporate different types of covariates in the model. We compare the performance of our hazard based model in predicting the user return time and in categorizing users into buckets based on their predicted return time, against several baseline regression and classification methods and find the hazard based approach to be superior.

For cold-start recommendation, it is important to rapidly profile new users and generate a good initial set of recommendations through an interview process --- users should be queried adaptively in a sequential fashion, and multiple items should be offered for opinion solicitation at each trial. In this work, we propose a novel algorithm that learns to conduct the interview process guided by a decision tree with multiple questions at each split. The splits, represented as sparse weight vectors, are learned through an L_1-constrained optimization framework. The users are directed to child nodes according to the inner product of their responses and the corresponding weight vector. More importantly, to account for the variety of responses coming to a node, a linear regressor is learned within each node using all the previously obtained answers as input to predict item ratings. A user study, preliminary but first in its kind in cold-start recommendation, is conducted to explore the efficient number and format of questions being asked in a recommendation survey to minimize user cognitive efforts. Quantitative experimental validations also show that the proposed algorithm outperforms state-of-the-art approaches in terms of both the prediction accuracy and user cognitive efforts.

Recent approaches to collaborative filtering have concentrated on estimating an algebraic or statistical model, and using the model for predicting missing ratings. In this paper we observe that different models have relative advantages in different regions of the input space. This motivates our approach of using stagewise linear combinations of collaborative filtering algorithms, with non-constant combination coefficients based on kernel smoothing. The resulting stagewise model is computationally scalable and outperforms a wide selection of state-of-the-art collaborative filtering algorithms.

We introduce a new dissimilarity function for ranked lists, the expected weighted Hoeffding distance, that has several advantages over current dissimilarity measures for ranked search results. First, it is easily customized for users who pay varying degrees of attention to websites at different ranks. Second, unlike existing measures such as generalized Kendall's tau, it is based on a true metric, preserving meaningful embeddings when visualization techniques like multi-dimensional scaling are applied. Third, our measure can effectively handle partial or missing rank information while retaining a probabilistic interpretation. Finally, the measure can be made computationally tractable and we give a highly efficient algorithm for computing it. We then apply our new metric with multi-dimensional scaling to visualize and explore relationships between the result sets from different search engines, showing how the weighted Hoeffding distance can distinguish important differences in search engine behavior that are not apparent with other rank-distance metrics. Such visualizations are highly effective at summarizing and analyzing insights on which search engines to use, what search strategies users can employ, and how search results evolve over time. We demonstrate our techniques using a collection of popular search engines, a representative set of queries, and frequently used query manipulation methods.

In consumer video conferencing, lighting conditions are usually not ideal thus the image qualities are poor. Lighting affects image quality on two aspects: brightness and skin tone. While there has been much research on improving the brightness of the captured images including contrast enhancement and noise removal (which can be thought of as components for brightness improvement), little attention has been paid to the skin tone aspect. In contrast, it is a common knowledge for professional stage lighting designers that lighting affects not only the brightness but also the color tone which plays a critical role in the perceived look of the host and the mood of the stage scene. Inspired by stage lighting design, we propose an active lighting system which automatically adjusts the lighting so that the image looks visually appealing. The system consists of computer controllable light emitting diode light sources of different colors so that it improves not only the brightness but also the skin tone of the face. Given that there is no quantitative formula on what makes a good skin tone, we use a data driven approach to learn a good skin tone model from a collection of photographs taken by professional photographers. We have developed a working system and conducted user studies to validate our approach.

This paper presents a framework to restore the 2D content printed on documents in the presence of geometric distortion and non-uniform illumination. Compared with textbased document imaging approaches that correct distortion to a level necessary to obtain sufficiently readable text or to facilitate optical character recognition (OCR), our work targets nontextual documents where the original printed content is desired. To achieve this goal, our framework acquires a 3D scan of the document's surface together with a high-resolution image. Conformal mapping is used to rectify geometric distortion by mapping the 3D surface back to a plane while minimizing angular distortion. This conformal "deskewing" assumes no parametric model of the document's surface and is suitable for arbitrary distortions. Illumination correction is performed by using the 3D shape to distinguish content gradient edges from illumination gradient edges in the high-resolution image. Integration is performed using only the content edges to obtain a reflectance image with significantly less illumination artifacts. This approach makes no assumptions about light sources and their positions. The results from the geometric and photometric correction are combined to produce the final output.

In this paper, we propose a novel framework called space-time light field rendering, which allows continuous exploration of a dynamic scene in both space and time. Compared to existing light field capture/rendering systems, it offers the capability of using unsynchronized video inputs and the added freedom of controlling the visualization in the temporal domain, such as smooth slow motion and temporal integration. In order to synthesize novel views from any viewpoint at any time instant, we develop a two-stage rendering algorithm. We first interpolate in the temporal domain to generate globally synchronized images using a robust spatial-temporal image registration algorithm followed by edge-preserving image morphing. We then interpolate these software-synchronized images in the spatial domain to synthesize the final view. In addition, we introduce a very accurate and robust algorithm to estimate subframe temporal offsets among input video sequences. Experimental results from unsynchronized videos with or without time stamps show that our approach is capable of maintaining photorealistic quality from a variety of real scenes.

We present a system to restore the 2D content printed on distorted documents. Our system works by acquiring a 3D scan of the document's surface together with a high-resolution image. Using the 3D surface information and the 2D image, we can ameliorate unwanted surface distortion and effects from non-uniform illumination. Our system can process arbitrary geometric distortions, not requiring any pre-assumed parametric models for the document's geometry. The illumination correction uses the 3D shape to distinguish content edges from illumination edges to recover the 2D content's reflectance image while making no assumptions about light sources and their positions. Results are shown for real objects, demonstrating a complete framework capable of restoring geometric and photometric artifacts on distorted documents