Zhuoshi Pan, Qizhi Pei, Yu Li, Zinan Tang, QiYao Sun, H. Vicky Zhao, Conghui He, Lijun Wu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Aligning Large Language Models (LLMs) with investor decision-making processes under herd behavior is a critical challenge in behavioral finance, which grapples with a fundamental limitation: the scarcity of real-user data needed for Supervised Fine-Tuning (SFT). While SFT can bridge the gap between LLM outputs and human behavioral patterns, its reliance on massive authentic data imposes substantial collection costs and privacy risks. We propose InvestAlign, a novel framework that constructs high-quality SFT datasets by leveraging theoretical solutions to similar and simple optimal investment problems rather than the complex scenarios. Our theoretical analysis demonstrates that training LLMs with InvestAlign-generated data achieves faster parameter convergence than using real-user data, suggesting superior learning efficiency. Furthermore, we develop InvestAgent, an LLM agent fine-tuned with InvestAlign, which shows significantly closer alignment to real-user data than pre-SFT models in both simple and complex investment problems. This highlights our proposed InvestAlign as a promising approach with the potential to address complex optimal investment problems and align LLMs with investor decision-making processes under herd behavior. Our code is publicly available at https://github.com/thu-social-network-research-group/InvestAlign.

To deliver coherent and personalized experiences in long-term conversations, existing approaches typically perform retrieval augmented response generation by constructing memory banks from conversation history at either the turn-level, session-level, or through summarization techniques. In this paper, we explore the impact of different memory granularities and present two key findings: (1) Both turn-level and session-level memory units are suboptimal, affecting not only the quality of final responses, but also the accuracy of the retrieval process. (2) The redundancy in natural language introduces noise, hindering precise retrieval. We demonstrate that *LLMLingua-2*, originally designed for prompt compression to accelerate LLM inference, can serve as an effective denoising method to enhance memory retrieval accuracy. Building on these insights, we propose **SeCom**, a method that constructs a memory bank with topical segments by introducing a conversation **Se**gmentation model, while performing memory retrieval based on **Com**pressed memory units. Experimental results show that **SeCom** outperforms turn-level, session-level, and several summarization-based methods on long-term conversation benchmarks such as *LOCOMO* and *Long-MT-Bench+*. Additionally, the proposed conversation segmentation method demonstrates superior performance on dialogue segmentation datasets such as *DialSeg711*, *TIAGE*, and *SuperDialSeg*.

In complex social environments, influential entities—such as governments, corporations, and public figures—face growing challenges in effectively managing both their reputation and influence during public engagement activities such as governance, service delivery, and crisis communication. In these interactions, the public forms perceptions and responds behaviorally, which can be reflected in two dynamic and interdependent attributes: reputation and influence. They are jointly shaped by the entity’s actual performance over time, such as the provision of products or services. Prior research has extensively examined the management of reputation and influence as separate issues, and often neglect their interactions. This study analyzes strategies based on the influential entity’s actual performance, accounting for the mutual interdependence between reputation and influence. Based on the reputation-influence co-evolution model, we propose a framework for determining the minimum time-invariant performance required to meet predefined thresholds for reputation and influence. We obtain the optimal strategy to satisfy both requirements. Simulation results on both real-world data and synthetic networks validate the proposed model and our theoretical analysis.

Graph Anomaly Detection (GAD) aims to identify uncommon, deviated, or suspicious objects within graph-structured data. Existing methods generally focus on a single graph object type (node, edge, graph, etc.) and often overlook the inherent connections among different object types of graph anomalies. For instance, a money laundering transaction might involve an abnormal account and the broader community it interacts with. To address this, we present UniGAD, the first unified framework for detecting anomalies at node, edge, and graph levels jointly. Specifically, we develop the Maximum Rayleigh Quotient Subgraph Sampler (MRQSampler) that unifies multi-level formats by transferring objects at each level into graph-level tasks on subgraphs. We theoretically prove that MRQSampler maximizes the accumulated spectral energy of subgraphs (i.e., the Rayleigh quotient) to preserve the most significant anomaly information. To further unify multi-level training, we introduce a novel GraphStitch Network to integrate information across different levels, adjust the amount of sharing required at each level, and harmonize conflicting training goals. Comprehensive experiments show that UniGAD outperforms both existing GAD methods specialized for a single task and graph prompt-based approaches for multiple tasks, while also providing robust zero-shot task transferability.

While state-of-the-art diffusion models (DMs) excel in image generation, concerns regarding their security persist. Earlier research highlighted DMs' vulnerability to data poisoning attacks, but these studies placed stricter requirements than conventional methods like 'BadNets' in image classification. This is because the art necessitates modifications to the diffusion training and sampling procedures. Unlike the prior work, we investigate whether BadNets-like data poisoning methods can directly degrade the generation by DMs. In other words, if only the training dataset is contaminated (without manipulating the diffusion process), how will this affect the performance of learned DMs? In this setting, we uncover bilateral data poisoning effects that not only serve an adversarial purpose (compromising the functionality of DMs) but also offer a defensive advantage (which can be leveraged for defense in classification tasks against poisoning attacks). We show that a BadNets-like data poisoning attack remains effective in DMs for producing incorrect images (misaligned with the intended text conditions). Meanwhile, poisoned DMs exhibit an increased ratio of triggers, a phenomenon we refer to as 'trigger amplification', among the generated images. This insight can be then used to enhance the detection of poisoned training data. In addition, even under a low poisoning ratio, studying the poisoning effects of DMs is also valuable for designing robust image classifiers against such attacks. Last but not least, we establish a meaningful linkage between data poisoning and the phenomenon of data replications by exploring DMs' inherent data memorization tendencies. Code is available at https://github.com/OPTML-Group/BiBadDiff.

In online social networks, millions of connected intelligent individuals actively interact with each other, which not only facilitates opinion sharing but also offers the platform to spread detrimental gossips and rumors. Therefore, it is of crucial importance to better understand how the avalanche of information propagates over social networks and affects our social life and economy. However, most model-based works on information diffusion either consider the spreading of one single message or assume that different information spreads independently. In this letter, we investigate how correlated information spreads together and jointly influences users' decisions from a graphical evolutionary game perspective. We model the multi-source information diffusion process, analyze the impact of information's correlation and time delay on the evolutionary dynamics and the evolutionary stable states (ESS). Simulation results on synthetic networks and Facebook real-world networks are consistent with our analytical results. This investigation offers important insights to the understanding and management of multi-source information diffusion.

The reliability and security of distributed detection systems have become increasingly important due to their growing prevalence in various applications. As advancements in human-machine systems continue, human factors, such as herding behaviors, are becoming influential in decision fusion process of these systems. The presence of malicious users further highlights the necessity to mitigate security concerns. In this paper, we propose a maximum entropy attack exploring the herding behaviors of users to amplify the hazard of attackers. Different from prior works that try to maximize the fusion error rate, the proposed attack maximizes the entropy of inferred system states from the fusion center, making the fusion results the same as a random coin toss. Moreover, we design static and dynamic attack modes to maximize the entropy of fusion results at the steady state and during the dynamic evolution stage, respectively. Simulation results show that the proposed attack strategy can cause the fusion accuracy to hover around 50% and existing fusion rules cannot resist our proposed attack, demonstrating its effectiveness.

In major public health events such as the outbreak of COVID-19, it is crucial to design effective and efficient quarantine policies to minimize both the epidemic spread as well as the economic loss. Most prior works in the literature either only consider the epidemic control while ignoring the economic loss or just tell the percentage of users to quarantine without explicitly pointing out whom to quarantine. In addition, many assume that we have perfect knowledge of each user's health states at all times, and they are not equipped to handle super-large networks with more than 100 000 users. In this work, we consider those pandemics with incubation periods and assume that we are unable to know the exact health states of all users. We propose a method to estimate each user's probability of being infectious and then propose the basic minimum-loss quarantine (basic-MinLoQ) algorithm to jointly minimize the epidemic spread, the economic loss as well as the number of people in quarantine. For super-large networks with more than 100 000 nodes, we propose the subgraph minimum-loss quarantine (subgraph-MinLoQ) algorithm to reduce the computation complexity. Simulation results show that our basic-MinLoQ algorithm outperforms several prior works, including Netshield, Acquaintance, and GreedyDrop, in terms of epidemic control and reducing economic loss. While our proposed subgraph-MinLoQ algorithm can efficiently identify people to quarantine and control the epidemic spread in super-large networks with more than100 000 users.

Viral information like rumors or fake news is spread over a communication network like a virus infection in a unidirectional manner: entity$i$conveys information to a neighbor$j$, resulting in two equally informed (infected) parties. Existing graph diffusion processes focus only on bidirectional diffusion on an undirected graph. Instead, leveraging recent research in graph signal processing (GSP), we propose a new directed acyclic graph (DAG) diffusion process to estimate the probability$x_{i}(t)$of node$i$'s infection at time$t$given an initial infected source node$s$, where$x_{i}(\infty)=1$. Specifically, given an undirected positive graph modeling node-to-node communication, we first estimate its graph embedding: a latent coordinate for each graph node in an assumed low-dimensional manifold space via extreme eigenvectors computed using LOBPCG. Next, we construct a DAG based on Euclidean distances between latent coordinates. Spectrally, we prove that the asymmetric DAG Laplacian matrix contains real non-negative eigenvalues, and that the DAG diffusion converges to the all-infection vector$\mathbf{x}(\infty)=1$as$t\rightarrow\infty$. Simulations show that our DAG diffusion process accurately estimates the probabilities of node infection over a variety of graph structures at different time instants.

Graph sampling is the problem of choosing a node subset via sampling matrix H ∈ {0, 1}K×Nto collect samples y = Hx ∈ℝK, KNcan be reconstructed in high fidelity. While sampling on undirected graphs is well studied, we propose the first eigen-decomposition-free sampling scheme tailored specifically for directed graphs, leveraging a previous undirected graph sampling method based on Gershgorin disc alignment (GDAS). Concretely, given a directed positive graph ${{\mathcal{G}}^d}$ specified by random-walk graph Laplacian matrix Lrw, we first define reconstruction of a smooth signal x∗from samples y using graph shift variation (GSV) $\left\| {{{\mathbf{L}}_{rw}}{\mathbf{x}}} \right\|_2^2$ as a signal prior. To minimize the worst-case reconstruction error of the linear system solution x∗= C−1H⊤y with symmetric coefficient matrix${\mathbf{C}} = {{\mathbf{H}}^ \top }{\mathbf{H}} + \mu {\mathbf{L}}_{rw}^ \top {{\mathbf{L}}_{rw}}$, the E-optimality sampling objective is to choose H to maximize the smallest eigenvalue λmin(C) of C. To circumvent eigen-decomposition, we maximize instead a lower bound $\lambda _{\min }^ - \left( {{\mathbf{SC}}{{\mathbf{S}}^{ - 1}}} \right)$ of λmin(C)—smallest Gershgorin disc left-end of a similarity transform of C—via a variant of GDAS based on Gershgorin circle theorem (GCT). Experimental results show that our sampling method yielded smaller signal reconstruction errors at a faster speed compared to competing schemes.

Online social networks connect people together and facilitate them to share their experiences and thoughts, while they also enable users with extreme opinions to further publicize their opinions and influence others. Thus, it is critical to study users' opinion formation process and to design effective mechanisms to control the opinion dynamics in social networks. In this work, we consider the scenario where users with extreme opinions may change the network structure, e.g., by adding new links or tuning the edge weights, to further spread their opinions to the public, and theoretically analyze its impact on the network opinions at equilibrium. We also propose a robust opinion control scheme that can reduce the influence of such network structure perturbation on network opinions. Simulations verify the correctness of our analysis and the effectiveness of our proposed opinion control algorithm.

Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers’ visual displays. To efficiently browse high-dimensional media, a navigational streaming model is considered: a client navigates the media space by dictating a navigation path to a server, who in response transmits the corresponding pre-encoded media data units (MDU) to the client one-by-one in sequence. Assuming that the MDU quality is pre-chosen and fixed, the problem resides in selecting and storing redundant representations of MDUs at the server in order to best trade off storage and transmission costs, while enabling adequate user’s random access. We address this problem with a landmark-based MDU optimization framework. The media space is divided into neighborhoods, each containing one landmark (a chosen MDU). MDUs in a neighborhood use the associated landmark as a predictor for inter-coding. Thus, for any MDU transition within the same neighborhood, only one inter-coded MDU transmission is required when the landmark resides in the decoder buffer. It results in lower transmission cost and enables navigational random access. To optimize an MDU structure, we employ tree-structured vector quantizer (TSVQ) to first optimize landmark locations, then iteratively add P-MDUs as refinements using a fast branch-and-bound technique. Taking interactive LF images and viewport adaptive 360° images as illustrative applications, and I-, P- and previously proposed merge frames to intra- and inter-code MDUs, we show experimentally that landmarked MDU structures can noticeably reduce the expected transmission cost compared with MDU structures without landmarks.

The past decades witness the rise and proliferation of online reputation systems. These reputation systems are vulnerable to malicious attacks, and most recent studies have focused on how to better defend the system. This paper aims to analyze the optimal attacking strategy, especially when considering the "message-based persuasion" phenomenon where users’ ratings tend to be influenced by earlier ones. Based on a simple model of users’ herding behavior in reputation systems, we study how attackers can explore this phenomenon to attack the system more effectively, and quantitatively analyze the optimal attacking strategies. This investigation is critical to the design of defensive mechanisms, and to the protection of online reputation systems.

In social network, the existence of malicious users can create lots of detrimental consequences. To diminish their negative influences, it is necessary for rational users to identify and interact with each neighbor carefully to protect themselves from malicious ones. Therefore, it is crucial to establish a rule for users’ interaction in order to mitigate malicious users’ influences. In this paper, we propose a smart evolution model based on evolutionary game theory by introducing the reputation mechanism. The model takes into account both current reputation and instant incentives during users’ decision-making process. On the basis of whether users share reputation values with others, we introduce schemes without reciprocity principle and with the indirect reciprocity principle respectively. With the social norm and reputation updating policy, we theoretically analyze the evolutionary dynamics and corresponding ESSs by explicitly considering the effects of malicious users. Finally, simulations based on synthetic networks and real-world data are conducted to validate the effectiveness of the proposed smart evolution model.

In social networks, to better understand the avalanche of information flow over networks and to investigate its impact on economy and our social life, it is of crucial importance to model and analyze the information diffusion process. To address the existence of "super users" in social networks who have higher social status and potentially larger influence, we propose a graphical evolutionary game theoretic framework to investigate the impact of such super users and their strategy update rules on information propagation. We analyze the evolutionary dynamics and the stable states. Simulation results are consistent with our theoretical analysis, and demonstrate that strategy update rule is the critical factor that influences the stable states of the information diffusion process.

Modeling and analysis of information diffusion over networks is of crucial importance to better understand the avalanche of information flow over social networks and to investigate its impact on economy and our social life. Different from prior works that study rational behavior in information diffusion, we focus on "irrational users e.g., those who always intentionally forward fake news even when they know it contains false information. We extend the graphical evolutionary game model for information diffusion, and analyze the impact of such irrational behavior on information propagation. Our simulation results on synthetic networks are consistent with our analytical results, and they show that even a few irrational users can significantly increase the number of users who adopt the forwarding strategy.

This paper proposes PRIMA: Probabilistic Ranking with Inter-item competition and Multi-Attribute utility function, which ranks items based on their probabilities of being a user's best choice. This framework is particularly important in E-commerce applications for making recommendations, predicting sales, and developing pricing strategies. To achieve mathematical tractability, it uses the weight-based multi-attribute utility function to address the inter-attribute tradeoff, where the weight reflects a user's personal preference for each attribute. The proposed work updates the weight from a user's past transactions using the concept of marginal rate of substitution from microeconomics, addresses the interitem competition, and computes the items' probabilities of being a user's best choice. Real user test results show that the proposed framework achieves comparable ranking accuracy to the state-of-the-art work with significant improvements in model simplicity and mathematical tractability.

A depth image provides partial geometric information of a 3D scene, namely the shapes of physical objects as observed from a particular viewpoint. This information is important when synthesizing images of different virtual camera viewpoints via depth-image-based rendering (DIBR). It has been shown that depth images can be efficiently coded using contour-adaptive codecs that preserve edge sharpness, resulting in visually pleasing DIBR-synthesized images. However, contours are typically losslessly coded as side information, which is expensive if the object shapes are complex. In this paper, we pursue a new paradigm in depth image coding for color-plus-depth representation of a 3D scene: in a pre-processing step, we pro-actively simplify object shapes in a depth and color image pair to reduce depth coding cost, at a penalty of a slight increase in synthesized view distortion. Specifically, we first mathematically derive a distortion upper-bound proxy for 3DSwIM—a quality metric tailored for DIBR-synthesized images. This proxy reduces inter-dependency among pixel rows in a block to ease optimization. We then approximate object contours via a dynamic programming algorithm to optimally tradeoff coding the cost of contours using arithmetic edge coding with our proposed view synthesis distortion proxy. We modify the depth and color images according to the approximated object contours in an inter-view consistent manner. These are then coded, respectively, using a contour-adaptive image codec based on graph Fourier transform for edge preservation and High Efficiency Video Coding (HEVC) intra. Experimental results show that by maintaining sharp but simplified object contours during contour-adaptive coding, for the same visual quality of DIBR-synthesized virtual views, our proposal can reduce depth image coding rate by up to 22% in 3DSwIM and 42% in peak signal-to-noise ratio compared with alternative coding strategies, such as HEVC intra.

Group recommendation plays a significant role in today's social media systems, where users form social groups to receive multimedia content together and interact with each other, instead of consuming the online content individually. Limitations of traditional group recommendation approaches are as follows. First, they usually infer group members’ preferences by their historical behaviors, failing to captureinactiveusers’ preferences from the sparse historical data. Second, relationships between group members are not studied by these approaches, which fail to capture the inherent personality of members in a group. To address these issues, we propose a social-aware group recommendation framework that jointly utilizes both social relationships and social behaviors to not only infer a group's preference, but also model thetoleranceandaltruismcharacteristics of group members. Based on the observation that thefollowingrelationship in the online social network reflects common interests of users, we propose a group preference model based on external experts of group members. Furthermore, we model users’ tolerance (willingness to receive content not preferred) and altruism (willingness to receive content preferred by friends). Finally, based on the group preference model, we design recommendation algorithms for users under different social contexts. Experimental results demonstrate the effectiveness of our approach, which significantly improves the recommendation accuracy against traditional approaches, especially in the cases of inactive group members.

Network latency prediction is important for server selection and quality-of-service estimation in real-time applications on the Internet. Traditional network latency prediction schemes attempt to estimate the latencies between all pairs of nodes in a network based on sampled round-trip times, through either Euclidean embedding or matrix factorization. However, these schemes become less effective in terms of estimating the latencies of personal devices, due to unstable and time-varying network conditions, triangle inequality violation and the unknown ranks of latency matrices. In this paper, we propose a matrix completion approach to network latency estimation. Specifically, we propose a new class of low-rank matrix completion algorithms, which predicts the missing entries in an extracted “network feature matrix” by iteratively minimizing a weighted Schatten-p norm to approximate the rank. Simulations on true low-rank matrices show that our new algorithm achieves better and more robust performance than multiple state-of-the-art matrix completion algorithms in the presence of noise. We further enhance latency estimation based on multiple “frames” of latency matrices measured in the past, and extend the proposed matrix completion scheme to the case of 3-D tensor completion. Extensive performance evaluations driven by real-world latency measurements collected from the Seattle platform show that our proposed approaches significantly outperform various state-of-the-art network latency estimation techniques, especially for networks that contain personal devices.

The popularity of Online Social Networks (OSNs) has attracted great research interests in different fields. In Economics, researchers use game theory to analyze the mechanism of network formation, which is called Network Formation Game. While in Computer Science, much effort has been done in building machine learning models to predict future or missing links. However, there are few works considering how to combine game theoretic analysis and machine learning models. Therefore, in this paper, we study the problem of Exploiting Game Theoretic Analysis for Link Recommendation in Social Networks. Our goal is to improve link recommendation accuracy via leveraging the power of Network Formation Games into machine learning models. We present two different approaches to solve this problem. First, we propose a three- phase method that straightforwardly combines game theoretic analysis with machine learning models. Second, we develop a unified model, BPRLGT, that incorporates Network Formation Game into a Bayesian ranking framework for link recommendation. Specifically, BPRLGT takes advantage of network topology and we design a game theoretic sampling approach to improve its training process. The experiments are conducted on four real world datasets and the results on all datasets demonstrate that both our proposed three-phase method and the unified ranking model outperform the baseline methods.

With an increasing popularity of real-time applications, such as live chat and gaming, latency prediction between personal devices including mobile devices becomes an important problem. Traditional approaches recover all-pair latencies in a network from sampled measurements using either Euclidean embedding or matrix factorization. However, these approaches targeting static or mean network latency prediction are insufficient to predict personal device latencies, due to unstable and time-varying network conditions, triangle inequality violation and unknown rank of latency matrices. In this paper, by analyzing latency measurements from the Seattle platform, we propose new methods for both static latency estimation as well as the dynamic estimation problem given 3D latency matrices sampled over time. We propose a distance-feature decomposition algorithm that can decompose latency matrices into a distance component and a network feature component, and further leverage the structured pattern inherent in the 3D sampled data to increase estimation accuracy. Extensive evaluations driven by real-world traces show that our proposed approaches significantly outperform various state-of-the-art latency prediction techniques.

A depth image provides geometric information of a 3D scene, namely the shapes of physical objects captured from a particular viewpoint. This information is important for synthesizing images corresponding to different virtual camera viewpoints via depth-image-based rendering (DIBR). Since it has been shown that blurring of object contours in the depth images leads to bleeding artefacts in virtual images. The most effective way to compress depth images relies on edge-adaptive image codecs that preserve contours, which are losslessly coded as side information (SI). However, lossless coding of the exact object contours can be expensive. In this paper, we argue that the contours themselves can be suitably approximated to save bits, while the depth images piecewise smooth (PWS) characteristic stays preserved. Specifically, we first propose a metric that estimates contour coding rate based on edge statistics. Given an initial rate estimate, we then pro-actively approximate object contours in a way that guarantees rate reduction when coded using arithmetic edge coding (AEC) as SI. Given the sharp but approximated contours, we finally encode the image using an edge-adaptive image codec with graph Fourier transform (GFT) for edge preservation. We show in our experiments that by maintaining sharp but slightly inaccurate object contours, the resulting quality of virtual views synthesized via DIBR exceeds those synthesized using depth images compressed with edge-adaptive codecs that losslessly encode object contours as SI, in particular when the total coding rate budget is low. This confirms that optimized coding of depth images results in an effective tradeoff in the representation of contour and respective depth information.

In free viewpoint video, a viewer can choose at will any camera angle or the so-called “virtual view” to observe a dynamic 3-D scene, enhancing his/her depth perception. The virtual view is synthesized using texture and depth videos of two anchor camera views via depth-image-based rendering (DIBR). We consider, for the first time, collaborative live streaming of a free viewpoint video, where a group of users may interactively pull and cooperatively share streams of different anchor views. There is a cost to access the anchor views from the live source, a cost to “reconfigure” the peer network due to a change in selected anchors during view switching, and a distortion cost due to the distance of the virtual views to the received anchor views at users. We optimize the anchor views allocated to users so as to minimize the overall streaming cost given by the access cost, reconfiguration cost, and view distortion cost. We first show that, if the reconfiguration cost due to view switching is negligible, the view allocation problem can be optimally and efficiently solved in polynomial time using dynamic programming. For the case of non-negligible reconfiguration cost, the problem becomes NP-hard. We thus present a locally optimal and centralized algorithm inspired by Lloyd's algorithm used in non-uniform scalar quantization. We further propose a distributed algorithm with convergence guarantee, where each peer group independently makes merge-and-split decisions with a well-defined fairness criteria. Simulation results show that our algorithms achieve low streaming cost due to its excellent anchor view allocation.

This work proposes a novel probabilistic multi-attribute item ranking framework to estimate the probability of an item being a user's best choice and rank items accordingly. It uses indifference curve from microeconomics to model users' personal preference, and addresses the inter-attribute tradeoff and inter-item competition issues at the same time with little information loss. The proposed framework also considers the fact that a user can only compare a few items at the same time, and models the user's selection process as a two-step process, where the user first selects a few candidates, and then makes detailed comparison. Simulation results show that the proposed framework significantly outperforms existing multiattribute ranking algorithms in terms of ranking quality.

With the emerging social networks, images have become a major medium for emotion delivery in social events due to their infectious and vivid characteristics. Discovering the emotionally representative images can help people intuitively understand the emotional aspects of social events. Prior works focus on finding the most visually representative images for the target queries or social events. However, the emotionally representative image should not only visually relevant with the social event, but also has a strong emotional appeal among people. In this paper, we propose an emotionally representative image discovery framework by jointly considering textual, visual and social factors. In particular, we build a hybrid link graph for images of each social event, where the weight of each link is measured by textual emotion information, visual similarity and social similarity. Then we propose the Visual-Social-Textual Rank (VSTRank) algorithm to calculate the importance score for each image, so that the emotionally representative images can be discovered under the constraint of textual, visual and social representativeness. To evaluate the effectiveness of our approach, we conduct a series of experiments with 15 social events extracted from real social media dataset, and evaluate the proposed method with both quantitative criterions and user study.

In free viewpoint video, a user can pull texture and depth videos captured from two nearby reference viewpoints to synthesize his chosen intermediate virtual view for observation via depth-image-based rendering (DIBR). For users who are observing the same video at the same time but not necessarily from the same virtual viewpoint, they have incentive to pull the same reference views so that the streaming cost can be shared. On the other hand, in general distortion of a synthesized virtual view increases with its distance to the reference views, and so a user also has incentive to select reference views that tightly “sandwich” his chosen virtual view, minimizing distortion. In a previous work, reference view sharing strategies-ones that optimally trade off shared streaming costs with synthesized view distortions-were investigated for the case when users are first divided into groups, and each user group independently pulls two reference views and shares the resulting streaming cost. In this paper, we generalize the previous notion of user group, so that a user can simultaneously belong to two groups, and each group shares the streaming cost of a single view. We also aim to find a Nash Equilibrium (NE) solution of reference view selection, which is stable and from which no one has incentive to unilaterally deviate. Specifically, we first derive a lemma based on known properties of synthesized view distortion functions. We then design a search algorithm to find a NE solution, leveraging on the derived lemma to reduce search complexity. Experimental results show that the stable NE solution increases the overall cost only slightly when compared to the unstable optimal reference selection that gives the lowest overall cost. Further, a larger network will give a lower average cost for each user, and thus, users tend to join large networks for cooperation.

In this paper, we consider a multi-user single-relay wireless network, where the relay facilitates transmissions of the users' signals to the destination. We study the relay power allocation among the users, and use bargaining theory to model the negotiation among the users on relay power allocation. By assigning a bargaining power to each user to indicate its transmission priority, we propose an asymmetric Nash bargaining solution (NBS)-based relay power allocation scheme. We also propose a distributed implementation for this solution, where each user only requires its local channel state information (CSI). We analytically investigate the impact of the bargaining powers on the relay power allocation and show that via proper selection of the bargaining powers, the proposed power allocation can achieve a balance between the network sum-rate and the user fairness. Then we generalize the NBS-based power allocation and its distributed implementation to multi-user multi-relay networks. Simulation results are shown to compare the proposed power allocation with sum-rate-optimal power allocation and even power allocation. The impact of the bargaining powers on the power allocation is also demonstrated via simulations.

This paper considers a multi-user single-relay wireless network, where the relay gets paid for helping the users forward signals, and the users pay to receive the relay service. We study the relay power allocation and pricing problem, and model the interaction between the users and the relay as a two-level Stackelberg game. In this game, the relay, modeled as the service provider and the leader of the game, sets the relay price to maximize its revenue; while the users are modeled as customers and the follower who buy power from the relay. For the relay power allocation among users, we use a bargaining game model to achieve a fair allocation. Based on the proposed fair relay power allocation rule, we then analyze the optimal relay power price that maximizes the relay's revenue, and derive the analytical solution. Simulation shows that the proposed power allocation scheme achieves a higher network sum-rate than the even power allocation, and is fairer than the sum-rate-optimal allocation. We also show that the proposed pricing and power allocation solution is consistent with the laws of supply and demand.

In interactive multiview video streaming (IMVS), users can periodically select one out of many captured views available for observation as video is played back in time. In single-view video streaming, to reduce server's upload burden, cooperative strategies where peers share received packets of the same video have proven to be effective, and incentive mechanisms are designed to stimulate user cooperation. Exploiting user cooperation in high dimensional IMVS, however, is more challenging. First, small number of peers in a local area are likely watching different views among large number of views available, making it difficult for a peer to find partners of the exact same view to cooperate. Second, even if a peer can identify cooperative partners of the same view, they will soon be watching different views after independent view-switching. In this paper, we study the use of a multiview video frame structure for IMVS that facilitates cooperative view switching, where even if peers are observing different views, they can nonetheless help each other. To stimulate user cooperation, we model peers' interaction as an indirect reciprocity game. Using Markov decision process (MDP) as a formalism, each peer makes distributed decisions to maximize his aggregate utilities within his lifetime. Simulation results show that when the cost to help others is much smaller than the utility gained from others' help, users fully cooperate. As the cost-to-gain ratio increases, users tend to behave differently at different views: given peers can predict their future view navigation paths probabilistically, a peer likely to enter a view-switching path not requiring others' help will have less incentive to cooperate. When the cost-to-gain ratio is very large, no users will cooperate.

In this paper, we consider a multi-source single-relay wireless network, where the relay facilitates transmissions of the sources' signals to the destination. We study the relay power allocation among the sources, and use the bargaining theory to model the negotiation among the sources on fair allocation of the relay power. By assigning a bargaining power to each source to indicate its transmission priority, we propose an asymmetric Nash bargaining solution (NBS)-based relay power allocation scheme. The impact of the bargaining powers on the relay power allocation and network performance is analyzed. We show that the proposed scheme addresses the tradeoff between the sum-rate of the network and the fairness among the sources, and can be adapted to meet different requirements in different applications by proper selection of the bargaining powers.

Users in multimedia social networks actively interact with each other. It is crucial to study the complex user dynamics and analyze its impact on the performance of multimedia social networks. This paper uses multimedia fingerprinting as an example and studies user dynamics in colluder social networks. During collusion, a group of attackers collectively attack multimedia fingerprinting system and use multimedia content illegally. This paper analyzes the incentives of cooperation among attackers and investigates how colluders form their coalitions to maximize their payoffs. We present a game-theoretic framework to model the complex dynamics among colluders, analyze when attackers cooperate with each other, and investigate how a colluder selects his/her fellow attackers to maximize his/her own payoff. We analyze multiuser collusion in two scenarios: when all attackers receive fingerprinted copies of the same resolution, and when they have copies of different resolutions. The proposed framework considers both the colluders' risk of being detected by the digital rights enforcer and the reward received from illegal usage of multimedia content. Our analysis shows that in both scenarios, colluding with more attackers does not always increase an attacker's utility, and attackers may not always want to cooperate with each other. We first examine the necessary conditions for attackers to collude together, and study how they select the collusion parameters such that cooperation benefits all colluders. We then study how the number of colluders affects each attacker's utility, and investigate the optimum strategy that an attacker should use to select fellow attackers and to form a coalition in order to maximize his or her own payoff.

In wireless multicast systems, cooperative multicast has been shown to be effective in dealing with heterogeneous channel conditions and improving the system performance. However, this mechanism requires users' voluntary contributions, which cannot be guaranteed since users are selfish and care only about their own performance. To stimulate user cooperation, in this work, we model the interaction among users in the wireless multicast system as a multi-buyer multi-seller price-based game, where users pay to receive relay service and get paid if they forward packets to others. It is a Stackelberg game, and backward induction is used to find the perfect Nash Equilibrium. We formulate the buyers' game as an evolutionary game and derive the evolutionarily stable strategy. Our simulation results demonstrate the effectiveness of our proposed incentive mechanism.

Receivers of wireless video broadcast can suffer catastrophic decoding errors when experiencing heavy packet losses due to transmission channel fades. Cooperative repair schemes, exploiting the "uncorrelatedness" in wireless channels of peers physically located more than one transmission wavelength apart, call for neighboring peers listening to the same video stream to locally share received packets via a secondary network. Since the likelihood of the entire peer group suffering fades in statistically independent channels at the same time is very small, cooperative peers can collectively recover lost packets via local packet sharing with high probability. For interactive multiview video streaming (IMVS), where a client receives and watches only one periodically selected view out of N available, the packet recovery problem is more challenging, since the likelihood of a neighboring cooperative peer watching the same view as a channel-corrupted peer is now 1/N. To enable cooperative recovery even when neighboring peers are watching different but correlated video views, cleverly designed redundantly coded in formation (RCI) such as Distributed Source Coded (DSC) frames are inserted into streams of different views. On one hand, RCI in the video streams promotes cooperative repair among peers watching different views; on the other, it leaves fewer available bits for channel coding, given a fixed transmission budget, to combat channel noise. In this paper, using game theoretical analysis, we search for the optimal amount of RCI in the video streams to foster the right balance between cooperation among peers and leftover bits for channel coding to maximize decoding success. Experimental results show that expected video decoding probability can be increased noticeably compared to non-optimized resource allocation schemes.

Compressed sensing is a novel technology to acquire and reconstruct sparse signals below the Nyquist rate. It has great potential in image and video acquisition to explore data redundancy and to significantly reduce the number of collected data. In this paper, we explore the temporal redundancy in videos, and propose a block-based adaptive framework for compressed video sampling. To address independent movement of different regions in a video, the proposed framework classifies blocks into different types depending on their inter-frame correlation, and adjusts the sampling and reconstruction strategy accordingly. Our framework also considers the diverse texture complexity of different regions, and adaptively adjusts the number of measurements collected for each region. The proposed framework also includes a frame rate selection module that selects the maximum achievable frame rate from a list of candidate frame rates under the hardware sampling rate and the perceptual quality constraints. Our simulation results show that compared to traditional raster scan, the proposed framework can increase the frame rate by up to six times depending on the scene complexity and the video quality constraint. We also observe a 1.5-7.8 dB gain in the average peak signal-to-noise ratio of the reconstructed frames when compared with prior works on compressed video sensing.

Multimedia social network is a network infrastructure in which the social network users share multimedia contents with all different purposes. Analyzing user behavior in multimedia social networks helps design more secured and efficient multimedia and networking systems. Multimedia fingerprinting protects multimedia from illegal alterations and multiuser collusion is a cost-effective attack. The colluder social network is naturally formed during multiuser collusion with which colluders gain reward by redistributing the colluded multimedia contents. Since the colluders have conflicting interest, the maximal-payoff collusion for one colluder may not be the maximal-payoff collusion for others. Hence, before a collusion being successful, the colluders must bargain with each other to reach agreements. We first model the bargaining behavior among colluders as a noncooperative game and study four different bargaining solutions of this game. Moreover, the market value of the redistributed multimedia content is often time-sensitive. The earlier the colluded copy being released, the more the people are willing to pay for it. Thus, the colluders have to reach agreements on how to distribute reward and risk among themselves as soon as possible. This paper further incorporates this time-sensitiveness of the colluders' reward and studies the time-sensitive bargaining equilibrium. The study in this paper reveals the strategies that are optimal for the colluders; thus, all the colluders have no inventive to disagree. Such understanding reduces the possible types of collusion into a small finite set.

Users using the same video streaming service within a wireless network share the same limited backbone bandwidth to the Internet. These users are motivated to collaborate with each other to obtain better-quality service. The decisions and actions of users influence the performance of others, hence they form a social network. In a fully-distributed wireless network, malicious attack can cause much more damage than over the Internet since the unstable wireless channel allows hostile users to mimic as a non-malicious user. Therefore, a robust attack-resistant cooperation strategy for non-malicious user is needed to combat the challenges in wireless networks. In this paper, we incorporate trust modelling into the cooperation among users to increase attack-resistance. Simulation results show our robust cooperation strategies can against 40% more attackers.

In the emerging peer-to-peer (P2P) live streaming, users cooperate with each other to support efficient delivery of video over networks. Pollution attack is an effective attack against P2P live streaming, where attackers upload useless data to their peers, which may cause distrust among users. To resist pollution attacks and stimulate user cooperation in P2P live streaming, this paper proposes a joint pollution detection and attacker identification system, where polluted chunks are detected as early as possible and trust management is used to identify polluters. We analyze its performance and propose different schemes to address the tradeoff between pollution resistance and system overhead. Our simulation results show that the proposed system can effectively resist pollution attacks while minimizing the user's computation overhead.

For wireless multicast applications where a group of users subscribe to the same service and receive the same data, a promising solution to combat channel fading is to explore the cooperative diversity and let users help each other forward packets. This paper investigates a distributed cooperative multicast scheme that uses a maximal ratio combiner to enhance the received signal-to-noise ratio (SNR), and provides a thorough performance analysis. We derive a close-form formulation of the average outage probability, examine its asymptotic behavior in the high SNR regime, and investigate the optimal power allocation. Our analytical and simulation results show that cooperative multicast performs better in denser networks with more relays helping, and user cooperation can significantly reduce the outage probability, especially in the high SNR region.

Compressed sensing is a novel technology to acquire and reconstruct signals below the Nyquist rate, and has great potential in image and video acquisition to explore the data redundancy and to significantly reduce the number of sampled data. In this paper, we explore the temporal redundancy in videos, and propose a block-based adaptive framework for compressed video sampling. It addresses the independent movement of different regions in a video, classifies blocks into different types depending on their inter-frame correlation, and adjusts the sampling and reconstruction strategies accordingly. Our framework also considers the diverse texture complexity of different regions, and adaptively adjusts the number of measurements collected for each region based on their sparsity. Our simulation results show that the proposed framework reduces the number of sampled measurements by 52% to 80% while still satisfying the quality constraint on the reconstructed frames. Compared to prior works, our proposed scheme improves the quality of the reconstructed frames and achieves a 0.8dB to 5.4dB gain in the average PSNR.

Human behavior analysis in video sharing social networks is an emerging research area, which analyzes the behavior of users who share multimedia content and investigates the impact of human dynamics on video sharing systems. Users watching live streaming in the same wireless network share the same limited bandwidth of backbone connection to the Internet, thus, they might want to cooperate with each other to obtain better video quality. These users form a wireless live-streaming social network. Every user wishes to watch video with high quality while paying as little as possible cost to help others. This paper focuses on providing incentives for user cooperation. We propose a game-theoretic framework to model user behavior and to analyze the optimal strategies for user cooperation simulation in wireless live streaming. We first analyze the Pareto optimality and the time-sensitive bargaining equilibrium of the two-person game. We then extend the solution to the multiuser scenario. We also consider potential selfish users' cheating behavior and malicious users' attacking behavior and analyze the performance of the proposed strategies with the existence of cheating users and malicious attackers. Both our analytical and simulation results show that the proposed strategies can effectively stimulate user cooperation, achieve cheat free and attack resistance, and help provide reliable services for wireless live streaming applications.

The popularity of multimedia multicast/broadcast applications over wireless networks makes it critical to address the error-prone, heterogeneous and dynamically changing nature of wireless channels. A promising solution to combat channel fading is to explore the cooperative diversity in which users may help each other forward packets. This paper investigates cooperative multicast schemes that use a maximal ratio combiner to enhance the received signal-to-noise ratio (SNR), and provides a thorough performance analysis. Two relay selection schemes are considered: the distributed and the genie-aided cooperation schemes. We derive the closed-form formulation and the approximations of their average outage probabilities.We also analyze the optimal power allocation and relay location strategies, and show that allocating half of the total transmission power to the source minimizes the average outage probability. Our analysis and simulation results show that cooperative multicast gives better performance when more relays help forward signals. Cooperative multicast helps achieve diversity order 2, and user cooperation can significantly reduce the outage probability, especially in the high SNR region. Finally, we compare the two cooperation strategies, and show that distributed cooperative multicast is preferred since it achieves a lower outage probability without introducing extra overhead for control messages.

To protect multimedia content in live streaming applications, it is of critical importance to employ effective access control and key management schemes and to prevent unauthorized access. Many live streaming applications experience a large variation in the user number, high reconnection rate, and a large number of short sessions. Such frequent membership update may introduce significant rekey overhead and poses challenges to efficient key management. This paper considers applications that can tolerate a small amount of content leak, explores the unique characteristics of live broadcast streaming in membership dynamics, and investigates efficient key management mechanisms. Our proposed scheme uses the reconnection algorithm, batch rekeying and grouped user placement to reduce the communication and the computation cost associated with key update. Our analytical and simulation results show that the proposed scheme can reduce more than 50% of the rekey messages, and helps decrease the number of rekey messages that a user needs to decrypt by more than 90%.

Multimedia social network is a network infrastructure in which the social network members share multimedia contents with all different purposes. Analyzing user behavior in multimedia social networks help design more secured and efficient multimedia and networking systems. In this paper, we focus on the colluder social network in multimedia fingerprinting systems in which colluders gain reward by redistributing the colluded multimedia signal. However, the market value of the redistributed multimedia content is time sensitive. The earlier the colluded copy being released, the more the people who are willing to pay for it. Thus the colluders have to reach agreement on how to distribute reward and the probability being detected among themselves as soon as possible. This paper incorporates the time-sensitiveness of the colluders' reward, and models the dynamics among colluders as a noncooperative game, and studies the time-restricted bargaining equilibrium. We provide the solution to the equilibrium that all the colluders have no inventive to disagree in order to maximize their own payoff.

Multimedia social network analysis is an emerging research area, which analyzes the behavior of users who share multimedia content and investigates the impact of human dynamics on multimedia systems. Users watching live streaming in the same wireless network share the same backbone connection to the Internet, thus they might want to cooperate with each other to obtain better video quality. These users form a wireless live-streaming social network and every user wishes to watch video with as high as possible quality while paying as less as cost for cooperation. Thus full cooperation cannot be guaranteed and the cooperation strategy must give incentives to the users. This paper proposes a game-theoretic framework to model user behavior and designs incentive-based strategies to stimulate user cooperation in wireless live streaming. We analyze the Pareto optimality and time-restricted bargaining equilibrium of the game. We also take into consideration selfish users' cheating behavior and propose cheat-proof strategies. Both our analytical and simulation results show that the proposed strategies can effectively stimulate user cooperation, achieve cheat free and help provide reliable services.

Photorealistic images can now be created using advanced techniques in computer graphics (CG). Synthesized elements could easily be mistaken for photographic (real) images. Therefore we need to differentiate between CG and real images. In our work, we propose and develop a new framework based on an aggregate of existing features. Our framework has a classification accuracy of 90% when tested on the de facto standard Columbia dataset, which is 4% better than the best results obtained by other prominent methods in this area. We further show that using feature selection it is possible to reduce the feature dimension of our framework from 557 to 80 without a significant loss in performance (Lt 1%). We also investigate different approaches that attackers can use to fool the classification system, including creation of hybrid images and histogram manipulations. We then propose and develop filters to effectively detect such attacks, thereby limiting the effect of such attacks to our classification system.

In the emerging peer-to-peer (P2P) live streaming, users cooperate with each other to support efficient delivery of video over networks in live streaming applications. Pollution attack is an effective attack against P2P live streaming, where attackers upload bogus multimedia data to their peers. The polluted data can spread over the entire network, and cause severe quality degradation of the videos. To resist pollution attacks in P2P live streaming, this paper proposes a trust management system that identifies attackers and excludes them from further sharing of multimedia data. We investigate possible attacks against the trust management system and analyze the attack resistance of the proposed system. Our simulation results show that the proposed trust management system can efficiently detect attackers and stimulate user cooperation even under attacks. It helps users receive more clean data and improves the performance of P2P live streaming.

Recent development in multimedia processing and network technologies has facilitated the distribution and sharing of multimedia through networks, and increased the security demands of multimedia contents. Traditional image content protection schemes use extrinsic approaches, such as watermarking or fingerprinting. However, under many circumstances, extrinsic content protection is not possible. Therefore, there is great interest in developing forensic tools via intrinsic fingerprints to solve these problems. Source coding is a common step of natural image acquisition, so in this paper, we focus on the fundamental research on digital image source coder forensics via intrinsic fingerprints. First, we investigate the unique intrinsic fingerprint of many popular image source encoders, including transform-based coding (both discrete cosine transform and discrete wavelet transform based), subband coding, differential image coding, and also block processing as the traces of evidence. Based on the intrinsic fingerprint of image source encoders, we construct an image source coding forensic detector that identifies which source encoder is applied, what the coding parameters are along with confidence measures of the result. Our simulation results show that the proposed system provides trustworthy performance: for most test cases, the probability of detecting the correct source encoder is over 90%.

In multimedia social networks, there exists complicated dynamics among users who share and exchange multimedia content. Using multimedia fingerprinting as an example, this paper investigates the human behavior dynamics in the multimedia social networks with side information. Side information is the information other than the colluded multimedia content that can help increase the probability of detection. We study the impact of side information in multimedia fingerprinting and show that the statistical means of the detection statistics can help the fingerprint detector significantly improve the collusion resistance. We then investigate how to probe the side information and model the dynamics between the fingerprint detector and the colluders as a two-stage extensive game with perfect information. We model the colluder-detector behavior dynamics as a two-stage game and find the equilibrium of the colluder-detector game using backward induction and show that the min-max solution is a Nash equilibrium, which gives no incentive for everyone in the multimedia fingerprint social network to deviate. This paper demonstrates that the proposed side information can significantly help improve the system performance to almost the same as the optimal correlation-based detector. Such result opens up a new scope in the research of fingerprinting system that given any fingerprint code, leveraging side information can improve the collusion resistance. Also, we provide the solutions to how to reach optimal collusion strategy and the corresponding detection, thus lead to a better protection of the multimedia content.

Multimedia social networks have become an emerging research area, in which analysis and modeling of the behavior of users who share multimedia are of ample importance in understanding the impact of human dynamics on multimedia systems. In peer-to-peer live-streaming social networks, users cooperate with each other to provide a distributed, highly scalable and robust platform for live streaming applications. However, every user wishes to use as much bandwidth as possible to receive a high-quality video, while full cooperation cannot be guaranteed. This paper proposes a game-theoretic framework to model user behavior and designs incentive-based strategies to stimulate user cooperation in peer-to-peer live streaming. We first analyze the Nash equilibrium and the Pareto optimality of two-person game and then extend to multiuser case. We also take into consideration selfish users' cheating behavior and malicious users' attacking behavior. Both our analytical and simulation results show that the proposed strategies can effectively stimulate user cooperation, achieve cheat free, attack resistance and help to provide reliable services.

Multimedia social network analysis is an emerging research area, which analyzes the behavior of users who share multimedia content and investigates the impact of human dynamics on multimedia systems. In peer-to-peer live-streaming social networks, user cooperate with each other to provide a distributed, highly scalable and robust platform for live streaming applications. However, every user wishes to use as much bandwidth as possible to receive a high-quality video, and full cooperation cannot be guaranteed. This paper proposes a game-theoretic framework to model user behavior and designs incentive-based strategies to stimulate user cooperation in peer-to-peer live streaming. We analyze the Nash equilibrium and the Pareto optimality of the game. We also take into consideration selfish users' cheating behavior and propose cheat-proof strategies. Both our analytical and simulation results show that the proposed strategies can effectively stimulate user cooperation, achieve cheat free and help provide reliable services.

Multimedia social network analysis is a research area with growing importance, in which the social network members share multimedia contents with all different purposes and analyzing their behavior help design more secured and efficient multimedia and networking systems. In this paper, we focus on multimedia fingerprinting social network, in which multi-user collusion being a powerful attack, where a group of attackers collectively undermine the traitor tracing capability. During collusion, different colluders have different objective thus, the colluders form a social network and an how to achieve agreement on distributing the risk/profit among colluders and ensure fairness of the attack is a crucial question. This paper models the dynamics among colluders as a non-cooperative game, propose a general model of utility functions and study four different bargaining solutions of this game.

Multiuser collusion is an effective attack against traitor-tracing multimedia fingerprinting, where a group of attackers collectively mount attacks to reduce their risk of being detected. During collusion, each attacker wishes to maximize his or her own payoff. To resolve the conflict, colluders have to negotiate with each other and achieve fair collusion. An attacker also needs to decide with whom he or she wants to collude. Though colluding with more people helps further reduce the risk, it also makes an attacker share with more people the rewards from illegal usage of multimedia. This paper uses game theory to model the complex colluder dynamics and analyzes the tradeoff between the risk and the rewards. We study how the selection of fellow attackers affects each colluder's utility, and analyze the optimum strategies that maximize colluders' payoffs.

Digital fingerprinting is an emerging technology in media security to identify the source of illicit copies and trace traitors. Collusion is a powerful attack, in which a group of attacker collectively mount attacks against digital fingerprinting. In multimedia fingerprinting, there exists complex dynamics between the colluders and the fingerprint detector, who have conflicting objectives and influence each other's performance and decisions. This paper proposes a game-theoretic framework to formulate and analyze the colluder-detector dynamics, in an effort to understand its impact on the traitor tracing performance of multimedia fingerprints. We investigate how colluders adjusts the collusion attacks to minimize their risk under the fairness constraint; and study how the fingerprint detector adapts his/her detection strategy accordingly to improve the collusion resistance, which is shown to be the min-max solution.

In non-intrusive forensic analysis, we wish to find information and properties about a piece of data without any reference to the original data prior to processing. An important first step to forensic analysis is the detection and estimation of block processing. Most existing work in block measurement uses strong assumptions on the data related to the block size or the method of compression. In this paper, we propose a new method to estimate the block size in digital images in a blind manner for use in a forensic context. We make no assumptions on the block size or the nature of any previous processing. Our scheme can accurately estimate block sizes in images up to a PSNR of 42 dB where block artifacts are perceptually invisible. We also offer a measure of detection accuracy which correctly classifies an image as block-processed with a probability of 95.0% while keeping the probability of false alarm at 7.4%.

During multi-user collusion attacks against digital fingerprinting, an important issue that colluders have to address is to distribute the risk evenly among all colluders and achieve fairness of the attack. Although they might agree so, some selfish colluders may break their agreement and process their fingerprinted copies before collusion in order to further reduce their own risk. To protect their own interest, other colluders have to detect these selfish colluders and exclude them from multi-user collusion. This paper studies this problem of traitors within traitors. We propose an autonomous selfish colluder detection and identification algorithm, in which colluders help each other detect selfish behavior. We show that the proposed algorithm can correctly identify all selfish colluders without falsely accusing any others, even when a small group of selfish colluders collaborate with each other to change the detection results.

Multi-user collusion is an cost-effective attack against digital fingerprinting, in which a group of attackers collectively undermine the traitor tracing capability of digital fingerprints. However, during multi-user collusion, each colluder wishes to minimize his/her own risk and maximize his/her own profit, and different colluders have different objectives. Thus, an important issue during collusion is to agree on how to distribute the risk/profit among colluders and ensure fairness of the attack. To have a better understanding of the attackers' behavior during collusion to achieve fairness, this paper models the dynamics among colluders as a non-cooperative game. We then study the Pareto-optimal set, where no colluder can further increase his/her own payoff without decreasing others', and analyze the Nash bargaining solution of this game.

In this digital era, digital multimedia contents are often transmitted over networks without any protection. This raises serious security concerns since the receivers/subscribers do not know what processes have been applied to multimedia data, and neither do they know whether this copy comes from a trusted source. Therefore, it is critical to provide forensic tools to identify the history of operations applied to multimedia data. In this paper, we focus on the identification of source coding techniques applied to multimedia, and we investigate the forensic analysis of transform based coding (both DCT and DWT based), subband coding, and linear predictive coding. Using the intrinsic fingerprints as trace of evidences, we construct an image source coding forensic system that analyzes which source encoder is used to compress the image and provides confidence measurements. Our simulation results show that the proposed system provides trustworthy performance: the probability of detecting the correct source encoder is 0.82 when PSNR = 40 dB, and it can correctly identify the source encoder with probability 0.98 with PSNR = 20dB.

Digital fingerprinting uniquely labels each distributed copy with user's ID and provides a proactive means to track the distribution of multimedia. Multi-user collusion is a powerful attack against digital fingerprinting, in which a group of attackers collectively mount attacks to remove the embedded identification information. To resist such multi-user collusion and support multimedia forensics, this paper investigates the side information based multimedia fingerprinting. We explore techniques to utilize side information of collusion attacks during colluder identification process, and show that the means of the detection statistics at the detector's side can significantly improve the traitor tracing capability. We also investigate how the fingerprint detector can probe such side information from the colluded copy, and our simulation results show that the proposed scheme helps the fingerprint detector achieve the optimum detection performance.

A cost effective attack against multimedia forensics is the multiuser collusion attack, in which several attackers mount attacks collectively to remove traces of the identifying fingerprints and hinder traitor tracing. An important issue in collusion is to ensure that all colluders have the same probability of being detected. While they might agree so, some selfish colluders may wish to further lower their own risk of being caught. This paper investigates this problem of "traitors within traitors", in an effort to formulate the dynamics among attackers during collusion. We consider scalable multimedia forensic systems where users receive fingerprinted copies of different quality due to network and device heterogeneity, and explore the techniques that a selfish colluder can use to minimize his/her probability of being detected. Our results show that changing the resolution of their received copies before multi-user collusion can help selfish colluders further reduce their risk, especially when the colluded copy has high resolution.

During collusion attacks against multimedia forensics, an important issue that colluders need to address is the fairness of the attack, i.e., whether all colluders take the same risk of being detected. Although they might agree so, some selfish colluders may break away from their fair-collusion agreement and process their fingerprinted copies before collusion to further lower their risk. On the other hand, to protect their own interests, other attackers may wish to detect and prevent such selfish pre-collusion processing. It is important to study this problem of traitors within traitors, formulate the dynamics among colluders and build a complete model of multi-user collusion. This paper investigates techniques that attackers can use to detect and identify selfish colluders without revealing the secrecy of any fingerprinted copies. Our simulation results show that the proposed scheme accurately identifies all selfish colluders without falsely accusing any others.

Cooperative diversity exploits the broadcast nature of wireless channels by allowing users to relay information for each other so as to create multiple signal paths. This technique could be used together with joint source-channel bit rate allocation to transmit conversational multimedia traffic. This paper studies, following a methodology that avoids the use of high or low SNR approximations, the interaction between joint source-channel bit rate allocation with AF and DF cooperation. Furthermore, the effect of QoS constraints on performance is studied. In addition to the advantages of user-cooperation at medium and low source-destination channel signal-to-noise ratio, the results in this paper point at the fact that significant extra performance gains could be obtained by careful selection of the relay node. Guidelines for choosing the relay are also discussed.

Multimedia security systems involve many users with different objectives and users influence each other's performance. To have a better understanding of multimedia security systems and offer stronger protection of multimedia, behavior forensics formulate the dynamics among users and investigate how they interact with and respond to each other. This paper analyzes the behavior forensics in multimedia fingerprinting and formulates the dynamics among attackers during multi-user collusion. In particular, this paper focuses on how colluders achieve the fair play of collusion and guarantee that all attackers share the same risk (i.e., the probability of being detected). We first analyze how to distribute the risk evenly among colluders when they receive fingerprinted copies of scalable resolutions due to network and device heterogeneity. We show that generating a colluded copy of higher resolution puts more severe constraints on achieving fairness. We then analyze the effectiveness of fair collusion. Our results indicate that the attackers take a larger risk of being captured when the colluded copy has higher resolution, and they have to take this tradeoff into consideration during collusion. Finally, we analyze the collusion resistance of the scalable fingerprinting systems in various scenarios with different system requirements, and evaluate the maximum number of colluders that the fingerprinting systems can withstand

Multimedia security systems have many users with different objectives and they influence each other's performance and decisions. Behavior forensics analyzes how users with conflicting interests interact with and respond to each other. Such investigation enables a thorough understanding of multimedia security systems and helps the digital rights enforcer offer stronger protection of multimedia. This paper analyzes the dynamics among attackers during multiuser collusion. The colluders share not only the profit from the redistribution of multimedia but also the risk of being detected by the content owner, and an important issue in collusion is fairness of the attack (i.e., whether all attackers share the same risk) (e.g., whether they have the same probability of being detected). While they might agree so, some selfish colluders may break their fair-play agreement in order to further lower their risk. This paper investigates the problem of "traitors within traitors" in multimedia forensics, in an effort to formulate the dynamics among attackers and understand their behavior to minimize their own risk and protect their own interests. As the first work on the analysis of this colluder dynamics, this paper explores some possible strategies that a selfish colluder can use to minimize his or her probability of being caught. We show that processing his or her fingerprinted copy before multiuser collusion helps a selfish colluder further lower his or her risk, especially when the colluded copy has high resolution and good quality. This paper also investigates the optimal precollusion processing strategies for selfish colluders to minimize their risk under the quality constraints

Digital fingerprinting is an emerging technology to protect multimedia content from illegal redistribution, where each distributed copy is labeled with unique identification information. In video streaming, huge amount of data have to be transmitted to a large number of users under stringent latency constraints, so the bandwidth-efficient distribution of uniquely fingerprinted copies is crucial. This paper investigates the secure multicast of anticollusion fingerprinted video in streaming applications and analyzes their performance. We first propose a general fingerprint multicast scheme that can be used with most spread spectrum embedding-based multimedia fingerprinting systems. To further improve the bandwidth efficiency, we explore the special structure of the fingerprint design and propose a joint fingerprint design and distribution scheme. From our simulations, the two proposed schemes can reduce the bandwidth requirement by 48% to 87%, depending on the number of users, the characteristics of video sequences, and the network and computation constraints. We also show that under the constraint that all colluders have the same probability of detection, the embedded fingerprints in the two schemes have approximately the same collusion resistance. Finally, we propose a fingerprint drift compensation scheme to improve the quality of the reconstructed sequences at the decoder's side without introducing extra communication overhead.

Digital fingerprinting inserts identification information in the content to track the usage of digital data and protect content security. To trace traitors for multimedia over heterogeneous networks, this paper studies scalable multimedia fingerprinting systems in which users receive fingerprinted multimedia of different quality. We investigate the cost-effective multi-user collusion on fingerprinting systems and focus on fair collusion attacks in which colluders share the same risk of being captured. In this paper, we examine the fairness constraints on collusion when attackers receive copies of different quality and analyze the performance of scalable fingerprinting systems under fair collusion attacks.

Digital fingerprinting is an important tool in multimedia forensics to trace traitors and protect multimedia content after decryption. This paper addresses the enforcement of digital rights when distributing multimedia over heterogeneous networks and studies the scalable multimedia fingerprinting systems in which users receive copies of different quality. We investigate the traitor tracing capability of such scalable fingerprinting systems, in particular, the robustness of the embedded fingerprints against multi-user collusion attacks. Under the fairness constraints on collusion that all attackers share the same risk of being captured, we analyze the maximum number of colluders that the fingerprinting systems can withstand, and our results show that multimedia fingerprints can survive collusion attacks by a few dozen colluders.

In digital fingerprinting and multimedia forensic systems, it is possible that multiple adversaries mount attacks collectively and effectively to undermine the forensic system's traitor tracing capability. During this collusion attack, an important issue that the adversaries need to address is the fairness of attack and ensuring that all colluders share the same risk of being caught. This paper studies the dynamics among attackers in enforcing the fairness of collusion and investigates the problem of traitors within traitors, in which some selfish colluders wish to minimize their own risk while still profiting from collusion. We explore the strategies that these selfish colluders can use to further lower their probability of being detected and analyze their performance. We show that by processing their fingerprinted copies before multi-user collusion, the selfish colluders can further reduce their risk at the cost of quality degradation of their fingerprinted copies.

Digital fingerprinting is a method for protecting digital data in which fingerprints that are embedded in multimedia are capable of identifying unauthorized use of digital content. A powerful attack that can be employed to reduce this tracing capability is collusion, where several users combine their copies of the same content to attenuate/remove the original fingerprints. In this paper, we study the collusion resistance of a fingerprinting system employing Gaussian distributed fingerprints and orthogonal modulation. We introduce the maximum detector and the thresholding detector for colluder identification. We then analyze the collusion resistance of a system to the averaging collusion attack for the performance criteria represented by the probability of a false negative and the probability of a false positive. Lower and upper bounds for the maximum number of colluders K(max) are derived. We then show that the detectors are robust to different collusion attacks. We further study different sets of performance criteria, and our results indicate that attacks based on a few dozen independent copies can confound such a fingerprinting system. We also propose a likelihood-based approach to estimate the number of colluders. Finally, we demonstrate the performance for detecting colluders through experiments using real images.

Digital fingerprinting is a technology for tracing the distribution of multimedia content and protecting them from unauthorized redistribution. Unique identification information is embedded into each distributed copy of multimedia signal and serves as a digital fingerprint. Collusion attack is a cost-effective attack against digital fingerprinting, where colluders combine several copies with the same content but different fingerprints to remove or attenuate the original fingerprints. In this paper, we investigate the average collusion attack and several basic nonlinear collusions on independent Gaussian fingerprints, and study their effectiveness and the impact on the perceptual quality. With unbounded Gaussian fingerprints, perceivable distortion may exist in the fingerprinted copies as well as the copies after the collusion attacks. In order to remove this perceptual distortion, we introduce bounded Gaussian-like fingerprints and study their performance under collusion attacks. We also study several commonly used detection statistics and analyze their performance under collusion attacks. We further propose a preprocessing technique of the extracted fingerprints specifically for collusion scenarios to improve the detection performance.

In networked video applications, to protect the multimedia content after decryption, digital fingerprinting can be used to trace the illegal redistribution of multimedia by uniquely labelling each distributed copy. It is crucial to efficiently distribute the uniquely fingerprinted copies without disclosing the secrecy of the embedded fingerprints. This paper investigates the bandwidth efficient transmission of fingerprinted video. To reduce the communication cost, we explore the special structure of the fingerprint design and propose a joint fingerprint design and distribution scheme, where some fingerprinted coefficients that are shared by a subgroup of users are securely multicast to them. From the simulations, the proposed scheme reduces the bandwidth requirement by 61% to 87%, depending on the number of users and the characteristics of video sequences.

Digital fingerprinting embeds unique information in each distributed copy and can be used to protect multimedia from illegal redistribution. In video streaming applications, there are a large number of users and a huge amount of data to transmit. Thus, given a multimedia fingerprinting system with the required robustness, it is essential to distribute fingerprinted copies efficiently without revealing the embedded fingerprints. We propose a bandwidth efficient fingerprint multicast scheme that can be used with most spread spectrum embedding based multimedia fingerprinting systems, and analyze its bandwidth efficiency. We also propose a fingerprint drift compensation for the fingerprint multicast scheme to improve the quality of the reconstructed frames at the receiver's side without increasing the communication cost.

Digital fingerprinting is a means to offer protection to digital data by which fingerprints embedded in the multimedia are capable of identifying unauthorized use of digital content. A powerful attack that can be employed to reduce this tracing capability is collusion. We study the collusion resistance of a fingerprinting system employing Gaussian distributed fingerprints and orthogonal modulation. We propose a likelihood-based approach to estimate the number of colluders, and introduce the thresholding detector for colluder identification. We first analyze the collusion resistance of a system to the average attack by considering the probability of a false negative and the probability of a false positive when identifying colluders. Lower and upper bounds for the maximum number of colluders are derived. We then show that the detectors are robust to different attacks. We further study different sets of performance criteria.

Digital fingerprinting is a technology for tracing the distribution of multimedia content and protecting them from unauthorized redistribution. Collusion attack is a cost effective attack against digital fingerprinting where several copies with the same content but different fingerprints are combined to remove the original fingerprints. In this paper, we investigate average and nonlinear collusion attacks of independent Gaussian fingerprints and study both their effectiveness and the perceptual quality. We also propose the bounded Gaussian fingerprints to improve the perceptual quality of the fingerprinted copies. We further discuss the tradeoff between the robustness against collusion attacks and the perceptual quality of a fingerprinting system.

Digital fingerprinting is a means to offer protection to digital data by which fingerprints embedded in the multimedia are capable of identifying unauthorized use of digital content. A powerful attack that can be employed to reduce this tracing capability is collusion. In this paper, we study the collusion resistance of a fingerprinting system employing Gaussian distributed fingerprints and orthogonal modulation. We propose a likelihood-based approach to estimate the number of colluders, and introduce the thresholding detector for colluder identification. We first analyze the collusion resistance of a system to the average attack by considering the probability of a false negative and the probability of a false positive when identifying colluders. Lower and upper bounds for the maximum number of colluders K/sub max/ are derived. We then show that the detectors are robust to different attacks. We further study different sets of performance criteria.

Digital fingerprinting is a technology for tracing the distribution of multimedia content and protecting them from unauthorized redistribution. Collusion attack is a cost effective attack against digital fingerprinting where several copies with the same content but different fingerprints are combined to remove the original fingerprints. In this paper, we consider average attack and several nonlinear collusion attacks on independent Gaussian based fingerprints, and study the detection performance of several commonly used detection statistics in the literature under collusion attacks. Observing that these detection statistics are not specifically designed for collusion scenarios and do not take into account the characteristics of the newly generated fingerprints under collusion attacks, we propose pre-processing techniques to improve the detection performance of the detection statistics under collusion attacks.

Digital fingerprinting is a technology for tracing the distribution of multimedia content and protecting them from unauthorized redistribution. Collusion attack is a cost effective attack against digital fingerprinting where several copies with the same content but different fingerprints are combined to remove the original fingerprints. In this paper, we investigate average and nonlinear collusion attacks of independent Gaussian fingerprints and study both their effectiveness and the perceptual quality. We also propose the bounded Gaussian fingerprints to improve the perceptual quality of the fingerprinted copies. We further discuss the tradeoff between the robustness against collusion attacks and the perceptual quality of a fingerprinting system.