Jiale Liu, Victor Bursztyn, Lin Ai, Haoliang Wang, Sunav Choudhary, Saayan Mitra, Qingyun Wu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Recent advances in video-language models have enabled applications like video retrieval, captioning, and assembly. However, executing such multi-stage pipelines efficiently on mobile devices remains challenging due to redundant model loads and fragmented execution. We introduce Atom, an on-device system that restructures video-language pipelines for fast and efficient execution. Atom decomposes a billion-parameter model into reusable modules, such as the visual encoder and language decoder, and parallely reuses them across subtasks like captioning, reasoning, and indexing. This reuse-centric design eliminates repeated model loading and enables parallel execution, reducing end-to-end latency without sacrificing performance. On commodity smartphones, Atom achieves 27-33% faster execution compared to non-reuse baselines, with only marginal performance drop (≤2.3 Recall@l in retrieval, ≤1.5 CIDEr in captioning). These results position Atom as a practical, scalable approach for efficient video-language understanding on edge devices.

We explore how body shapes influence human motion synthesis, an aspect often overlooked in existing text-to-motion generation methods due to the ease of learning a homogenized, canonical body shape. However, this homogenization can distort the natural correlations between different body shapes and their motion dynamics. Our method addresses this gap by generating body-shape-aware human motions from natural language prompts. We utilize a finite scalar quantization-based variational autoencoder (FSQ-VAE) to quantize motion into discrete tokens and then leverage continuous body shape information to de-quantize these tokens back into continuous, detailed motion. Additionally, we harness the capabilities of a pretrained language model to predict both continuous shape parameters and motion tokens, facilitating the synthesis of text-aligned motions and decoding them into shape-aware motions. We evaluate our method quantitatively and qualitatively, and also conduct a comprehensive perceptual study to demonstrate its efficacy in generating shape-aware motions. Project URL: https://shape-move.github.io/.

We present SKALD, a multi-shot video assembly method that constructs coherent video sequences from candidate shots with minimal reliance on text. Central to our approach is the Learned Clip Assembly (LCA) score, a learning-based metric that measures temporal and semantic relationships between shots to quantify narrative coherence. We tackle the exponential complexity of combining multiple shots with an efficient beam-search algorithm guided by the LCA score. To train our model effectively with limited human annotations, we propose two tasks for the LCA encoder: Shot Coherence Learning, which uses contrastive learning to distinguish coherent and incoherent sequences, and Feature Regression, which converts these learned representations into a real-valued coherence score. We develop two variants: a base SKALD model that relies solely on visual coherence and SKALD-text, which integrates auxiliary text information when available. Experiments on the VSPD and our curated MSV3C datasets show that SKALD achieves an improvement of up to 48.6% in IoU and a 43% speedup over the state-of-the-art methods. A user study further validates our approach, with 45% of participants favoring SKALD-assembled videos, compared to 22% preferring text-based assembly methods.

A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control the spread of an epidemic. In such settings, the SUTVA assumption for randomized-controlled trials is violated due to network interference, or spill-over effects, as treatments to group A can potentially also affect the control group B. When the underlying social network is known exactly, prior works have demonstrated how to conduct A/B tests adequately to estimate the global average treatment effect (GATE). However, in practice, it is often impossible to obtain knowledge about the exact underlying network. In this paper, we present UNITE: a novel estimator that relax this assumption and can identify GATE while only relying on knowledge of the superset of neighbors for any subject in the graph. Through theoretical analysis and extensive experiments, we show that the proposed approach performs better in comparison to standard estimators.

Text-to-image generative models can generate high-quality humans, but realism is lost when generating hands. Common artifacts include irregular hand poses, shapes, incorrect numbers of fingers, and physically implausible finger orientations. To generate images with realistic hands, we propose a novel diffusion-based architecture called HanDiffuser that achieves realism by injecting hand embeddings in the generative process. HanDiffuser consists of two components: a Text-to-Hand-Params diffusion model to generate SMPL-Body and MANO-Hand parameters from input text prompts, and a Text-Guided Hand-Params-to-Image diffusion model to synthesize images by conditioning on the prompts and hand parameters generated by the previous component. We incorporate multiple aspects of hand representation, including 3D shapes and joint-level finger positions, orientations and articulations, for robust learning and reliable performance during inference. We conduct extensive quantitative and qualitative experiments and perform user studies to demonstrate the efficacy of our method in generating images with high-quality hands. Project page: https://supreethn.github.io/research/handiffuser/index.html

Large language models (LLMs) have demonstrated powerful decision-making and planning capabilities in solving complicated real-world problems. LLM-based autonomous agents can interact with diverse tools (e.g., functional APIs) and generate solution plans that execute a series of API function calls in a step-by-step manner. The multitude of candidate API function calls significantly expands the action space, amplifying the critical need for efficient action space navigation. However, existing methods either struggle with unidirectional exploration in expansive action spaces, trapped into a locally optimal solution, or suffer from exhaustively traversing all potential actions, causing inefficient navigation. To address these issues, we propose ToolChain*, an efficient tree search-based planning algorithm for LLM-based agents. It formulates the entire action space as a decision tree, where each node represents a possible API function call involved in a solution plan. By incorporating the A$^*$ search algorithm with task-specific cost function design, it efficiently prunes high-cost branches that may involve incorrect actions, identifying the most low-cost valid path as the solution. Extensive experiments on multiple tool-use and reasoning tasks demonstrate that ToolChain* efficiently balances exploration and exploitation within an expansive action space. It outperforms state-of-the-art baselines on planning and reasoning tasks by 3.1% and 3.5% on average while requiring 7.35x and 2.31x less time, respectively.

Brands use cookies and device identifiers to link different web visits to the same consumer. However, with increasing demands for privacy, these identifiers are about to be phased out, making identity fragmentation a permanent feature of the online world. Assessing treatment effects via randomized experiments (A/B testing) in such a scenario is challenging because identity fragmentation causes a) users to receive hybrid/mixed treatments, and b) hides the causal link between the historical treatments and the outcome. In this work, we address the problem of estimating treatment effects when a lack of identification leads to incomplete knowledge of historical treatments. This is a challenging problem which has not been addressed in literature yet. We develop a new method called DIET, which can adjust for users being exposed to mixed treatments without the entire history of treatments being available. Our method takes inspiration from the Cox model, and uses a proportional outcome approach under which we prove that one can obtain consistent estimates of treatment effects even under identity fragmentation. Our experiments, on one simulated and two real datasets, show that our method leads to up to 20% reduction in error and 25% reduction in bias over the naive estimate.

There is increasing attention toward search indexes based on dense vector representations as they can encode latent semantic features and can be generated efficiently, thanks to the advances in representation learning. Building an index on vectors was addressed by methods like locality-sensitive hashing (LSH), Inverted File Index (IVF), and near-neighbor graphs and perfected for high performance. However, the current near-neighbor (NN) indexes cannot be directly used in a real recommendation engine where both learned dense features and the constraints-attributes are used. The existing methods are cascaded index with vector search followed by a naive matching or inverted index for filtering on attribute tokens. This filtering on post-vector search matched sets often limits the control on output size and adds latency due to repeated calls to the NN engine. We aim to make a single-stage retrieval model that can retrieve in a single pass and control the output size without compromising on latency. An NN index amenable to this should have a very similar structure to the attribute token-based inverted index. Hence, we develop an efficient constraint search engine based on the high dimensional sparse embeddings of semantic features added with attribute tokens. With this, we get an Inverted index-based Constraint Near Neighbor search-ICONN, where we retrieve search results with 100% match on query attributes and close to its semantic features. We achieve a better latency vs recall10@10 tradeoff compared to the standard NN search followed by attribute filtering.

In this paper, we consider a heavy inner product identification problem, which generalizes the Light Bulb problem ([1]): Given two sets $A \subset\{-1,+1\}^{d}$ and $B \subset\{-1,+1\}^{d}$ with $|A|=|B|=n$, if there are exact k pairs whose inner product passes a certain threshold, i.e., $\{\left(a_{1}, b_{1}\right), \cdots,\left(a_{k}, b_{k}\right)\} \subset A \times B$ such that $\forall i \in[k],\left\langle a_{i}, b_{i}\right\rangle \geq \rho \cdot d$, for a threshold $\rho \in(0,1)$, the goal is to identify those k heavy inner products. We provide an algorithm that runs in $O(n^{2 \omega / 3+o(1)})$ time to find the k inner product pairs that surpass $\rho \cdot d$ threshold with high probability, where $\omega$ is the current matrix multiplication exponent. By solving this problem, our method speed up the training of neural networks with ReLU activation function.

Vanilla federated learning does not support learning in an online environment, learning a personalized model on each client, and learning in a decentralized setting. There are existing methods extending federated learning in each of the three aspects. However, some important applications on enterprise edge servers (e.g. online item recommendation at global scale) involve the three aspects at the same time. Therefore, we propose a new learning setting Decentralized Personalized Online Federated Learning that considers all the three aspects at the same time.In this new setting for learning, the first technical challenge is how to aggregate the shared model parameters from neighboring clients to obtain a personalized local model with good performance on each client. We propose to directly learn an aggregation by optimizing the performance of the local model with respect to the aggregation weights. This not only improves personalization of each local model but also helps the local model adapting to potential data shift by intelligently incorporating the right amount of information from its neighbors. The second challenge is how to select the neighbors for each client. We propose a peer selection method based on the learned aggregation weights enabling each client to select the most helpful neighbors and reduce communication cost at the same time. We verify the effectiveness and robustness of our proposed method on three real-world item recommendation datasets and one air quality prediction dataset.

In Federated Learning (FL), adaptive optimization is an effective approach to addressing the statistical heterogeneity issue but cannot adapt quickly to concept drifts. In this work, we propose a novel adaptive optimizer called Flash that simultaneously addresses both statistical heterogeneity and the concept drift issues. The fundamental insight is that a concept drift can be detected based on the magnitude of parameter updates that are required to fit the global model to each participating client's local data distribution. Flash uses a two-pronged approach that synergizes client-side early-stopping training to facilitate detection of concept drifts and the server-side drift-aware adaptive optimization to effectively adjust effective learning rate. We theoretically prove that Flash matches the convergence rate of state-of-the-art adaptive optimizers and further empirically evaluate the efficacy of Flash on a variety of FL benchmarks using different concept drift settings.

The growth of low-end hardware has led to a proliferation of machine learning-based services in edge applications. These applications gather contextual information about users and provide some services, such as personalized offers, through a machine learning (ML) model. A growing practice has been to deploy such ML models on the user's device to reduce latency, maintain user privacy, and minimize continuous reliance on a centralized source. However, deploying ML models on the user's edge device can leak proprietary information about the service provider. In this work, we investigate on-device ML models that are used to provide mobile services and demonstrate how simple attacks can leak proprietary information of the service provider. We show that different adversaries can easily exploit such models to maximize their profit and accomplish content theft. Motivated by the need to thwart such attacks, we present an end-to-end framework, SODA, for deploying and serving on edge devices while defending against adversarial usage. Our results demonstrate that SODA can detect adversarial usage with 89% accuracy in less than 50 queries with minimal impact on service performance, latency, and storage.

Consider two brands that want to jointly test alternate web experiences for their customers with an A/B test. Such collaborative tests are today enabled usingthird-party cookies, where each brand has information on the identity of visitors to another website, ensuring a consistent treatment experience. With the imminent elimination of third-party cookies, such A/B tests will become untenable. We propose a two-stage experimental design, where the two brands only need to agree on high-level aggregate parameters of the experiment to test the alternate experiences. Our design respects the privacy of customers. We propose an unbiased estimator of the Average Treatment Effect (ATE), and provide a way to use regression adjustment to improve this estimate. On real and simulated data, we show that the approach provides valid estimate of the ATE and is robust to the proportion of visitors overlapping across the brands. Our demonstration describes how a marketer can design such an experiment and analyze the results.

In sequential recommender system applications, it is important to develop models that can capture users' evolving interest over time to successfully recommend future items that they are likely to interact with. For users with long histories, typical models based on recurrent neural networks tend to forget important items in the distant past. Recent works have shown that storing a small sketch of past items can improve sequential recommendation tasks. However, these works all rely on static sketching policies, i.e., heuristics to select items to keep in the sketch, which are not necessarily optimal and cannot improve over time with more training data. In this paper, we propose a differentiable policy for sketching (DiPS), a framework that learns a data-driven sketching policy in an end-to-end manner together with the recommender system model to explicitly maximize recommendation quality in the future. We also propose an approximate estimator of the gradient for optimizing the sketching algorithm parameters that is computationally efficient. We verify the effectiveness of DiPS on real-world datasets under various practical settings and show that it requires up to 50% fewer sketch items to reach the same predictive quality than existing sketching policies.

Earlier recommendation techniques, such as Collaborative Filtering (CF), assume the users’ preferences do not change over time and strive to maximize the immediate reward. In recent studies, Reinforcement Learning (RL) has been used to make interactive recommendation systems that capture users’ preferences over time and maximize the long-term reward. However, these methods have two limitations. First, they assume that items are independently distributed and do not consider the relations between items. This assumption ignores the power of the relations between items for recommendation systems, as demonstrated by CF. RL-based methods rely primarily on users’ positive feedback to understand their preferences, and sampling is used to incorporate their negative feedback. In a practical setting, users’ negative feedback is just as crucial as their positive feedback for understanding their preferences. We present a novel Deep Reinforcement Learning (DRL) recommendation framework to address the limitations above. We specifically utilize the actor-critic paradigm, which considers the recommendation problem a sequential decision-making process to adapt to users’ behaviors and maximize the long-term reward. Motivated by the intuition that similar users like similar items, we extract the relations between items using CF and integrate it into our framework to boost overall performance. Instead of negative sampling, our proposed framework relies on all users’ positive and negative feedback to understand users’ preferences more accurately. Extensive experiments with our dataset and two public datasets demonstrate the effectiveness of our proposed framework.

Content personalization is one of the foundations of today’s digital marketing. Often the same image needs to be adapted for different design schemes for content that is created for different occasions, geographic locations or other aspects of the target population. We present a novel reinforcement learning (RL) based method for automatically stylizing images to complement the design scheme of media, e.g., interactive websites, apps, or posters. Our approach considers attributes related to the design of the media and adapts the style of the input image to match the context. We do so using a preferential reward system in the RL framework that learns a reward function using human feedback. We conducted several user studies to evaluate our approach and demonstrate that we are able to effectively adapt image styles to different design schemes. In user studies, images stylized through our approach were the most preferred variation across a majority of our experiments. Additionally, we also release a dataset consisting of perceptual associations of web context with the associated image style.

We describe a novel algorithm for an open-domain trending hashtag recommendation task using zero-shot hashtag prediction in an online learning paradigm. Our method utilizes joint representation learning of latent embeddings for features extracted from long-form videos and semantic embeddings of hashtags trending on social platforms. In particular, we apply graph convolutional networks to a link prediction task using videos and hashtags as nodes in a heterogeneous graph. Comparing it to the existing models for closely related tasks, we demonstrate state-of-the-art results in trending hashtag recommendations for videos. The architecture is designed to be modular in a plug-and-play fashion to enable quick and easy incorporation of the latest advances in natural language understanding and image and video processing, with a practical view to its implementation in a real-time, online setting.

We present BOhance, an efficient solution for optimizing digital content like images. Our approach enhances the standard and widely-used method for optimizing content, A/B testing, by using Bayesian Optimization. Our work effectively extends A/B testing in the continuous domain where A/B testing cannot efficiently test infinitely many variants. We test our approach on an image enhancement task where we use iterative human feedback on different variants of an image to arrive at the optimal variant. BOhance auto-generates candidate content variants to be tested based on the human feedback on prior variants. We demonstrate with user-studies conducted on Amazon Mechanical Turk that BOhance can be both time and cost-efficient; and a superior alternative to existing solutions. Furthermore, we conduct a Visual Turing Test to obtain human impressions on the optimum variants generated by BOhance. Our experiments show that given a human-enhanced image and an image generated by BOhance, 53% users think that the BOhance image was generated by a human expert.

Deep neural networks have been widely used in video analysis applications such as automatic metadata generation, action recognition, and video summarization. A fundamental module in the pipeline of such DNN-based applications is feature extraction. However, extracting features for videos is a major bottleneck since it is performed on every frame of each video sequentially. In addition, the long training time of these complex networks also hinders their usability. In this work, we identify fine-grained and coarse-grained parallelism techniques to speed up vital components in video analysis applications through inter-frame and intra-video parallelism. We demonstrate these techniques on the feature extraction and summarization modules. We leverage frame-level parallelism in feature extraction and intra-video parallelism to speed up video summarization and implement them in a distributed environment using Hadoop Map-Reduce framework to get a speed up of 2.67X on a 4-node setup. Furthermore, we show in our results that our approach has similar accuracy to the sequential applications.

Maximizing utility with a budget constraint is the primary goal for advertisers in real-time bidding (RTB) systems. The policy maximizing the utility is referred to as the optimal bidding strategy. Earlier works on optimal bidding strategy apply model-based batch reinforcement learning methods which can not generalize to unknown budget and time constraint. Further, the advertiser observes a censored market price which makes direct evaluation infeasible on batch test datasets. Previous works ignore the losing auctions to alleviate the difficulty with censored states; thus significantly modifying the test distribution. We address the challenge of lacking a clear evaluation procedure as well as the error propagated through batch reinforcement learning methods in RTB systems. We exploit two conditional independence structures in the sequential bidding process that allow us to propose a novel practical framework using the maximum entropy principle to imitate the behavior of the true distribution observed in real-time traffic. Moreover, the framework allows us to train a model that can generalize to the unseen budget conditions than limit only to those observed in history. We compare our methods on two real-world RTB datasets with several baselines and demonstrate significantly improved performance under various budget settings.

Predicting user engagement (e.g., click-through rate, conversion rate) on the display ads plays a critical role in delivering the right ad to the right user in online advertising. Existing techniques spanning Logistic Regression to Factorization Machines and their derivatives, focus on modeling the interactions among handcrafted features to predict the user engagement. Little attention has been paid on how the ad fits with the context (e.g., hosted webpage, user demographics). In this paper, we propose to include the metadata feature, which captures the visual appearance of the ad, in the user engagement prediction task. In particular, given a data sample, we combine both the basic context features, which have been widely used in existing prediction models, and the metadata feature, which is extracted from the ad using a state-of-the-art deep learning framework, to predict user engagement. To demonstrate the effectiveness of the proposed metadata feature, we compare the performance of the widely used prediction models before and after integrating the metadata feature. Our experimental results on a real-world dataset demonstrate that the metadata feature is able to further improve the prediction performance.

Logo detection in images is particularly challenging due to limited access to well-labelled data. Many existing logo detection methods are not scalable to larger datasets due to tedious bounding-box annotation work. As a result, with only a small number of logo classes and limited well-labelled images per class, their performance deteriorates on real-world applications. In this work, we propose a data augmentation and training pipeline to tackle these challenges. Specifically, we develop an incremental learning approach that starts training using synthetic data, followed by iteratively obtaining real training images from a given source and updating the current model with the newly-obtained data. To avoid model drift, we add a human curation step where incorrect detections (false-positives) are filtered out by simple-clicks using a User Interface, we designed. With this approach, we were able to generate a large (173,000 images of 173 logo classes) dataset termed Logo173 where all images are annotated with bounding-boxes. This image dataset can also be used to train a frame-by-frame baseline logo detector for videos. We demonstrate with extensive experiments that the proposed pipeline significantly saves time and effort for tedious data annotation and outperforms a current state-of-the-art logo detection method.

In programmatic advertising, ad slots are usually sold using second-price (SP) auctions in real-time. The highest bidding advertiser wins but pays only the second-highest bid (known as the winning price). In SP, for a single item, the dominant strategy of each bidder is to bid the true value from the bidder's perspective. However, in a practical setting, with budget constraints, bidding the true value is a sub-optimal strategy. Hence, to devise an optimal bidding strategy, it is of utmost importance to learn the winning price distribution accurately. Moreover, a demand-side platform (DSP), which bids on behalf of advertisers, observes the winning price if it wins the auction. For losing auctions, DSPs can only treat its bidding price as the lower bound for the unknown winning price. In literature, typically censored regression is used to model such partially observed data. A common assumption in censored regression is that the winning price is drawn from a fixed variance (homoscedastic) uni-modal distribution (most often Gaussian). However, in reality, these assumptions are often violated. We relax these assumptions and propose a heteroscedastic fully parametric censored regression approach, as well as a mixture density censored network. Our approach not only generalizes censored regression but also provides flexibility to model arbitrarily distributed real-world data. Experimental evaluation on the publicly available dataset for winning price estimation demonstrates the effectiveness of our method. Furthermore, we evaluate our algorithm on one of the largest demand-side platforms and significant improvement has been achieved in comparison with the baseline solutions.

Factorization Machines (FM) have been widely applied in industrial applications for recommendations. Traditionally FM models are trained in batch mode, which entails training the model with large datasets every few hours or days. Such training procedure cannot capture the trends evolving in real time with large volume of streaming data. In this paper, we propose an online training scheme for FM with the alternating least squares (ALS) technique, which has comparable performance with existing batch training algorithms. We incorporate an online update mechanism to the model parameters at the cost of storing a small cache. The mechanism also stabilizes the training error more than a traditional online training technique like stochastic gradient descent (SGD) as data points come in, which is crucial for real-time applications. Experiments on large scale datasets validate the efficiency and robustness of our method.

Advertisements are an integral part of internet economics and culture, and video ads are the most popular and arguably the most entertaining form of advertisements. With the recent growth in digital marketing, video ads have seen unprecedented growth and are growing in importance as an advertising means. Video ads are expensive to create and are not always effective. The effectiveness of a video ad is usually not known before its deployment, which is non-ideal for creators, advertisers, and ad platforms. In this paper, we outline an idea to provide feedback before an ad is placed on its effectiveness based on the video along with the historical data about the effectiveness of other video ads. We propose a multi-modal mixture based algorithm to predict the effectiveness automatically. Specifically, we exploit rich textual information often found with an advertisement as well as visual information to learn a finite mixture model. Our experiments on a publicly available dataset show that our approach can outperform other baseline approaches.

Most of existing digital content recommendation systems use explicit feedback data like user's ratings. While such systems rely on user input, those do not sense the context. In this paper, we propose a framework for digital content recommendation using only implicit feedback data (i.e., information collected from session usage without any direct feedback from user), which not only considers interactions among users and contents but also various other implicit information available during a video session. To capture interactions among such attributes, we choose Higher-Order Factorization Machines (HoFM) as our predictor and test our approach on real-world video usage data. In the experiments we explore different possible factors that may affect the performance of HoFM predictor. We observe that increasing the number of sessions of users considered to build the predictor significantly improves prediction accuracy, whereas increasing the order or depth of interactions may not. We also present an application of our work to a video recommendation system.

In the analysis of digital content consumption, session progress provides a good alternative to using manual ratings for measuring user engagement. A good prediction of session progress is useful for optimizing and personalizing the end-user experience. Most prevalent methods of predicting session progress are based on matrix completion and only consider the interaction among users and videos, while the associated contextual information is usually not used. In this paper, we present our approach for video recommendation, based on session progress prediction and incorporating the context. We test our approach on real-world session progress data, and observe considerable improvement in prediction accuracy achieved by incorporating selected context. Our experiments also show that proper context selection and the number of observed sessions for users are two key factors affecting the prediction accuracy.

Context-Aware Recommendation Systems has gained lots of attention in both industry and academic research. Factorization Machines (FM) based recommendation has been successfully used in sparse industrial datasets for user personalized video recommendations. FM is a collaborative filtering technique for predicting a target such as user rating, given observations of interaction between some users and items. The model can incorporate any available auxiliary information about the user, the item, or the interaction which serves as context or features of the data. In this paper, we propose a framework to automatically select features on FM-based recommender systems to improve the prediction quality. FM requires the input data as a one-hot encoded feature vector in a binary space domain. We use the values of the FM parameters in the binary space to determine the importance of the context. We consider the density of the important features in the binary space to rank and select the relevant features in the original data. Experiments on multiple datasets have been conducted to validate the efficiency and robustness of our method.

Account sharing is a significant problem for online recommender systems to generate accurate personalized recommendations. To solve this problem, one not only has to identify whether an account is shared, but also needs to recognize the different users sharing that account. However, to generate relevant, personalized recommendations, the particular user under a shared account has to be correctly identified at the time of delivering the recommendations. In this paper, we address this problem by first identifying users behind each account using a projection based unsupervised method, and then learning a function which can predict a user's preference accurately based on their 'contextual' information. This approach allows us to generate personalized recommendations for each of the users sharing a single account. We empirically show that on real and synthetic data set our approach performs better than other state-of-the-art approaches.

Increasingly, video delivery over the internet is being monetized through advertisement or paid services. Almost all monetized video is delivered encrypted to the clients. Clients are authorized to receive the encryption keys after watching advertisements or based on payments. Although the exact same audio and video compression standards are used, the way media is encrypted is very different in different eco-systems like Adobe Flash, Apple HTTP Live Streaming, MPEG Dynamic Adaptive Streaming over HTTP, etc., making them incompatible with each other. In some cases, it is sample (frame) based encryption while it is packet based in others. Even while using sample based encryption, different parts of a sample are selectively encrypted by different schemes. As encryption algorithms typically use different chaining modes (e.g., Cipher Block Chaining) there is some continuity from one encryption block to another. Although there is considerable overlap between encrypted data, the encryption chains are constructed differently across different formats. The goal of this paper is to define a single mezzanine file format to serve as a Universal Encryption Format that a client platform can implement to playback media from different encrypted video delivery systems. We use some basic encryption characteristics to identify and preserve the chains by storing minimal house keeping information about which part of the data is encrypted, where chains are broken, additional initialization vectors, etc. We design and propose an encryption map that is stored typically with the media sample headers. The map securely and efficiently stores information about encryption runs in terms of sizes, IVs, and offsets. We propose additional optimizations in the format to make the encryption maps compact and the decryption, efficient.

The basic tenet of HTTP streaming is to deliver fragments of video and audio that are individually addressable chunks of content over HTTP. Some media players consume incoming video and audio data only in a time ordered multiplexed format. If alternate tracks need to be added post packaging of the media, it has to be repackaged that involves duplication resulting in multiple multiplexed files. Additionally for adaptive streaming, a set of all those files need to be added for each bitrate. Alternatively, it is more efficient to store component tracks separately, fetching only the required tracks and multiplexing audio and video in the client before sending the data to the decoder. To deliver an optimal viewing experience, the client has to take care of the seemingly conflicting constraints viz., handling the network jitter, minimizing the time to switch to an alternate track and minimizing the live latency. For instance, to absorb more network jitter more data should be available in the buffers but this would increase the switching latency. We introduce a formal buffer model for a client that gathers video and audio fragments and multiplexes them on the fly. This model uses separate video and audio buffers, a multiplexed buffer in the application, and decoding buffer associated with the decoder. We model the buffer sizes, their thresholds to request data from the network, and the rate of transfer of data between buffers. We show that these buffers can be designed varying these parameters to optimize for the above constraints. This buffer model can also be leveraged for deciding when to switch in adaptive bitrate streaming. We further validate these by experimental results from our implementation.

The ability to use the Internet can provide an important contribution to an older adult's quality of life. Communication via email with family, friends and service providers has become a critical factor for improving ones ability to cope with modern society as individual's age. The problem is that as users age, natural physical and cognitive impairments make it more difficult for them to use the required technology. The present study investigates the use of error detection as a means of improving web access amongst older adults. Specifically, error detection strategies are compared to observation as a means of identifying the impairments of Internet users.

We address the problem of optimizing mediator-based service composition where the services and the desired composition (goal) functionality are represented as i/o automata with loops. The objective of optimization is to minimize the costs of communications and computations necessary to realize the goal from the existing services. We develop an algorithm to compute the minimum cost of an automaton representing the choreographed behavior of services realizing the goal. This forms the central theme of our technique for developing automatically a strategy of decentralized mediation that will result in the optimized composition of services.

We study the problem of synthesis of a choreographer in Web service composition for a given set of services and a goal. Services and goal are represented using I/O automata which can succinctly and precisely describe the interfaces of the services. Our technique considers existence and synthesis of two types of the choreographers: a simple choreographer capable of only relaying outputs from one service to input of another and a transducing choreographer which is capable of storing and reusing inputs/outputs from the services. The central theme of our technique relies on generating I/O automata representation of all possible choreographed behavior of existing services (captured in form of universal service automaton, a concept introduced in this paper) and verifying that the goal can be simulated by the universal set of choreographed behaviors.

We present a goal-directed, local and on-the-fly algorithm for verifying the existence and synthesizing a choreographer forWeb service composition. We use i/o-automata to represent services, the desired functionality of the composition, and a choreographer to achieve the desired service by composing the existing ones. Choreographer existence and synthesis are typically performed by identifying all possible compositions realizable from the existing services and verifying whether one such composition conforms to the desired required functionality. Such a technique is subject to state-space explosion. In light of this, we have developed a tabled-logic programming technique which generates and explores compositions in a goal-directed fashion to prove/disprove the existence of choreographer and to infer whether the desired functionality is realizable. We present a prototype implementation and show the practical applicability of our technique using a variety of composition problems with the corresponding computational savings in terms of number of states and transitions explored.