DeepFake, an AI-driven face-swapping technique, has been weaponized to spread disinformation. In response, researchers have developed forensic detectors to identify such manipulations. To circumvent these defenses, a growing body of work now focuses on generating adversarial samples—carefully perturbed forgeries designed to deceive detection tools. However, most existing adversarial generation methods sacrifice image quality to achieve undetectability, introducing perceptible artifacts that ironically make them more detectable under human scrutiny. To address this limitation, we propose a novel spectral fusion approach to multimodally synthesize forgery traces from authentic facial images. Unlike traditional noise injection methods, our technique integrates diffusion-based noise during image preprocessing, embedding perturbations in the forward process of a diffusion model. This approach not only deceives forensic detectors more effectively but also preserves high visual fidelity. Through extensive experiments, our method achieves state-of-the-art DeepFake anti-forensic performance while preserving high visual fidelity, ensuring that the adversarial samples remain indistinguishable from real images.

Social media platforms face escalating harm from concentrated waves of toxic interactions, which we term Neg Storms. Unlike isolated abusive remarks, these storms emerge through rapid, correlated actions that amplify negativity and create severe risks for targets and communities. Existing moderation approaches largely focus on piecemeal detection of individual comments, missing the situational dynamics that drive escalation. This paper introduces a proactive framework for forecasting neg storms using early conversational signals. We formalize Comment Storm Severity (CSS) as a time-aware metric of thread-level toxicity, propose models that predict CSS from only the first$k$comments, and evaluate feature sets combining timing and content cues. Experiments on Reddit and Instagram show that timing features alone outperform content-only features, and that combining both yields the best performance (ROC-AUC$\approx 79.7 \%; R^{2} \approx 0.24$). While predictive scores are modest, these results validate the feasibility of anticipating harmful situations before they fully unfold. We discuss practical implications for platforms, including early checkpoints to prioritize high-risk threads, apply reversible friction, and route uncertain cases for human review. This work establishes an important starting point for research on situationlevel modeling of toxicity and proactive moderation in online communities. Note: This paper deals with a sensitive topic and includes examples of negative online comments.

The rapid rise of AI-generated and edited images has made it increasingly difficult to distinguish authentic photographs from manipulated content. Digital image forensics addresses this challenge by detecting inconsistencies between image content and metadata. Among various forensic cues, the exposure triangle parameters namely ISO Speed Ratings (ISO), aperture (F-number), and shutter speed offer a physically grounded reference for verifying authenticity. We propose a physics-guided latent triad regression framework that predicts these parameters directly from image pixel content while enforcing exposure value ($E V$) consistency through the exposure equation. Our model predicts$I S O, F$-number, and$E V$in log space, deriving shutter speed to ensure physically coherent and non-redundant predictions. Trained on RAISE-2K, it achieves strong correlations with ground-truth parameters ($R^{2} \approx 0.32$to 0.35) and high$\text{E V}$consistency ($R^{2}=0.69$). By embedding physical exposure laws into learning, the framework produces interpretable, exposure-consistent predictions that enhance metadata verification, camera provenance analysis, and image authenticity assessment.

DeepFake, an AI technology that can automatically synthesize facial forgeries, has recently attracted worldwide attention. While DeepFakes can be entertaining, they can also be used to spread falsified information or be weaponized as cognition warfare. Forensic researchers have been dedicated to designing defensive algorithms to combat such disinformation. However, attacking technologies have been developed to make DeepFake products more aggressive. For example, by launching anti-forensics and adversarial attacks, DeepFakes can be disguised as authentic media to evade forensic detectors. However, such manipulations often sacrifice image quality for satisfactory undetectability. To address this issue, we propose a method to generate a novel adversarial sharpening mask for launching black-box anti-forensics attacks. Unlike many existing methods, our approach injects perturbations that allow DeepFakes to achieve high anti-forensics performance while maintaining pleasant sharpening visual effects. Experimental evaluations demonstrate that our method successfully disrupts state-of-the-art DeepFake detectors. Moreover, compared to images processed by existing DeepFake anti-forensics methods, our method’s quality of anti-forensics DeepFakes rendered is significantly improved. Our code is available at https://github.com/fb-reps/HQ-AF_GAN .

Generative Adversarial Networks (GANs) have enabled the creation of highly authentic facial images, which are increasingly used in deceptive social media profiles and other forms of disinformation, resulting in serious consequences. Significant progress has been made in developing GAN-generated face detection systems to identify these fake images. This study offers a comprehensive review of recent advancements in GAN-generated face detection, focusing on techniques that detect facial images generated by GAN models. We categorize detection methods into three groups: (1) deep learning-based approaches, (2) physics-based methods, and (3) physiology-based methods. We summarize key concepts in each category, connecting them to relevant implementations, datasets, and evaluation metrics. Additionally, we provide a comparative analysis between automated detection and human visual performance to highlight the strengths and weaknesses of both approaches. Furthermore, we review related surveys, including detecting morphed faces, manipulated faces, DeepFake, and faces generated by diffusion models. Finally, we discuss unresolved challenges and suggest potential directions for future research.

Cloud-based online document editing services, such as Google Docs and Office 365, provide an inexpensive and efficient means of managing documents. However, storing data on the cloud also raises certain security and privacy concerns, especially when the data is of confidential and sensitive nature. We argue that in online editing environments, user data should never be exposed to the cloud in plaintext form. Significant prior work around secure collaborative editing has several limitations and is not practical. Thus, in this paper, we propose a secure online editing framework, called SecureC2Edit, which is based on structured peer-to-peer architecture, uses hybrid differential synchronization, allows collaborative and concurrent access to the document, yet ensures security and privacy by encrypting the data before storing it on the cloud. The framework is evaluated for security, operability, and performance.

This paper proposes a dual-channel network for DeepFake detection. The network comprises two channels: one using a stacked Maxvit block to process the downsampled original images, and the other using a stacked ResNet basic block to capture features from the discrete cosine transform of the image spectrums. The components extracted from the two channels are concatenated using a linear layer to train the entire model for exposing DeepFakes. Experimental results demonstrate that the proposed method could achieve satisfactory forensics performance. Besides, the experiments of cross-dataset evaluations prove it is also high in generalizability.

Generating falsified faces by artificial intelligence, widely known as DeepFake, has attracted attention worldwide since 2017. Given the potential threat brought by this novel technique, forensics researchers dedicated themselves to detect the video forgery. Except for exposing falsified faces, there could be extended research directions for DeepFake such as anti-forensics. It can disclose the vulnerability of current DeepFake forensics methods. Besides, it could also enable DeepFake videos as tactical weapons if the falsified faces are more subtle to be detected. In this paper, we propose a GAN model to behave as an anti-forensics tool. It features a novel architecture with additional supervising modules for enhancing image visual quality. Besides, a loss function is designed to improve the efficiency of the proposed model. After experimental evaluations, we show that the DeepFake forensics detectors are susceptible to attacks launched by the proposed method. Besides, the proposed method can efficiently produce anti-forensics videos in satisfying visual quality without noticeable artifacts. Compared with the other anti-forensics approaches, this is tremendous progress achieved for DeepFake anti-forensics. The attack launched by our proposed method can be truly regarded as DeepFake anti-forensics as it can fool detecting algorithms and human eyes simultaneously.

Harmful algal blooms pose a threat to the lakes’ habitation and economy in many parts of the United States. State administrators have been devising a state-of-the-art monitoring and forecasting system for these harmful events. The efficacy of a monitoring and forecasting system relies on the performance of algal blooms detection. This paper proposes a deep active genetic learning framework, combining deep active learning as a classifier and a genetic algorithm as a feature selector, for the detection of harmful algal bloom events in the state of New York using climate data. A spatio-temporal weather data point training sample is introduced to retrieve relevant information of both harmful and non-harmful bloom classes. The most informative sample is selected through information entropy criterion to feed the model in classifying harmful blooms, and the most related features are selected using genetic algorithms. The proposed framework provides a classification accuracy of 97.14% with a reduced sample size and feature vector.

Since the advent of the smartphone, the number of group images taken every day is rising exponentially. The photographers' struggle is to make sure everyone looks at the camera while taking the picture. More specifically, in a group image, if everybody is not looking in the same direction, then the image's aesthetic quality and utility are depreciated. The photographer usually discards the image, and then subsequently, several images are taken to mitigate this issue. Usually, users have to manually check if the image is uniformly gazed, which is tedious and time-consuming. This paper proposes a method for classifying a given group image as uniformly gazed or nonuniformly gazed by calculating the Gaze Uniformity Index. We evaluate the proposed method on a subset of the `Images of Groups' dataset. The proposed method achieved an accuracy of 67%.

Preserving the privacy of people in video surveillance systems is quite challenging, and a significant amount of research has been done to solve this problem in recent times. Majority of existing techniques are based on detecting bodily cues such as face and/or silhouette and obscuring them so that people in the videos cannot be identified. We observe that merely hiding bodily cues is not enough for protecting identities of the individuals in the videos. An adversary, who has prior contextual knowledge about the surveilled area, can identify people in the video by exploiting the implicit inference channels such as behavior, place, and time. This article presents an anonymous surveillance system, called Watch Me from Distance (WMD), which advocates for outsourcing of surveillance video monitoring (similar to call centers) to the long-distance sites where professional security operators watch the video and alert the local site when any suspicious or abnormal event takes place. We find that long-distance monitoring helps in decoupling the contextual knowledge of security operators. Since security operators at the remote site could turn into adversaries, a trust computation model to determine the credibility of the operators is presented as an integral part of the proposed system. The feasibility study and experiments suggest that the proposed system provides more robust measures of privacy yet maintains surveillance effectiveness.

Message Locked Encryption (MLE) provides a way to achieve deduplication of data over the cloud in a secured and efficient manner. Various cryptographic protocols that are variants of the MLE scheme have been introduced, but they are either vulnerable to poison attack or consume a large amount of bandwidth. We introduce a new client-side deduplication protocol, called DICE (Dual Integrity Convergent Encryption), in which we perform tag checking and send the hash instead of the entire message over the network, resulting in a reduction of communication and computation cost without any loss of security. Our strategy is secure against both the erasure and the duplicate-faking attacks (known as a poison attack when performed together). Comparative analysis with other existing strategies validate the efficacy of the proposed protocol.

We often send our confidential information such as passport, credit card, social security numbers over email without concern about the security of email services. Existing network security mechanisms provide adequate security from external malicious adversaries and eavesdroppers, but they don't guarantee that the email service providers (ESPs) wouldn't or can't access our email data themselves, which in some cases could be highly confidential. One of the ways to protect email data from ESPs is to use Pretty Good Privacy (PGP) that has many limitations including key storage problem and dependability on third party services, making it cumbersome to use in practice. In this paper, we present S3Email method that provides email security against ESPs. The proposed method uses a cryptographic secret sharing technique in a novel way and encrypts the email metadata, body and attachments before the email is sent. In the proposed solution, the email sender and receiver must have at least two email accounts on the existing ESPs, which is not unusual today. Experiments and analysis show that the S3Email method provides information theoretic security with minimal computational overhead.

The benefits of high-end computation infrastructure facilities provided by cloud-based multimedia systems are attracting people all around the globe. However, such cloud-based systems possess security issues as third party servers become involved in them. Rendering data in an unreadable form so that no information is revealed to the cloud data centers will serve as the best solution to these security issues. One such image encryption scheme based on a Permutation Ordered Binary Number System has been proposed in this work. It distributes the image information in totally random shares, which can be stored at the cloud data centers. Further, the proposed scheme authenticates the shares at the pixel level. If any tampering is done at the cloud servers, the scheme can accurately identify the altered pixels via authentication bits and localizes the tampered area. The tampered portion is also reflected back in the reconstructed image that is obtained at the authentic user end. The experimental results validate the efficacy of the proposed scheme against various kinds of possible attacks, tested with a variety of images. The tamper detection accuracy has been computed on a pixel basis and found to be satisfactorily high for most of the tampering scenarios.

Cloud data centers (CDCs) are becoming a cost-effective method for processing and storage of multimedia data including images, video, and audio. Since CDCs are physically located in different jurisdictions, and are managed by external parties, data security is a growing concern. Data encryption at CDCs is commonly practiced to improve data security. However, to process the data at CDCs, data must often be decrypted, which raises issues in security. Thus, there is a growing demand for data processing techniques in encrypted domain in such an outsourced environment. In this article, we analyze encrypted domain speech content processing techniques for noise reduction. Noise contaminates speech during transmission or during the acquisition process by recording. As a result, the quality of the speech content is degraded. We apply Shamir’s secret sharing as the cryptosystem to encrypt speech data before uploading it to a CDC. We then propose finite impulse response digital filters to reduce white and wind noise in the speech in the encrypted domain. We prove that our proposed schemes meet the security requirements of efficiency, accuracy, and checkability for both semi-honest and malicious adversarial models. Experimental results show that our proposed filtering techniques for speech noise reduction in the encrypted domain produce similar results when compared to plaintext domain processing.

Cyberbullying is an important socio-technical challenge in Online Social Networks (OSN). With the growth trends of heterogeneous data in OSN, better network characterization, and textual feature sophistication, recent efforts have realized the value of looking at heterogeneous modes of information including textual features, social features, and image-based features for better cyberbullying detection. These approaches, however, still use these features either individually or combine them `naively' without considering the different confidence levels associated with each feature or the interdependencies between features. We propose a novel probabilistic information fusion framework that utilizes confidence score and interdependencies associated with different social and textual features and uses those to build better predictors for cyberbullying. The performance of the proposed approach was compared to a recent approach in literature which used a similar dataset and features and the proposed approach resulted in significant improvements in terms of cyberbullying detection.

Abstract Phishing is a fraudulent technique that is used over the Internet to deceive users with the goal of extracting their personal information such as username, passwords, credit card, and bank account information. The key to phishing is deception. Phishing uses email spoofing as its initial medium for deceptive communication followed by spoofed websites to obtain the needed information from the victims. Phishing was discovered in 1996, and today, it is one of the most severe cybercrimes faced by the Internet users. Researchers are working on the prevention, detection, and education of phishing attacks, but to date, there is no complete and accurate solution for thwarting them. This paper studies, analyzes, and classifies the most significant and novel strategies proposed in the area of phished website detection, and outlines their advantages and drawbacks. Furthermore, a detailed analysis of the latest schemes proposed by researchers in various subcategories is provided. The paper identifies advantages, drawbacks, and research gaps in the area of phishing website detection that can be worked upon in future research and developments. The analysis given in this paper will help academia and industries to identify the best anti‐phishing technique. Copyright © 2016 John Wiley & Sons, Ltd.

Cloud-based multimedia systems are becoming increasingly common. These systems offer not only storage facility, but also high-end computing infrastructure which can be used to process data for various analysis tasks ranging from low-level data quality enhancement to high-level activity and behavior identification operations. However, cloud data centers, being third party servers, are often prone to information leakage, raising security and privacy concerns. In this article, we present a Shamir's secret sharing based method to enhance the quality of encrypted image data over cloud. Using the proposed method we show that several image enhancement operations such as noise removal, antialiasing, edge and contrast enhancement, and dehazing can be performed in encrypted domain with near-zero loss in accuracy and minimal computation and data overhead. Moreover, the proposed method is proven to be information theoretically secure.

The use of multiple heterogeneous cameras is becoming more common in today's surveillance systems. In order to perform surveillance tasks, effective coordination and control in multi-camera systems is very important, and is catching significant research attention these days. This survey aims to provide researchers with a state-of-the-art overview of various techniques for multi-camera coordination and control ( MC 3 ) that have been adopted in surveillance systems. The existing literature on MC 3 is presented through several classifications based on the applicable architectures, frameworks and the associated surveillance tasks. Finally, a discussion on the open problems in surveillance area that can be solved effectively using MC 3 and the future directions in MC 3 research is presented

Secret image sharing is a popular image hiding scheme that typically uses (3, 3, n) multi-secret sharing to hide the colors of a secret image. The use of (3, 3, n) multi-secret sharing, however, can lead to information loss. In this paper, we study this loss of information from an image perspective, and show that one-third of the color values of the secret image can be leaked when the sum of any two selected share numbers is equal to the considered prime number in the secret sharing. Furthermore, we show that if the selected share numbers do not satisfy this condition (for example, when the value of each of the selected share number is less than the half of the value of the prime number), then the colors of the secret image are not leaked. In this case, a noise-like image is reconstructed from the knowledge of less than three shares.

Advances in cloud computing have allowed volume rendering tasks, typically done by volume ray-casting, to be outsourced to cloud data centers. The availability of volume data and rendered images (which can contain important information such as the disease information of a patient) to a third-party cloud provider, however, presents security and privacy challenges. This paper addresses these challenges by proposing a secure cloud-based volume ray-casting framework that distributes the rendering tasks among the data centers and hides the information that is exchanged between the server and a data center, between two data centers, and between a data center and the client by using Shamir's secret sharing, such that none of the data centers has enough information to know the secret data and/or rendered image. Experiments and analyses show that our framework is highly secure and requires low computation cost.

Secret image sharing is a method for distributing a secret image amongst n data stores, each storing a shadow image of the secret, such that the original secret image can be recovered only if any k out of the n shares is available. Existing secret image sharing schemes, however, do not support scaling and cropping operations on the shadow image, which are useful for zooming on large images. In this paper, we propose an image sharing scheme that allows the user to retrieve a scaled or cropped version of the secret image by operating directly on the shadow images, therefore reducing the amount of data sent from the data stores to the user. Results and analyses show that our scheme is highly secure, requires low computational cost, and supports a large number of scale factors with arbitrary crop.

In this article, a novel logo watermarking scheme is proposed based on wavelet frame transform, singular value decomposition and automatic thresholding. The proposed scheme essentially rectifies the ambiguity problem in the SVD-based watermarking. The core idea is to randomly upscale the size of host image using reversible random extension transform followed by the embedding of logo watermark in the wavelet frame domain. After embedding, a verification phase is casted with the help of a binary watermark and toral automorphism. At the extraction end, the binary watermark is first extracted followed by the verification of watermarked image. The logo watermark is extracted if and only if the watermarked image is verified. The security, attack and comparative analysis confirm high security, efficiency and robustness of the proposed watermarking system.

Despite the fact that performance improvements have been reported in the last years, semantic concept detection in video remains a challenging problem. Existing concept detection techniques, with ontology rules, exploit the static correlations among primitive concepts but not the dynamic spatiotemporal correlations. The proposed method rewards (or punishes) detected primitive concepts using dynamic spatiotemporal correlations of the given ontology rules and updates these ontology rules based on the accuracy of detection. Adaptively learned ontology rules significantly help in improving the overall accuracy of concept detection as shown in the experimental result.

Outsourcing the tasks of medical data visualization to cloud centers presents new security challenges. In this paper, we propose a framework for cloud-based remote medical data visualization that protects the security of data at the cloud centers. To achieve this, we integrate the cryptographic secret sharing with pre-classification volume ray-casting and propose a secure volume ray-casting pipeline that hides the color-coded information of the secret medical data during rendering at the data centers. Results and analysis show the utility of the proposed framework.

Surveillance and monitoring systems generally employ a large number of cameras to capture people's activities in the environment. These activities are analyzed by hosts (human operators and/or computers) for threat detection. Threat detection is a target centric task in which the behavior of each target is analyzed separately, which requires a significant amount of human attention and is a computationally intensive task for automatic analysis. In order to meet the real-time requirements of surveillance, it is necessary to distribute the video processing load over multiple hosts. In general, cameras are statically assigned to the hosts; we show that this is not a desirable solution as the workload for a particular camera may vary over time depending on the number of targets in its view. In the future, this uneven distribution of workload will become more critical as the sensing infrastructures are being deployed on the cloud. In this paper, we model the camera workload as a function of the number of targets, and use that to dynamically assign video feeds to the hosts. Experimental results show that the proposed model successfully captures the variability of the workload, and that the dynamic workload assignment provides better results than a static assignment.

Public transport safety is an important issue that has recently gained large attention, especially with the rise of violence happening abroad. To avoid such incidents and to perform post-incident investigations, many buses today are equipped with surveillance cameras. These cameras are usually in stalled at important places such as doors, the front and the middle of the bus. This camera placement is often performed manually based on human intuition and knowledge; however, there is no scientific basis to justify: 1) how many cameras would be sufficient, and 2) where they should be placed, to increase the area of coverage at a minimum cost. In this pa per we present this as an optimization problem and propose a method to compute the approximate coverage of a camera inside the 3D bus model. The utility of proposed method is demonstrated for a single camera setup.

Video surveillance is a very effective tool of surveillance that enables a single security agent to monitor wide areas. However, it compromises the privacy of the individuals. There have been attempts to obfuscate face and silhouette regions of the images to hide the identity of individuals. We recognize that in traditional surveillance systems, the viewer generally has sufficient contextual knowledge about location of the camera, time, and activity patterns; which can lead to identity leakage even when the visual cues (face and appearance) are not present. In this way, the viewer can relate the identity of individuals to the sensitive information in the video causing privacy loss. In order to provide robust privacy preservation, the context knowledge needs to be decoupled from the video; however, human monitoring of the videos is also necessary for the assessment of the situation. In this paper we propose anonymous surveillance framework that decouples the contextual knowledge and video to the minimal extent required for situation assessment. The experimental results confirm that the proposed framework is very effective in protecting the privacy, yet does not affect much of the surveillance utility of the data.

Current surveillance systems consist of large numbers of cameras. The video feeds from cameras are automatically processed for threat detection, which is a computationally intensive task. In order to meet the real-time requirements of surveillance, we need to distribute the video processing over multiple computers. Generally the cameras are statically assigned to the processors; we show that this is not a desirable solution as the workload for a particular camera may vary over time depending on the number of the targets in its view. In future, this uneven distribution of workload will become more critical as the sensing infrastructures are being deployed on the cloud. In this work, we model the camera workload as a function of the number of targets, and use that to dynamically assign video feeds to the processors. Experimental results show that the proposed model successfully captures the variability of the workload, and that dynamic workload assignment provides better results than a static assignment.

Camera surveillance systems have proved useful for public safety. The main disadvantage is that since the camera views are monitored in a remote control room, it is often difficult for security officers to reach the crime-scene in time. During this time, the assailant(s) have likely caused sufficient damage and threaten many lives. To overcome this problem, this paper proposes to take a standard surveillance system, augment it with simple weaponry for the purpose of disabling potential assailants, and use mathematical models to develop decision criteria for selecting the safest and most effective weapon in a given situation. The feasibility of the proposed system is examined using simulation results which also validate the utility of the proposed decision models.

Current sensor-based monitoring systems use multiple sensors in order to identify high-level information based on the events that take place in the monitored environment. This information is obtained through low-level processing of sensory media streams, which are usually noisy and imprecise, leading to many undesired consequences such as false alarms, service interruptions, and often violation of privacy. Therefore, we need a mechanism to compute the quality of sensor-driven information that would help a user or a system in making an informed decision and improve the automated monitoring process. In this article, we propose a model to characterize such quality of information in a multisensor multimedia monitoring system in terms of certainty, accuracy/confidence and timeliness. Our model adopts a multimodal fusion approach to obtain the target information and dynamically compute these attributes based on the observations of the participating sensors. We consider the environment context, the agreement/disagreement among the sensors, and their prior confidence in the fusion process in determining the information of interest. The proposed method is demonstrated by developing and deploying a real-time monitoring system in a simulated smart environment. The effectiveness and suitability of the method has been demonstrated by dynamically assessing the value of the three quality attributes with respect to the detection and identification of human presence in the environment.

The utilization of multimedia devices is growing rapidly in surveillance and monitoring applications. These multimedia surveillance systems need to process large amounts of multimodal sensor data in order to detect events and objects. While processing this large amount of data, the system faces many processing and network bottlenecks. The design of efficient multimedia surveillance system requires intelligent architectural decisions and performance evaluation to cope with these resource demands. One critical issue among all these architectures is task assignment among processing units. To study the effect of this task assignment on system performance with quantifiable performance measures is very useful and challenging. We define a Functionality Delegation Coefficient which abstracts the delegation of functionality among processing units of a distributed surveillance system and show its effect on event blocking probability and response time. Simulation and real implementation results are provided to validate the model.

Secret image sharing is a technique used to secure digital images against disclosure and tampering. In this paper, we analyze the security of the secret image sharing scheme proposed by Thien and Lin and expose its weakness. Their method uses a permutation step which requires a key. The security of their method is dependent more on this key rather than on the unconditional security characteristics of the secret sharing scheme. We propose an improved scheme for the secret image sharing by removing the permutation step. The core idea of our scheme is to divide the secret image into k sections and use the pixel values from these sections as the coefficients of the polynomial used for creating shares in Shamir's secret sharing scheme. The security analysis and experimental results are presented to demonstrate the utility of our method.

Video cameras are being extensively used in many applications. Huge amounts of video are being recorded and stored everyday by surveillance systems. Any proposed application of this data raises severe privacy concerns. An assessment of privacy loss is necessary before any potential application of the data. In traditional methods of privacy modeling, researchers have focused on explicit means of identity leakage like facial information, etc. However, other implicit inference channels through which individual's an identity can be learned have not been considered. For example, an adversary can observe the behavior, look at the places visited and combine that with the temporal information to infer the identity of the person in the video. In this work, we thoroughly investigate privacy issues involved with the video data considering both implicit and explicit channels. We first establish an analogy with the statistical databases and then propose a model to calculate the privacy loss that might occur due to publication of the video data. The experimental results demonstrate the utility of the proposed model.

Effective and robust visual tracking is one of the most important tasks for the intelligent visual surveillance. In this paper, we proposed a novel method for detecting and tracking moving people using the spatiotemporal latent semantic cues and the incremental eigenspace tracking techniques. During tracking process, the target appearance model is incrementally learned in low dimensional tensor eigenspace by adaptively updating the eigenbasis and sample mean. At the same time, the spatiotemporal latent semantic cues calibrate the estimation of tracking and detect new moving people coming in the same surveillance scene. Experiment results show that with the calibration based on spatiotemporal latent semantic cues, the proposed method can track the moving people automatically and effectively.

In large scale surveillance systems, a number of CCTV cameras are installed in distributed premises and are connected to a central control station, where human operators observe the different camera views for identifying a probable security breach. In such situations, it is particularly difficult for the operator to pay attention to all camera views. Studies have shown that a human operator can effectively monitor only four camera views at a time. This paper attempts to solve the problem of dynamically selecting and scheduling the four best CCTV views. We adopt a human-centric approach in which the system computes the operatorpsilas attention in the CCTV views to automatically determine the importance of events captured by the respective cameras. The experiments show that the proposed method helps a human operator in identifying important events occuring in the environment.

Multi-sensor systems are increasingly being deployed in many application scenarios due to the enormous potential they can offer. However, as the processing of sensory data often results in imprecise outcome, measuring the quality of information (QoI) in these systems has become an important issue. The measurement of QoI is usually performed by processing the elementary data provided by the heterogeneous sensors, which is also influenced by the techniques involved in sensor management. However, the effect of context, such as environmental geometry, sensor placement, orientation, time, and other parameters in computing QoI has not yet been explored extensively in the literature. This paper proposes a context evolution model and studies its impact in the QoI computation. In particular, we show that the dynamic context information can be utilized to manage a multi-sensor system to improve its QoI.

Multimedia systems utilize multiple media streams, each of which have different confidence levels in accomplishing various detection tasks. For example, in a multimedia surveillance system, one would usually have higher confidence in an audio stream compared to a video stream for detecting human shouting events. The pre-computation of these confidence levels is cumbersome especially when new media streams are dynamically added to the system. This paper proposes a novel method, which dynamically computes the confidence levels of new streams based on the past history of their agreement/disagreement with the already trusted streams. To demonstrate the utility of the proposed method, we provide the experimental results for detecting events in a multimedia surveillance scenario.

A multimedia analysis system utilizes a set of correlated media streams, each of which, we assume, has a confidence level and a cost associated with it, and each of which partially helps in achieving the system goal. However, the fact that at any instant, not all of the media streams contribute towards a system goal brings up the issue of finding the best subset from the available set of media streams. For example, a subset of two video cameras and two microphones could be better than any other subset of sensors at some time instance to achieve a surveillance goal (e.g. event detection). This article presents a novel framework that finds the optimal subset of media streams so as to achieve the system goal under specified constraints. The proposed framework uses a dynamic programming approach to find the optimal subset of media streams based on three different criteria: first, by maximizing the probability of achieving the goal under the specified cost and confidence; second, by maximizing the confidence in the achieved goal under the specified cost and probability with which the goal is achieved; and third, by minimizing the cost to achieve the goal with a specified probability and confidence. Each of these problems is proven to be NP-Complete. From an AI point of view, the solution we propose is heuristic-based, and for each criterion, utilizes a heuristic function which for a given problem, combines optimal solutions of small-sized subproblems to yield a potential near-optimal solution to the original problem. The proposed framework allows for a tradeoff among the aforementioned three criteria, and offers the flexibility to compare whether any one set of media streams of low cost would be better than any other set of higher cost, or whether any one set of media streams of high confidence would be better than any other set of low confidence. To show the utility of our framework, we provide the experimental results for event detection in a surveillance scenario.

With the increasing use of audio sensors in surveillance and monitoring applications, event detection using audio streams has emerged as an important research problem. This paper presents a hierarchical approach for audio based event detection for surveillance. The proposed approach first classifies a given audio frame into vocal and nonvocal events, and then performs further classification into normal and excited events. We model the events using a Gaussian mixture model and optimize the parameters for four different audio features ZCR, LPC, LPCC and LFCC. Experiments have been performed to evaluate the effectiveness of the features for detecting various normal and the excited state human activities. The results show that the proposed top-down event detection approach works significantly better than the single level approach

Segmentation of foreground and background has been an important research problem arising out of many applications including video surveillance. A method commonly used for segmentation is "background subtraction" or thresholding the difference between the estimated background image and current image. Adaptive Gaussian mixture based background modelling has been proposed by many researchers for increasing the robustness against environmental changes. However, all these methods, being computationally intensive, need to be optimized for efficient and real-time performance especially at a higher image resolution. In this paper, we propose an improved foreground/background segmentation method which uses experiential sampling technique to restrict the computational efforts in the region of interest. We exploit the fact that the region of interest in general is present only in a small part of the image, therefore, the attention should only be focused in those regions. The proposed method shows a significant gain in processing speed at the expense of minor loss in accuracy. We provide experimental results and detailed analysis to show the utility of our method

A multimedia system utilizes a set of correlated media streams each of which partially help in achieving the system goal. However, since not all of the streams always contribute towards the goal, there is a need for determining the most informative subset from the available set of media streams at any instant. For example, a subset of two video cameras and two microphones could be better than any other subset of multimedia sensors at some time instance. This paper presents a novel framework to find the optimal subset of media streams that achieves the system goal under specified constraints. The proposed framework uses a dynamic programming approach to find the optimal subset of media streams based on two criteria; first, by maximizing the probability of achieving the goal under the specified maximum cost, and second by minimizing the cost of using the streams so that the goal is achieved with a specified minimum probability. To show the utility of our framework, we provide the simulation results for hypothesis testing.

The fusion of multiple correlated observations of a multimedia system is a research problem arising in many multimedia applications. In this paper, we propose a novel framework for the probabilistic fusion of correlated multimedia observations. Assuming that each of the media stream has a priori probability of achieving the goal and their underlying correlations are known, our framework fuses the individual probabilities using the quantitative correlation based on a Bayesian approach. The simulation results show that fewer highly-positively-correlated observations better achieve a specified goal when compared to the use of a larger number of observations with low correlation.

Ensuring the integrity of a digital video is an important and challenging research problem arising out of many video applications. In this paper, we present a hierarchical framework for video authentication based on cryptographic secret sharing that protects a video from spatial cropping and temporal jittering, yet is robust against frame dropping in the streaming video scenario. Our algorithm provides a tradeoff between security and robustness by having configurable inputs. The authentication signature is compact and very sensitive against spatial attacks such as region tampering, and interframe attacks like frame replacement, major frame dropping, and frame reordering. Given a video, we identify the key frames based on different energy between the frames. Considering video frames as shares, we compute the secret at three hierarchical levels. The master secret is used as digital signature to authenticate the video. We present extensive experimental results which show the utility of our technique.