Current generative technologies have enabled the creation of realistic synthetic media, including face images. Although such synthetic data may pose a threat when used for malicious purposes, e.g., the spread of disinformation or impersonation, these data can be leveraged to improve the performance of Face Recognition Systems (FRS). This paper then proposes a strategy that leverages synthetic face images to reduce biases in FRS; in other words, to reduce unexpected and incorrect results due to imbalances in the training data in terms of ethnicity, gender, and other human traits. The proposed strategy balances the samples available for training with synthetic images, which are used exclusively during an initial training process. Once such an initial training process is complete, our strategy resumes training on the reference dataset (real data) to learn the target identities. This strategy has two main advantages: (1) it can bias the model towards a specific group of people to which a person of interest belongs, and (2) it can increase the authentication performance for a person of interest by boosting specificity. Our experiments on several datasets, including one for crime suspect authentication, confirm the advantages of our strategy.

Orthogonal transforms are key components of several image and video compression systems and standards, as they provide a de-correlated representation of signals to enhance compressibility. However, the most commonly used transforms for compression, such as the Discrete Cosine transforms (DCT) and Discrete Sine transforms (DST), are fixed and non-adaptive, limiting their ability to capture complex or varying signal characteristics. Graph-based transforms (GBTs) have shown improved energy compaction and reconstruction performance, but face two major limitations: the need to signal graph information in the compressed bitstream, which increases overhead and may complicates decoder synchronization, and a dependency on offline training process, which is highly dependent on the quality and completeness of the training data. To address these issues, this paper introduces a novel framework, GBT-ONL, which learns GBTs online in the context of block-based predictive transform coding. The proposed GBT-ONL framework uses a shallow fully connected neural network to predict the graph Laplacian needed for both the forward and inverse GBT. By relying only on information available during encoding, GBT-ONL eliminates the need to signal additional information in the compressed bitstream, and removes the requirement for any prior offline training. Evaluations on several video sequences show that GBT-ONL outperforms both traditional (non-learnable) transforms and existing learnable transforms in terms of energy compaction, reconstruction error, and compression efficiency, as measured by BD-PSNR and BD-Rate metrics.

Face detection technology is critical in video surveillance applications. Unfortunately, many of the existing approaches fail to detect tiny faces, especially in videos acquired in uncontrolled public environments. In this work, we address this challenge by proposing a novel detection framework that relies on a new multiscale detector that does not require the input to be re-scaled. Our framework leverages existing detectors to detect relatively large faces along with a new detector specifically designed for tiny faces. The core part of our framework is the use of an integral score to detect tiny faces by densely scanning several support regions at different scales. Although this strategy may pose high computational complexity in high-resolution videos, the integral operations used by our framework significantly reduce the computational cost, making it feasible for high-resolution videos and real-time applications. Experiments on the WIDER FACE and MEVA datasets show a significant improvement in performance, particularly for tiny faces depicted in surveillance videos acquired in uncontrolled environments.

Monocular Depth Estimation (MDE) models have shown significant potential in estimating depth from single images. However, the resulting depth maps occasionally lack fine-grained details, blurred edges, and missing objects, limiting their practical utility. To solve those limitations, we propose a dual-encoder model with a feature fusion module - FFM, a customised loss function, and a depth-based diffusion module to enhance predicted depth maps. Our model integrates edge information by incorporating an edge encoder that utilises edge data and an RGB feature extractor followed by an FFM module to fuse the information from both encoders. The edge loss function is used to evaluate the accuracy of edge outputs generated using the Scharr filter to improve edge detection in depth maps. As most current MDE studies predict depth maps at half the spatial resolution, we developed a depth-based diffusion module to generate higher-resolution depth maps, preserving fine details. Our approach demonstrates state-of-the-art performance, with precise edge capture, when evaluated on the KITTI and NYU-v2 datasets.

For efficient human-agent interaction, an agent should proactively recognize their target user and prepare for upcoming interactions. We formulate this challenging problem as a novel task of jointly forecasting a person’s intent to interact with the agent, their attitude towards the agent and the action they will perform, from the agent’s (egocentric) perspective. We propose SocialEgoNet - a graph-based spatiotemporal framework that exploits task dependencies through a hierarchical multitask learning approach. SocialEgoNet uses whole-body skeletons (keypoints from face, hands and body) extracted from only 1 second of video input for high inference speed. For evaluation, we augment an existing egocentric human-agent interaction dataset with new class labels and bounding box annotations. Extensive experiments on this augmented dataset, named JPL-Social, demonstrate real-time inference and superior performance (average accuracy across all tasks: 83.15%) of our model outperforming several competitive baselines. The additional annotations and code are available at github.com/biantongfei/SocialEgoNet.

We propose CLIP-EBC, the first fully CLIP-based model designed for accurate crowd density estimation. Existing classification-based crowd counting frameworks face challenges when directly applied with CLIP. For instance, these methods quantize count values into bordering real-valued bins, which are inconsistent with CLIP’s pretraining corpus. Besides, this quantization strategy also introduces label ambiguity near shared bin borders. To address these issues, we propose the Enhanced Blockwise Classification (EBC) framework, which utilizes integer-valued bins and additionally incorporates a density-map-based loss to further improve count accuracy. Building on this backbone-agnostic framework, CLIP-EBC fully leverages CLIP’s recognition capabilities for crowd counting. Experiments show that EBC improves classification-based methods by up to 44.5% in mean absolute error (MAE) on the UCF-QNRF dataset. Furthermore, CLIP-EBC achieves state-of-the-art performance on the NWPU-Crowd test set, with an MAE of 58.2, surpassing the previous best method by 8.6%. Our code is available at https://github.com/Yiming-M/CLIP-EBC.

The survival of numerous endangered wildlife species is increasingly jeopardized by drastic climate changes, ecological disturbances, and human activities, leading to rapid population declines. Identifying endangered and rare species using computer vision is an important task that can aid in biodiversity conservation. However, many rare species are recognizable only by specialized biologists, making the labeling process of images both challenging and costly. Moreover, even with labeled data, training models with imbalanced datasets, i.e., datasets with under-represented categories, often results in inherent learning biases. To address high labeling costs and challenges associated with sparse label learning, we propose a novel semi-supervised learning framework, 4S-Classifier, which comprises two key modules: Rare Species Bank (RSBank) and Attention-based Rare Species Embedding (RSEmbed). The RSBank module stores embeddings of rare species across multiple training epochs, using clustering, kernel density estimation, and confidence scores to enhance the learning of under-represented categories. The RSEmbed module acts as a fusion-based augmentation approach that employs embeddings to improve model performance on these sparse and rare species data. By integrating both modules, our framework achieves a classification accuracy of 88.37% on an endangered species dataset (iNaturalist) and 91.56% on a wildlife dataset (Wildlife Insights) with only 25% of the labeled data, demonstrating its outstanding performance. The code is available at https://github.com/SIPLab24/4S-Classifier

Designing 3D indoor layouts is a crucial task with significant applications in embodied robot intelligence, virtual reality, and interior design. Existing methods for 3D layout design either rely on diffusion models, which utilize spatial relationship priors, or heavily leverage the inferential capabilities of proprietary Large Language Models (LLMs) , which require extensive prompt engineering and in-context exemplars via black-box trials. These methods often face limitations in generalization and dynamic scene editing. In this paper, we introduce LLplace, a novel 3D indoor scene layout designer based on lightweight, fine-tuned, open-source LLM Llama3. LLplace circumvents the need for spatial relationship priors and in-context exemplars, enabling efficient and credible room layout generation based solely on user inputs specifying the room type and desired objects. We curated a new dialogue dataset based on the 3D-Front dataset, expanding the original data volume and incorporating dialogue data for adding and removing objects. This dataset can enhance the LLM’s spatial understanding. Furthermore, through dialogue, LLplace activates the LLM’s capability to understand 3D layouts and perform dynamic scene editing, enabling the addition and removal of objects. Our approach demonstrates that LLplace can effectively generate and edit 3D indoor layouts interactively and outperform existing methods in delivering high-quality 3D design solutions.

Automatic indoor layout generation has attracted increasing attention due to its potential in interior design, virtual environment construction, and embodied AI. Existing methods fall into two categories: prompt-driven approaches that leverage proprietary LLM services (e.g., GPT APIs), and learning-based methods trained on layout data upon diffusion-based models. Prompt-driven methods often suffer from spatial inconsistency and high computational costs, while learning-based methods are typically constrained by coarse relational graphs and limited datasets, restricting their generalization to diverse room categories. In this paper, we revisit LLM-based indoor layout generation and present 3D-SynthPlace, a large-scale dataset that combines synthetic layouts generated via a `GPT synthesize, Human inspect' pipeline, upgraded from the 3D-Front dataset. 3D-SynthPlace contains nearly 17,000 scenes, covering four common room types—bedroom, living room, kitchen, and bathroom—enriched with diverse objects and high-level spatial annotations. We further introduce OptiScene, a strong open-source LLM optimized for indoor layout generation, fine-tuned based on our 3D-SynthPlace dataset through our two-stage training. For the warum-up stage I, we adopt supervised fine-tuning (SFT), which is taught to first generate high-level spatial descriptions then conditionally predict concrete object placements. For the reinforcing stage II, to better align the generated layouts with human design preferences, we apply multi-turn direct preference optimization (DPO), which significantly improving layout quality and generation success rates. Extensive experiments demonstrate that OptiScene outperforms traditional prompt-driven and learning-based baselines. Moreover, OptiScene shows promising potential in interactive tasks such as scene editing and robot navigation, highlighting its applicability beyond static layout generation.

Pose Invariant Face Recognition (PIFR) has significantly advanced with Generative Adversarial Networks (GANs), which rotate face images acquired at any angle to a frontal view for enhanced recognition. However, such frontalization methods typically need ground-truth frontal-view images, often collected under strict laboratory conditions, making it challenging and costly to acquire the necessary training data. Additionally, traditional self-supervised PIFR methods rely on external rendering models for training, further complicating the overall training process. To tackle these two issues, we propose a new framework called Mask Rotate. Our framework introduces a novel training approach that requires no paired ground truth data for the face image frontalization task. Moreover, it eliminates the need for an external rendering model during training. Specifically, our framework simplifies the face image frontalization task by transforming it into a face image completion task. During the inference or testing stage, it employs a reliable pre-trained rendering model to obtain a frontal-view face image, which may have several regions with missing texture due to pose variations and occlusion. Our framework then uses a novel self-supervised Random Mask Attention Generative Adversarial Network (RMAGAN) to fill in these missing regions by considering them as randomly masked regions. Furthermore, our proposed Mask Rotate framework uses a reliable post-processing model designed to improve the visual quality of the face images after frontalization. In comprehensive experiments, the Mask Rotate framework eliminates the requirement for complex computations during training and achieves strong results, both qualitative and quantitative, compared to the state-of-the-art.

Generalized age feature extraction is crucial for age-related facial analysis tasks, such as age estimation and age-invariant face recognition (AIFR). Despite the recent successes of models in homogeneous-dataset experiments, their performance drops significantly in cross-dataset evaluations. Most of these models fail to extract generalized age features as they only attempt to map extracted features with training age labels directly without explicitly modeling the natural ordinal progression of aging. In this paper, we propose Order-Enhanced Contrastive Learning (OrdCon), a novel contrastive learning framework designed explicitly for ordinal attributes like age. Specifically, to extract generalized features, OrdCon aligns the direction vector of two features with either the natural aging direction or its reverse to model the ordinal process of aging. To further enhance generalizability, OrdCon leverages a novel soft proxy matching loss as a second contrastive objective, ensuring that features are positioned around the center of each age cluster with minimal intra-class variance and proportionally away from other clusters. By modeling the ageing process, the framework can enhance generalizability by improving the alignment of samples from the same class and reducing the divergence of direction vectors. We demonstrate that our proposed method achieves comparable results to state-of-the-art methods on various benchmark datasets in homogeneous-dataset evaluations for both age estimation and AIFR. In cross-dataset experiments, OrdCon outperforms other methods by reducing the mean absolute error by approximately 1.38 on average for the age estimation task and boosts the average accuracy for AIFR by 1.87%.

Block-based intra-prediction is a key process of several state-of-the-art image and video codecs. Recently, supervised machine learning (ML) has improved this type of prediction. However, the performance of these ML-based methods depends on the amount, quality, and relevance of the training data. Furthermore, they require that the model’s parameters are stored so the imaging data can be reconstructed from the compressed bit-stream. Such a requirement, unfortunately, may hinder compression performance. This work departs from the current trend of training deep neural networks (NNs) and instead focuses on overfitted shallow NNs that can be optimized online. Specifically, it introduces the Reversible Regression Network (R2-Net) for block-based intra-prediction of imaging data. The R2-Net is an overfitted fully connected NN that can accurately predict imaging data while allowing for the recovery of such data by reversing the overfitting process. The R2-Net is optimized in an online manner based on parameters initialized to known values. Such an optimization strategy removes the need to store the model’s parameters. Performance evaluations on several benchmark video sequences show that the R2-Net outperforms the traditional block-based intra-prediction strategy with gains of up to 37.61 dB PSNR.

Cross-age facial images are typically challenging and expensive to collect, making noise-free age-oriented datasets relatively small compared to widely-used large-scale facial datasets. Additionally, in real scenarios, images of the same subject at different ages are usually hard or even impossible to obtain. Both of these factors lead to a lack of supervised data, which limits the versatility of supervised methods for age-invariant face recognition, a critical task in applications such as security and biometrics. To address this issue, we propose a novel semi-supervised learning approach named Cross-Age Contrastive Learning (CACon). Thanks to the identity-preserving power of recent face synthesis models, CACon introduces a new contrastive learning method that leverages an additional synthesized sample from the input image. We also propose a new loss function in association with CACon to perform contrastive learning on a triplet of samples. We demonstrate that our method not only achieves state-of-the-art performance in homogeneous-dataset experiments on several age-invariant face recognition benchmarks but also outperforms other methods by a large margin in cross-dataset experiments.

Explainable Artificial Intelligence (xAI) has the potential to enhance the transparency and trust of AI-based systems. Although accurate predictions can be made using Deep Neural Networks (DNNs), the process used to arrive at such predictions is usually hard to explain. In terms of perceptibly human-friendly representations, such as word phrases in text or super-pixels in images, prototype-based explanations can justify a model’s decision. In this work, we introduce a DNN architecture for image classification, the Enhanced Prototypical Part Network (EPPNet), which achieves strong performance while discovering relevant prototypes that can be used to explain the classification results. This is achieved by introducing a novel cluster loss that helps to discover more relevant human-understandable prototypes. We also introduce a faithfulness score to evaluate the explainability of the results based on the discovered prototypes. Our score not only accounts for the relevance of the learned prototypes but also the performance of a model. Our evaluations on the CUB-2002011 dataset show that the EPPNet outperforms state-of-the-art xAI-based methods, in terms of both classification accuracy and explainability.

Face image synthesis detection is considerably gaining attention because of the potential negative impact on society that this type of synthetic data brings. In this paper, we propose a data-agnostic solution to detect the face image synthesis process. Specifically, our solution is based on an anomaly detection framework that requires only real data to learn the inference process. It is therefore data-agnostic in the sense that it requires no synthetic face images. The solution uses the posterior probability with respect to the reference data to determine if new samples are synthetic or not. Our evaluation results using different synthesizers show that our solution is very competitive against the state-of-the-art, which requires synthetic data for training.

Online collaborative tools for medical diagnosis produced from digital pathology images have experimented an increase in demand in recent years. Due to the large sizes of pathology images, rate control (RC) techniques that allow an accurate control of compressed file sizes are critical to meet existing bandwidth restrictions while maximizing retrieved image quality. Recently, some RC contributions to Region of Interest (RoI) coding for pathology imaging have been presented. These encode the RoI without loss and the background with some loss, and focus on providing high RC accuracy for the background area. However, none of these RC contributions deal efficiently with arbitrary RoI shapes, which hinders the accuracy of background definition and rate control. This manuscript presents a novel coding system based on prediction with a novel RC algorithm for RoI coding that allows arbitrary RoIs shapes. Compared to other methods of the state of the art, our proposed algorithm significantly improves upon their RC accuracy, while reducing the compressed data rate for the RoI by 30%. Furthermore, it offers higher quality in the reconstructed background areas, which has been linked to better clinical performance by expert pathologists. Finally, the proposed method also allows lossless compression of both the RoI and the background, producing data volumes 14% lower than coding techniques included in DICOM, such as HEVC and JPEG-LS.

Despite the prevailing transition from single-task to multi-task approaches in video anomaly detection, we observe that many adopt sub-optimal frameworks for individual proxy tasks. Motivated by this, we contend that optimizing single-task frameworks can advance both single- and multi-task approaches. Accordingly, we leverage middle-frame prediction as the primary proxy task, and introduce an effective hybrid framework designed to generate accurate predictions for normal frames and flawed predictions for abnormal frames. This hybrid framework is built upon a bi-directional structure that seamlessly integrates both vision transformers and ConvLSTMs. Specifically, we utilize this bi-directional structure to fully analyze the temporal dimension by predicting frames in both forward and backward directions, significantly boosting the detection stability. Given the transformer's capacity to model long-range contextual dependencies, we develop a convolutional temporal transformer that efficiently associates feature maps from all context frames to generate attention-based predictions for target frames. Furthermore, we devise a layer-interactive ConvLSTM bridge that facilitates the smooth flow of low-level features across layers and time-steps, thereby strengthening predictions with fine details. Anomalies are eventually identified by scrutinizing the discrepancies between target frames and their corresponding predictions. Several experiments conducted on public benchmarks affirm the efficacy of our hybrid framework, whether used as a standalone single-task approach or integrated as a branch in a multi-task approach. These experiments also underscore the advantages of merging vision transformers and ConvLSTMs for video anomaly detection. The implementation of our hybrid framework is available at https://github.com/SHENGUODONG19951126/ConvTTrans-ConvLSTM.

With the rise of deep learning (DL), our world braces for artificial intelligence (AI) in every edge device, creating an urgent need for edge-AI SoCs. This SoC hardware needs to support high throughput, reliable and secure AI processing at ultra-low power (ULP), with a very short time to market. With its strong legacy in edge solutions and open processing platforms, the EU is well-positioned to become a leader in this SoC market. However, this requires AI edge processing to become at least 100 times more energy-efficient, while offering sufficient flexibility and scalability to deal with AI as a fast-moving target. Since the design space of these complex SoCs is huge, advanced tooling is needed to make their design tractable. The CONVOLVE project (currently in Inital stage) addresses these roadblocks. It takes a holistic approach with innovations at all levels of the design hierarchy. Starting with an overview of SOTA DL processing support and our project methodology, this paper presents 8 important design choices largely impacting the energy efficiency and flexibility of DL hardware. Finding good solutions is key to making smart-edge computing a reality.

This paper presents a novel class of Graph-based Transform based on 3D convolutional neural networks (GBT-CNN) within the context of block-based predictive transform coding of imaging data. The proposed GBT-CNN uses a 3D convolutional neural network (3D-CNN) to predict the graph information needed to compute the transform and its inverse, thus reducing the signalling cost to reconstruct the data after transformation. The GBT-CNN outperforms the DCT and DCT /DST, which are commonly employed in current video codecs, in terms of the percentage of energy preserved by a subset of transform coefficients, the mean squared error of the reconstructed data, and the transform coding gain according to evaluations on several video frames and medical images.

Video anomaly detection is a challenging task because most anomalies are scarce and non-deterministic. Many approaches investigate the reconstruction difference between normal and abnormal patterns, but neglect that anomalies do not necessarily correspond to large reconstruction errors. To address this issue, we design a Convolutional LSTM Auto-Encoder prediction framework with enhanced spatiotemporal memory exchange using bi-directionalilty and a higher-order mechanism. The bi-directional structure promotes learning the temporal regularity through forward and backward predictions. The unique higher-order mechanism further strengthens spatial information interaction between the encoder and the decoder. Considering the limited receptive fields in Convolutional LSTMs, we also introduce an attention module to highlight informative features for prediction. Anomalies are eventually identified by comparing the frames with their corresponding predictions. Evaluations on three popular benchmarks show that our framework outperforms most existing prediction-based anomaly detection methods.

The task of pose invariant face recognition (PIFR) has recently seen important improvements by introducing generative adversarial networks (GANs). These GAN-based models synthesize a frontal face image from an image in any pose to boost recognition performance. Most of these methods, however, require the ground-truth frontal face image during optimization as they rely on supervised or semi-supervised learning. In this work, we introduce the self-supervised Frontalization and Rotation GAN (FRGAN), which can synthesize a frontal face image from a non-frontal face image. For self-supervision, the synthesized image is rotated back to the original pose based on reconstruction and adversarial losses. To improve performance, the FRGAN uses the Random Swap, a parameter-free data augmentation strategy that swaps key facial regions between the input image and its reconstructed version to force the generator to synthesize more realistic images. Our qualitative and quantitative experiments on benchmark datasets confirm the strong performance of the FRGAN compared to the state-of-the-art (SOTA).

State-of-the-art crowd counting models follow an encoder-decoder approach. Images are first processed by the encoder to extract features. Then, to account for perspective distortion, the highest-level feature map is fed to extra components to extract multiscale features, which are the input to the decoder to generate crowd densities. However, in these methods, features extracted at earlier stages during encoding are underutilised, and the multiscale modules can only capture a limited range of receptive fields, albeit with considerable computational cost. This paper proposes a novel crowd counting architecture (FusionCount), which exploits the adaptive fusion of a large majority of encoded features instead of relying on additional extraction components to obtain multiscale features. Thus, it can cover a more extensive scope of receptive field sizes and lower the computational cost. We also introduce a new channel reduction block, which can extract saliency information during decoding and further enhance the model’s performance. Experiments on two benchmark databases demonstrate that our model achieves state-of-the-art results with reduced computational complexity. PyTorch implementation of the model and weights trained on these two datasets are available at https://github.com/YimingMa/FusionCount.

Perception of auditory events is inherently multimodal relying on both audio and visual cues. A large number of existing multimodal approaches process each modality using modality-specific models and then fuse the embeddings to encode the joint information. In contrast, we employ heterogeneous graphs to explicitly capture the spatial and temporal relationships between the modalities and represent detailed information about the underlying signal. Using heterogeneous graph approaches to address the task of visually-aware acoustic event classification, which serves as a compact, efficient and scalable way to represent data in the form of graphs. Through heterogeneous graphs, we show efficiently modelling of intra- and inter-modality relationships both at spatial and temporal scales. Our model can easily be adapted to different scales of events through relevant hyperparameters. Experiments on AudioSet, a large benchmark, shows that our model achieves state-of-the-art performance. Our code is available at github.com/AmirSh15/VAED HeterGraph.

We introduce the problem of multi-camera trajectory forecasting (MCTF), which involves predicting the trajectory of a moving object across a network of cameras. While multi-camera setups are widespread for applications such as surveillance and traffic monitoring, existing trajectory forecasting methods typically focus on single-camera trajectory forecasting (SCTF), limiting their use for such applications. Furthermore, using a single camera limits the field-of-view available, making long-term trajectory forecasting impossible. We address these shortcomings of SCTF by developing an MCTF framework that simultaneously uses all estimated relative object locations from several viewpoints and predicts the object's future location in all possible viewpoints. Our framework follows a Which-When-Where approach that predicts in which camera(s) the objects appear and when and where within the camera views they appear. To this end, we propose the concept of trajectory tensors: a new technique to encode trajectories across multiple camera views and the associated uncertainties. We develop several encoder-decoder MCTF models for trajectory tensors and present extensive experiments on our own database (comprising 600 hours of video data from 15 camera views) created particularly for the MCTF task. Results show that our trajectory tensor models outperform coordinate trajectory-based MCTF models and existing SCTF methods adapted for MCTF.

With the increasing popularity of convolutional neural networks (CNNs), recent works on face-based age estimation employ these networks as the backbone. However, state-of-the-art CNN-based methods treat each facial region equally, thus entirely ignoring the importance of some facial patches that may contain rich age-specific information. In this paper, we propose a face-based age estimation framework, called Attention-based Dynamic Patch Fusion (ADPF). In ADPF, two separate CNNs are implemented, namely the AttentionNet and the FusionNet. The AttentionNet dynamically locates and ranks age-specific patches by employing a novel Ranking-guided Multi-Head Hybrid Attention (RMHHA) mechanism. The FusionNet uses the discovered patches along with the facial image to predict the age of the subject. Since the proposed RMHHA mechanism ranks the discovered patches based on their importance, the length of the learning path of each patch in the FusionNet is proportional to the amount of information it carries (the longer, the more important). ADPF also introduces a novel diversity loss to guide the training of the AttentionNet and reduce the overlap among patches so that the diverse and important patches are discovered. Through extensive experiments, we show that our proposed framework outperforms state-of-the-art methods on several age estimation benchmark datasets.

This paper introduces the GBT-NN, a novel class of Graph-based Transform within the context of block-based predictive transform coding using intra-prediction. The GBT-NNis constructed by learning a mapping function to map a graph Laplacian representing the covariance matrix of the current block. Our objective of learning such a mapping functionis to design a GBT that performs as well as the KLT without requiring to explicitly com-pute the covariance matrix for each residual block to be transformed. To avoid signallingany additional information required to compute the inverse GBT-NN, we also introduce acoding framework that uses a template-based prediction to predict residuals at the decoder. Evaluation results on several video frames and medical images, in terms of the percentageof preserved energy and mean square error, show that the GBT-NN can outperform the DST and DCT.

Methods based on machine learning (ML) have been recently proposed to improve upon traditional block-based intra-prediction algorithms in modern video codecs [1,2]. Their performance, however, depends on the amount, quality and relevance of the training data. Furthermore, they require signaling the learned parameters to the decoder, thus increasing compressed data volumes. In this work, six new prediction modes based on fully-connected neural networks (FC-NNs) are proposed that avoid the two aforementioned shortcomings. To do so, 1-layer FC-NNs are used, whose parameters are rened by overfitting on the data samples being predicted. This allows to replicate the parameter optimization process at the decoder under a lossless compression regime without requiring any additional side information. Each proposed ML-based mode is based on a 1-layer FC-NN that predicts a block of size k k in a column-wise or row-wise manner using as input a subset of the reference samples used by traditional intra-prediction. Each subset, which varies for each column or row, is computed by averaging a number of reference samples to reduce noise [3]. Experimental results based on several video frames indicate that the proposed ML-based modes are selected as the best modes between 11% and 93% of the times. (see Table 1). When used in a hybrid intra-prediction framework that also includes HEVC's modes, the proposed ML-based modes increase prediction accuracy by between 0.56 dB and 7.01 dB PSNR, with respect to using only HEVC's modes.

In perceptual image coding applications, the main objective is to decrease, as much as possible, Bits Per Pixel (BPP) while avoiding noticeable distortions in the reconstructed image. In this paper, we propose a novel perceptual image coding technique, named Perceptual Color Compression (PCC). PCC is based on a novel model related to Human Visual System (HVS) spectral sensitivity and CIELAB Just Noticeable Color Difference (JNCD). We utilize this modeling to capitalize on the inability of the HVS to perceptually differentiate photons in very similar wavelength bands (e.g., distinguishing very similar shades of a particular color or different colors that look similar). The proposed PCC technique can be used with RGB (4:4:4) image data of various bit depths and spatial resolutions. In the evaluations, we compare the proposed PCC technique with a set of reference methods including Versatile Video Coding (VVC) and High Efficiency Video Coding (HEVC) in addition to two other recently proposed algorithms. Our PCC method attains considerable BPP reductions compared with all four reference techniques including, on average, 52.6% BPP reductions compared with VVC (VVC in All Intra still image coding mode). Regarding image perceptual reconstruction quality, PCC achieves a score of SSIM ≥ 0.99 in all tests in addition to a score of MS-SSIM ≥ 0.99 in all but one test. Moreover, MOS = 5 is attained in 75% of subjective evaluation assessments conducted.

Video memorability is a cornerstone in social media platform analysis, as a highly memorable video is more likely to be noticed and shared. This paper proposes a new framework to fuse multi-modal information to predict the likelihood of remembering a video. The proposed framework relies on late fusion of text, visual and motion features. Specifically, two neural networks extract features from the captions describing the video’ s content; two ResNet models extract visual features from specific frames, and two 3DResNet models, combined with Fisher Vectors, extract features from the video’ s motion information. The extracted features are used to compute several memorability scores via Bayesian Ridge regression, which are then fused based on a greedy search of the optimal fusion parameters. Experiments demonstrate the superiority of the proposed framework on the MediaEval2019 dataset, outperforming the state-of-the-art.

In predictive transform video coding, optimal bit allocation and quantization parameter (QP) estimation are important to control the bit rate of blocks, frames and the whole sequence. Common solutions to this problem rely on trained models to approximate the rate-distortion (R-D) characteristics of the video content during coding. Moreover, these solutions are mainly targeted for natural content sequences, whose characteristics differ greatly from those of screen content (SC) sequences. In this article, we depart from such trained R-D models and propose a low-complexity RC method for SC sequences that leverages the availability of information about the R-D characteristics of previously coded blocks within a frame. Namely, our method first allocates bits at the frame- and block-levels based on their motion and texture characteristics. It then approximates the R-D and R-QP curves of each block by a set control points and random sample consensus (RANSAC). Finally, it computes the appropriate block-level QP values to attain a target bit rate with the minimum distortion possible. The proposed RC method is embedded into a standard High-Efficiency Video Coding (H.265/HEVC) encoder and evaluated on several SC sequences. Our results show that our method not only attains better R-D performance than that of H.265/HEVC and other methods designed for SC sequences but also attains a more constant and higher reconstruction quality on all frames.

The vanilla Generative Adversarial Networks (GANs) are commonly used to generate realistic images depicting aged and rejuvenated faces. However, the performance of such vanilla GANs in the age-oriented face synthesis task is often compromised by the mode collapse issue, which may produce poorly synthesized faces with indistinguishable visual variations. In addition, recent age-oriented face synthesis methods use the L1 or L2 constraint to preserve the identity information in synthesized faces, which implicitly limits the identity permanence capabilities when these constraints are associated with a trivial weighting factor. In this paper, we propose a method for the age-oriented face synthesis task that achieves high synthesis accuracy with strong identity permanence capabilities. Specifically, to achieve high synthesis accuracy, our method tackles the mode collapse issue with a novel Conditional Discriminator Pool, which consists of multiple discriminators, each targeting one particular age category. To achieve strong identity permanence capabilities, our method uses a novel Adversarial Triplet loss. This loss, which is based on the Triplet loss, adds a ranking operation to further pull the positive embedding towards the anchor embedding to significantly reduce intra-class variances in the feature space. Through extensive experiments, we show that our proposed method outperforms state-of-the-art methods in terms of synthesis accuracy and identity permanence capabilities, both qualitatively and quantitatively.

This paper proposes a rate control (RC) algorithm for intra-coded sequences (I-frames) within the context of block-based predictive transform coding that departs from using trained models to approximate the rate-distortion (R-D) characteristics of the video sequence. Our algorithm employs piecewise linear approximations of the rate-distortion (R-D) curve of a frame at the block-level. Specifically, it employs information about the rate and distortion of already compressed blocks within the current frame to linearly approximate the slope of the R-D curve of each block. The proposed algorithm is implemented in the High-Efficiency Video Coding (H.265/HEVC) standard and compared with its current RC algorithm, which is based on a trained model. Evaluations on a variety of intra-coded sequences show that the proposed RC algorithm not only attains the overall target bit rate more accurately than the RC algorithm used by H.265/HEVC algorithm but is also capable of encoding each I-frame at a more constant bit rate according to the overall bit budget.

Cyber-Physical Systems (CPS) are dynamic and reactive systems interacting with processes, environment and, sometimes, humans. They are often distributed with sensors and actuators, smart, adaptive, predictive and react in real-time. Indeed, as sight for human beings, image- and video-processing pipelines are a prime source for environmental information for systems allowing them to take better decisions according to what they see. Therefore, in FitOptiVis we are developing novel methods and tools to integrate complex image and video processing pipelines. FitOptiVis aims to deliver a reference architecture for describing and optimizing quality and resource management for imaging and video pipelines in CPS both at design- and run-time. The architecture is concretized in low-power, high-performance, smart components, and in methods and tools for combined design-time and run-time multi-objective optimization and adaptation within system and environment constraints.

We propose a computational framework for ranking images (group photos in particular) taken at the same event within a short time span. The ranking is expected to correspond with human perception of overall appeal of the images. We hypothesize and provide evidence through subjective analysis that the factors that appeal to humans are its emotional content, aesthetics and image quality. We propose a network which is an ensemble of three information channels, each predicting a score corresponding to one of three visual appeal factors. For group emotion estimation, we propose a convolutional neural network (CNN) based architecture for predicting group emotion from images. This new architecture enforces the network to put emphasis on the important regions in the images, and achieves comparable results to the state-of-the-art. Next, we develop a network for the image ranking task that combines group emotion, aesthetics and image quality scores. Owing to the unavailability of suitable databases, we created a new database of manually annotated group photos taken during various social events. We present experimental results on this database and other benchmark databases whenever available. Overall, our experiments show that the proposed framework can reliably predict the overall appeal of images with results closely corresponding to human ranking.

Understanding human actions from videos captured by drones is a challenging task in computer vision due to the unfamiliar viewpoints of individuals and changes in their size due to the camera's location and motion. This work proposes DroneCaps, a capsule network architecture for multi-label human action recognition (HAR) in videos captured by drones. DroneCaps uses features computed by 3D convolution neural networks plus a new set of features computed by a novel Binary Volume Comparison layer. All these features, in conjunction with the learning power of CapsNets, allow understanding and abstracting the different viewpoints and poses of the depicted individuals very efficiently, thus improving multi-label HAR. The evaluation of the DroneCaps architecture's performance for multi-label classification shows that it outperforms state-of-the-art methods on the Okutama-Action dataset.

Face recognition algorithms have improved significantly in recent years since the introduction of deep learning and the availability of large training datasets. However, their performance is still inadequate when the face pose varies as pose variation can dramatically increase intra-person variability. This work proposes a novel generative adversarial architecture called the Attention Selective Network (ASN) to address the problem of pose-invariant face recognition. The ASN introduces an efficient attention mechanism and a multi-part loss function to generate realistic-looking frontal face images from other face poses that can be used for recognizing faces under various poses. Thanks to the high quality of the synthesized images, the ASN achieves superior performance in terms of recognition rates compared to the state-of-the-art supervised methods.

Video anomaly detection is a challenging task not only because it involves solving many sub-tasks such as motion representation, object localization and action recognition, but also because it is commonly considered as an unsupervised learning problem that involves detecting outliers. Traditionally, solutions to this task have focused on the mapping between video frames and their low-dimensional features, while ignoring the spatial connections of those features. Recent solutions focus on analyzing these spatial connections by using hard clustering techniques, such as K-Means, or applying neural networks to map latent features to a general understanding, such as action attributes. In order to solve video anomaly in the latent feature space, we propose a deep probabilistic model to transfer this task into a density estimation problem where latent manifolds are generated by a deep denoising autoencoder and clustered by expectation maximization. Evaluations on several benchmarks datasets show the strengths of our model, achieving outstanding performance on challenging datasets.

This paper introduces the problem of multiple object forecasting (MOF), in which the goal is to predict future bounding boxes of tracked objects. In contrast to existing works on object trajectory forecasting which primarily consider the problem from a birds-eye perspective, we formulate the problem from an object-level perspective and call for the prediction of full object bounding boxes, rather than trajectories alone. Towards solving this task, we introduce the Citywalks dataset, which consists of over 200k high-resolution video frames. Citywalks comprises of footage recorded in 21 cities from 10 European countries in a variety of weather conditions and over 3.5k unique pedestrian trajectories. For evaluation, we adapt existing trajectory forecasting methods for MOF and confirm cross-dataset generalizability on the MOT-17 dataset without fine-tuning. Finally, we present STED, a novel encoder-decoder architecture for MOF. STED combines visual and temporal features to model both object-motion and ego-motion, and outperforms existing approaches for MOF. Code & dataset link: https://github.com/olly-styles/Multiple-Object-Forecasting

This paper introduces the GBT-L, a novel class of Graph-based Transform within the context of block-based predictive transform coding. The GBT-L is constructed using a 2D graph with unit edge weights and weighted self-loops in every vertex. The weighted selfloops are selected based on the residual values to be transformed. To avoid signalling any additional information required to compute the inverse GBT-L, we also introduce a coding framework that uses a template-based strategy to predict residual blocks in the pixel and residual domains. Evaluation results on several video frames and medical images, in terms of the percentage of preserved energy and mean square error, show that the GBT-L can outperform the DST, DCT and the Graph-based Separable Transform.

Capsule Networks (CapsNets) are recently introduced to overcome some of the shortcomings of traditional Convolutional Neural Networks (CNNs). CapsNets replace neurons in CNNs with vectors to retain spatial relationships among the features. In this paper, we propose a CapsNet architecture that employs individual video frames for human action recognition without explicitly extracting motion information. We also propose weight pooling to reduce the computational complexity and improve the classification accuracy by appropriately removing some of the extracted features. We show how the capsules of the proposed architecture can encode temporal information by using the spatial features extracted from several video frames. Compared with a traditional CNN of the same complexity, the proposed CapsNet improves action recognition performance by 12.11% and 22.29% on the KTH and UCF-sports datasets, respectively.

Reliable anticipation of pedestrian trajectory is imperative for the operation of autonomous vehicles and can significantly enhance the functionality of advanced driver assistance systems. While significant progress has been made in the field of pedestrian detection, forecasting pedestrian trajectories remains a challenging problem due to the unpredictable nature of pedestrians and the huge space of potentially useful features. In this work, we present a deep learning approach for pedestrian trajectory forecasting using a single vehicle-mounted camera. Deep learning models that have revolutionized other areas in computer vision have seen limited application to trajectory forecasting, in part due to the lack of richly annotated training data. We address the lack of training data by introducing a scalable machine annotation scheme that enables our model to be trained using a large dataset without human annotation. In addition, we propose Dynamic Trajectory Predictor (DTP), a model for forecasting pedestrian trajectory up to one second into the future. DTP is trained using both human and machine-annotated data, and anticipates dynamic motion that is not captured by linear models. Experimental evaluation confirms the benefits of the proposed model.

In this paper, we propose a compact and low-complexity binary feature descriptor for video analytics. Our binary descriptor encodes the motion information of a spatio-temporal support region into a low-dimensional binary string. The descriptor is based on a binning strategy and a construction that binarizes separately the horizontal and vertical motion components of the spatio-temporal support region. We pair our descriptor with a novel Fisher Vector (FV) scheme for binary data to project a set of binary features into a fixed length vector in order to evaluate the similarity between feature sets. We test the effectiveness of our binary feature descriptor with FVs for action recognition, which is one of the most challenging tasks in computer vision, as well as gait recognition and animal behavior clustering. Several experiments on the KTH, UCF50, UCF101, CASIA-B, and TIGdog datasets show that the proposed binary feature descriptor outperforms the state-of-the-art feature descriptors in terms of computational time and memory and storage requirements. When paired with FVs, the proposed feature descriptor attains a very competitive performance, outperforming several state-of-the-art feature descriptors and some methods based on convolutional neural networks.

Recently, deep learning frameworks have been shown to be successful and efficient in processing digital histology images for various detection and classification tasks. Among these tasks, cell detection and classification are key steps in many computer-assisted diagnosis systems. Traditionally, cell detection and classification is performed as a sequence of two consecutive steps by using two separate deep learning networks: one for detection and the other for classification. This strategy inevitably increases the computational complexity of the training stage. In this paper, we propose a synchronized deep autoencoder network for simultaneous detection and classification of cells in bone marrow histology images. The proposed network uses a single architecture to detect the positions of cells and classify the detected cells, in parallel. It uses a curve-support Gaussian model to compute probability maps that allow detecting irregularly shape cells precisely. Moreover, the network includes a novel neighborhood selection mechanism to boost the classification accuracy. We show that the performance of the proposed network is superior than traditional deep learning detection methods and very competitive compared to traditional deep learning classification networks. Runtime comparison also shows that our network requires less time to be trained.

The use of whole-slide images (WSIs) in pathology entails stringent storage and transmission requirements because of their huge dimensions. Therefore, image compression is an essential tool to enable efficient access to these data. In particular, color transforms are needed to exploit the very high degree of inter-component correlation and obtain competitive compression performance. Even though the state-of-the-art color transforms remove some redundancy, they disregard important details of the compression algorithm applied after the transform. Therefore, their coding performance is not optimal. We propose an optimization method called mosaic optimization for designing irreversible and reversible color transforms simultaneously optimized for any given WSI and the subsequent compression algorithm. Mosaic optimization is designed to attain reasonable computational complexity and enable continuous scanner operation. Exhaustive experimental results indicate that, for JPEG 2000 at identical compression ratios, the optimized transforms yield images more similar to the original than the other state-of-the-art transforms. Specifically, irreversible optimized transforms outperform the Karhunen-Loève Transform in terms of PSNR (up to 1.1 dB), the HDR-VDP-2 visual distortion metric (up to 3.8 dB), and the accuracy of computer-aided nuclei detection tasks (F1 score up to 0.04 higher). In addition, reversible optimized transforms achieve PSNR, HDR-VDP-2, and nuclei detection accuracy gains of up to 0.9 dB, 7.1 dB, and 0.025, respectively, when compared with the reversible color transform in lossy-to-lossless compression regimes.

Digital pathology images are multi giga-pixel color images that usually require large amounts of bandwidth to be transmitted and stored. Lossy compression using intra-prediction offers an attractive solution to reduce the storage and transmission requirements of these images. In this paper, we evaluate the performance of the Graph-based Transform within the context of block-based predictive transform coding. To this end, we introduce a novel framework that eliminates the need to signal graph information to the decoder to recover the coefficients. This is accomplished by computing the GBT based on predicted residual blocks, which are predicted by a modeling approach that employs only the reference samples and information about the prediction mode. Evaluation results, in terms of the energy preserved when a small percentage of the largest coefficients are used for reconstruction of several pathology images, show that the GBT can outperform the DST and DCT.

Millions of surveillance cameras are currently installed in public places around the world, making it necessary to intelligently analyse the acquired data to detect the occurrence of abnormal events. A vast number of methods to detect such events have been recently proposed; unfortunately, there is a lack of methods capable of detecting these events as frames are acquired, also known as online processing. In this paper, we present an online framework for video anomaly detection that employs binary features to encode motion information, and low-complexity probabilistic models for detection. Evaluation results on the popular UCSD dataset and on a recently introduced real-event video surveillance dataset show that our framework outperforms non-online and online methods.

The JCT - VC standardized Screen Content Coding (SCC) extension in the HEVC HM RExt + SCM reference codec offers an impressive coding efficiency performance when compared with HM RExt alone; however, it is not significantly perceptually optimized. For instance, it does not include advanced HVS-based perceptual coding methods, such as Just Noticeable Distortion (JND)-based spatiotemporal masking schemes. In this paper, we propose a novel JND-based perceptual video coding technique for HM RExt + SCM. The proposed method is designed to further improve the compression performance of HM RExt + SCM when applied to YCbCr 4:4:4 SC video data. In the proposed technique, luminance masking and chrominance masking are exploited to perceptually adjust the Quantization Step Size (QStep) at the Coding Block (CB) level. Compared with HM RExt 16.10 + SCM 8.0, the proposed method considerably reduces bitrates (Kbps), with a maximum reduction of 48.3%. In addition to this, the subjective evaluations reveal that SC-PAQ achieves visually lossless coding at very low bitrates.

This paper proposes a rate control (RC) algorithm for intra-coded sequences (I-frames) within the context of block-based predictive transform coding (PTC) that employs piecewise linear approximations of the rate-distortion (RD) curve of each frame. Specifically, it employs information about the rate (R) and distortion (D) of already compressed blocks within the current frame to linearly approximate the slope of the corresponding RD curve. The proposed algorithm is implemented in the High-Efficiency Video Coding (HEVC) standard and compared with the current HEVC RC algorithm, which is based on a trained rate-lambda (R-λ) model. Evaluations on a variety of intra-coded sequences show that the proposed RC algorithm not only attains the overall target bit rate more accurately than the current RC algorithm but is also capable of encoding each I-frame at a more constant bit rate according to the overall bit budget, thus avoiding high bit rate fluctuations across the sequence.

Nowadays, big imaging data are very common in many fields of study. As a result, detecting small objects in very large images is challenging and computationally demanding. Taking advantage of the intrinsic cumulative properties of the Fisher Score, we propose the Integral Fisher Score (IFS) for low-complexity and accurate object detection in big imaging data. The IFS, which is a multi-dimensional extension of the Integral Image, allows computing the Fisher Vector associated with a spatial region using only four operations. This considerably reduces the computational cost of searching for a small query object on a very large target image. Evaluations for the detection of small object on high-resolution HUB telescope and digital pathology images show that IFS attains a high accuracy with short processing times.

Convolutional Neural Networks (CNN) have been applied to age-related research as the core framework. Although faces are composed of numerous facial attributes, most works with CNNs still consider a face as a typical object and do not pay enough attention to facial regions that carry age-specific feature for this particular task. In this paper, we propose a novel CNN architecture called Fusion Network (Fusion-Net) to tackle the age estimation problem. Apart from the whole face image, the FusionNet successively takes several age-specific facial patches as part of the input to emphasize the age-specific features. Through experiments, we show that the FusionNet significantly outperforms other state-of-the-art models on the MORPH II benchmark.

Retrieving information from social networks is a first and primordial step in many data analysis fields such as Natural Language Processing and Machine Learning. Important data science tasks rely on historical data gathering for further predictive results. Recent works use public platforms for collecting public streams of information like Twitter API, which allows querying chronological tweets from periods no longer than three weeks. In this paper, we present Twitter Scrapy, a new methodology for collecting historical tweets from time periods of arbitrary duration using web scraping techniques that bypass Twitter API restrictions.

The conventional strategy to download images captured by satellites is to compress the data on board and then transmit them via the downlink. It often happens that the capacity of the downlink is too small to accommodate all the acquired data, so the images are trimmed and/or transmitted through lossy regimes. This paper introduces a coding system that increases the amount and quality of the downloaded imaging data. The main insight of this paper is to use both the uplink and the downlink to code the images. The uplink is employed to send reference information to the satellite so that the onboard coding system can achieve higher efficiency. This reference information is computed on the ground, possibly employing extensive data and computational resources. The proposed system is called dual link image coding. As it is devised in this paper, it is suitable for Earth observation satellites with polar orbits. Experimental results obtained for data sets acquired by the Landsat 8 satellite indicate significant coding gains with respect to conventional methods.

The increasing availability of digital pathology images has motivated the design of tools to foster multidisciplinary collaboration among researchers, pathologists, and computer scientists. Telepathology plays an important role in the development of collaborative tools, as it facilitates the transmission and access to pathology images by multiple users. However, the huge file size associated with pathology images usually prevents full exploitation of the collaborative telepathology system potential. Within this context, rate control (RC) is an important tool that allows meeting storage and bandwidth requirements by controlling the bit rate of the coded image. In this paper, we propose a novel graph-based RC algorithm with lossless region of interest (RoI) coding for pathology images. The algorithm, which is designed for block-based predictive transform coding methods, compresses the non-RoI in a lossy manner according to a target bit rate and the RoI in a lossless manner. It employs a graph where each node represents a constituent block of the image to be coded. By incorporating information about the coding cost similarities of blocks into the graph, a graph kernel is used to distribute a target bit budget among the non-RoI blocks. In order to increase RC accuracy, the algorithm uses a rate-lambda (R- ) model to approximate the slope of the rate-distortion curve of the non-RoI, and a graph-based approach to guarantee that the target bit rate is accurately attained. The algorithm is implemented in the High-Efficiency Video Coding standard and tested over a wide range of pathology images with multiple RoIs. Evaluation results show that it outperforms the other state-of-the-art methods designed for single images by very accurately attaining the target bit rate of the non-RoI.

We propose a novel Coding Unit (CU) level luma-chroma perceptually adaptive quantization technique for HEVC; we name this technique C-BAQ. C-BAQ is designed to improve upon the luma-only adaptive perceptual quantization method adopted by JCT-VC. This luma-only perceptual quantization technique, named AdaptiveQP, has demonstrated coding improvements compared with Uniform Reconstruction Quantization; it increases or decreases the Quantization Parameter (QP) of an entire 2N×2N CU based on the spatial activity of raw samples in the constituent luma Coding Block (CB). AdaptiveQP does not take into account the spatial activity of raw samples in the corresponding chroma Cb and chroma Cr CBs, however, which constitutes a significant shortcoming of the method. In contrast to AdaptiveQP, the proposed C-BAQ technique accounts for the spatial activity of the raw samples in both the luma CBs and also the chroma CBs when modifying the QP at the CU level. The primary objective with C-BAQ is as follows: to significantly decrease bitrates, in comparison with AdaptiveQP, without incurring a discernible loss of reconstruction quality.

The Pap test is a manual screening procedure that is used to detect the precursor lesions of cervical cancer by analyzing changes in nuclei and cytoplasms of cervical cells. Due to the sensitivity of the Pap test to intra- and inter-observer variability, automating the procedure using digital image analysis test is still an open problem. Within this context, segmentation of overlapping cervical cells is a key component to develop image analysis methods. In this paper, we propose a framework for segmenting the cytoplasm of each individual cell depicted within an image of overlapping cervical cells. The proposed framework uses a patch-based approach where a parametric active contour detects, on a patch-by-path basis, the cytoplasm boundary of each overlapping cell. The active contour within the patch deforms under the influence of Gradient Vector Flow (GVF) forces computed based on the local edges depicted in each patch region. Results show that the proposed framework achieves more accurate cytoplasm segmentation results compared to the current state-of-art methods.

Gait recognition can be performed without subject cooperation under harsh conditions, thus it is an important tool in forensic gait analysis, security control, and other commercial applications. One critical issue that prevents gait recognition systems from being widely accepted is the performance drop when the camera viewpoint varies between the registered templates and the query data. In this paper, we explore the potential of combining feature optimisers and representations learned by convolutional neural networks (CNN) to achieve efficient view-invariant gait recognition. The experimental results indicate that CNN learns highly discriminative representations across moderate view variations, and these representations can be further improved using view-invariant feature selectors, achieving a high matching accuracy across views.

Rate control (RC) is an important tool that helps to control the bit rate of the compressed media. Current HEVC implementations include an RC algorithm, based on a R - λ model, that does not consider the wide range of luma values depicted in HDR content. This paper proposes a multi-R - λ-model approach for RC in HEVC that accurately attains a target bit rate while improving the reconstruction quality of HDR content, especially for very bright regions. Results, in terms of the mPSNR and HDR-VDP-2 metrics, confirm the advantages of our approach compared to the current RC algorithm in HEVC.

There is an increasing consumer demand for high bit-depth 4:4:4 HD video data playback due to its superior perceptual visual quality compared with standard 8-bit subsampled 4:2:0 video data. Due to vast file sizes and associated bitrates, it is desirable to compress raw high bit-depth 4:4:4 HD video sequences as much as possible without incurring a discernible decrease in visual quality. In this paper, we propose a Coding Block (CB)-level perceptual video coding technique for HEVC named Full Color Perceptual Quantization (FCPQ). FCPQ is designed to adjust the Quantization Parameter (QP) at the CB level - i.e., the luma CB and the chroma Cb and Cr CBs - according to the variances of pixel data in each CB. FCPQ is based on the default perceptual quantization method in HEVC called AdaptiveQP. AdaptiveQP adjusts the QP of an entire 2N×2N CU based only on the spatial activity of the constituent luma CB. As demonstrated in this paper, by not accounting for the spatial activity of the constituent chroma CBs, as is the case with AdaptiveQP, coding performance can be significantly affected; this is because the variance of pixel data in a luma CB is notably different from the variances of pixel data in chroma Cb and Cr CBs. FCPQ, therefore, addresses this problem. In terms of coding performance, FCPQ achieves BD-Rate improvements of up to 39.5% (Y), 16% (Cb) and 29.9% (Cr) compared with AdaptiveQP.

The Consultative Committee for Space Data Systems (CCSDS) has issued several data compression standards devised to reduce the amount of data transmitted from satellites to ground stations. This paper introduces a contextual arithmetic encoder for on-board data compression. The proposed arithmetic encoder checks the causal adjacent neighbors, at most, to form the context and uses only bitwise operations to estimate the related probabilities. As a result, the encoder consumes few computational resources, making it suitable for on-board operation. Our coding approach is based on the prediction and mapping stages of CCSDS-123 lossless compression standard, an optional quantizer stage to yield lossless or near-lossless compression and our proposed arithmetic encoder. For both lossless and near-lossless compression, the achieved coding performance is superior to that of CCSDS-123, M-CALIC, and JPEG-LS. Taking into account only the entropy encoders, fixed-length codeword is slightly better than MQ and interleaved entropy coding.

Level set methods have been widely used to implement active contours for image segmentation applications due to their good boundary detection accuracy. In the context of medical image segmentation, weak edges and inhomogeneities remain important issues that may hinder the accuracy of any segmentation method based on active contours implemented using level set methods. This paper proposes a method based on active contours implemented using level set methods for segmentation of such medical images. The proposed method uses a level set evolution that is based on the minimization of an objective energy functional whose energy terms are weighted according to their relative importance in detecting boundaries. This relative importance is computed based on local edge features collected from the adjacent region located inside and outside of the evolving contour. The local edge features employed are the edge intensity and the degree of alignment between the image's gradient vector flow field and the evolving contour's normal. We evaluate the proposed method for segmentation of various regions in real MRI and CT slices, X-ray images, and ultra sound images. Evaluation results confirm the advantage of weighting energy forces using local edge features to reduce leakage. These results also show that the proposed method leads to more accurate boundary detection results than state-of-the-art edge-based level set segmentation methods, particularly around weak edges.

Over the past decade, video anomaly detection has been explored with remarkable results. However, research on methodologies suitable for online performance is still very limited. In this paper, we present an online framework for video anomaly detection. The key aspect of our framework is a compact set of highly descriptive features, which is extracted from a novel cell structure that helps to define support regions in a coarse-to-fine fashion. Based on the scene's activity, only a limited number of support regions are processed, thus limiting the size of the feature set. Specifically, we use foreground occupancy and optical flow features. The framework uses an inference mechanism that evaluates the compact feature set via Gaussian Mixture Models, Markov Chains, and Bag-of-Words in order to detect abnormal events. Our framework also considers the joint response of the models in the local spatio-temporal neighborhood to increase detection accuracy. We test our framework on popular existing data sets and on a new data set comprising a wide variety of realistic videos captured by surveillance cameras. This particular data set includes surveillance videos depicting criminal activities, car accidents, and other dangerous situations. Evaluation results show that our framework outperforms other online methods and attains a very competitive detection performance compared with state-of-the-art non-online methods.

The coding and transmission of the massive datasets captured by Earth Observation (EO) satellites is a critical issue in current missions. The conventional approach is to use compression on board the satellite to reduce the size of the captured images. This strategy exploits spatial and/or spectral redundancy to achieve compression. Another type of redundancy found in such data is the temporal redundancy between images of the same area that are captured at different instants of time. This type of redundancy is commonly not exploited because the required data and computing power are not available on board the satellite. This paper introduces a coding scheme for EO satellites able to exploit this redundancy. Contrary to traditional approaches, the proposed scheme employs both the downlink and the uplink of the satellite. Its main insight is to compute and code the temporal redundancy on the ground and transmit it to the satellite via the uplink. The satellite then uses this information to compress more efficiently the captured image. Experimental results for Landsat 8 images indicate that the proposed dual link image coding scheme can achieve higher coding performance than traditional systems for both lossless and lossy regimes.

Whole-slide images (WSIs) are high-resolution, 2D, color digital images that are becoming valuable tools for pathologists in clinical, research and formative scenarios. However, their massive size is hindering their widespread adoption. Even though lossy compression can effectively reduce compressed file sizes without affecting subsequent diagnoses, no lossy coding scheme tailored for WSIs has been described in the literature. In this paper, a novel strategy called OptimizeMCT is proposed to increase the lossy coding performance for this type of images. In particular, an optimization method is designed to find image-specific multi-component transforms (MCTs) that exploit the high inter-component correlation present in WSIs. Experimental evidence indicates that the transforms yielded by OptimizeMCT consistently attain better coding performance than the Karhunen-Loève Transform (KLT) for all tested lymphatic, pancreatic and renal WSIs. More specifically, images reconstructed at the same bitrate exhibit average PSNR values 2.85~dB higher for OptimizeMCT than for the KLT, with differences of up to 5.17 dB.

High Efficiency Video Coding (HEVC), and its standardized extensions including Scalable HEVC (SHVC), contains a feature to enable default or custom Quantization Matrices (QMs). The default intra QM technique in HEVC, originally designed for the JPEG standard [1, 2], is based on a perceptual Human Visual System (HVS) Contrast Sensitivity Function (CSF) approach and its associated Modulation Transfer Function (MTF) [3]. Visual Display Units (VDUs), capable of displaying video data at High Definition (HD) and Ultra HD (UHD) display resolutions, are frequently utilized in a vast variety of technological domains. Compression artifacts caused by quantization, which are not noticeable in low display resolution environments, are clearly visible on HD and UHD video data and VDUs.

Predicting the future actions of individuals from geospatial data has the potential to provide a basis for tailored services. This work presents the Predictive Context Tree (PCT), a new hierarchical classifier based on the Context Tree summary model [8]. The PCT is capable of predicting the future contexts and locations of individuals to provide a basis for understanding not only where a user will be, but also what type of activity they will be performing. Through a comparison to established techniques, this paper demonstrates the applicability of the PCT by showing increased accuracies for location prediction, and increased utility through context prediction.

The lossless compression of Whole Slide pathology Images (WSIs) using HEVC is investigated in this paper. Recently proposed intra-prediction algorithms based on differential pulse-code modulation (DPCM) and edge prediction provide significant bitrate improvements for a wide range of natural and screen content sequences, including WSIs. However, coding times remain relatively high due to the high number (35) of modes to be tested. In this paper, FastIn-tra, a novel method that requires testing only four modes is proposed. Among these four modes, FastIntra introduces a novel median edge predictor designed to accurately predict edges in different directionalities. Performance evaluations on various WSIs show average compression time reductions of 23.5% with important lossless coding improvements as compared to current block-wise intra-prediction and DPCM-based methods.

Lossy compression techniques based on multi-component transformation (MCT) can effectively enhance the storage and transmission of whole-slide images (WSIs) without adversely affecting subsequent diagnosis processes. Component transforms that are designed for other types of images or that do not take into account all aspects of the compression algorithm applied on the transformed components yield suboptimal coding performance. Recently, an MCT optimization framework adapted to the particularities of the input WSI and the following compression was proposed, yielding superior coding performance than the state of the art. However, its time complexity is too high for practical purposes. In this work FastOptimizeMCT, a fast version of this framework based on smart sampling of regions depicting tissue, is proposed. Exhaustive experimental evidence indicates that FastOptimizeMCT exhibits reasonable time complexity results-similar to that of scanning the WSIs- and coding performance that outperforms the KLT and the OST by 1.47 dB and 1.07 dB, respectively.

Inspired by the binary-based descriptors (e.g. LBP, ALOHA, FREAK, BRISK), we propose the 3D Binary Pair Differences (3DBPD) video descriptor for action recognition. By comparing several spatio-temporal sub-regions around interests points, our descriptor is a feature vector with a dimensionality of up to 30% smaller than that of existing state-of-the-art descriptors. We demonstrate the effectiveness of the 3DBPD descriptor for action recognition with a SVM classifier and a simple Bag Of Video Words (BOV) generated using k-means. The proposed descriptor has very competitive recognition rates compared to other state-of-the-art descriptors, with an outstanding performance in terms of speed. Additionally, the 3DBPD descriptor requires a small codebook compared to those required by existing BOV-based descriptors.

Image coding systems have been traditionally tailored for multiple instruction, multiple data (MIMD) computing. In general, they partition the (transformed) image in codeblocks that can be coded in the cores of MIMD-based processors. Each core executes a sequential flow of instructions to process the coefficients in the codeblock, independently and asynchronously from the others cores. Bitplane coding is a common strategy to code such data. Most of its mechanisms require sequential processing of the coefficients. The last years have seen the upraising of processing accelerators with enhanced computational performance and power efficiency whose architecture is mainly based on the single instruction, multiple data (SIMD) principle. SIMD computing refers to the execution of the same instruction to multiple data in a lockstep synchronous way. Unfortunately, current bitplane coding strategies cannot fully profit from such processors due to inherently sequential coding task. This paper presents bitplane image coding with parallel coefficient (BPC-PaCo) processing, a coding method that can process many coefficients within a codeblock in parallel and synchronously. To this end, the scanning order, the context formation, the probability model, and the arithmetic coder of the coding engine have been re-formulated. The experimental results suggest that the penalization in coding performance of BPC-PaCo with respect to the traditional strategies is almost negligible.

The lossless intra-prediction coding modality of the High Efficiency Video Coding standard provides high coding performance while allowing frame-by-frame basis access to the coded data. This is of interest in many professional applications, such as medical imaging, automotive vision, and digital preservation in libraries and archives. Various improvements to lossless intra-prediction coding have been proposed recently, most of them based on sample-wise prediction using differential pulse code modulation (DPCM). Other recent proposals aim at further reducing the energy of intra-predicted residual blocks. However, the energy reduction achieved is frequently minimal due to the difficulty of correctly predicting the sign and magnitude of residual values. In this paper, we pursue a novel approach to this energy-reduction problem using piecewise mapping (pwm) functions. In particular, we analyze the range of values in residual blocks and apply accordingly a pwm function to map specific residual values to unique lower values. We encode the appropriate parameters associated with the pwm functions at the encoder, so that the corresponding inverse pwm functions at the decoder can map values back to the same residual values. These residual values are then used to reconstruct the original signal. This mapping is, therefore, reversible and introduces no losses. We evaluate the pwm functions on 4 × 4 residual blocks computed after DPCM-based prediction for lossless coding of a variety of camera-captured and screen content sequences. Evaluation results show that the pwm functions can attain the maximum bitrate reductions of 5.54% and 28.33% for screen content material compared with DPCM-based and block-wise intra-prediction, respectively. Compared with intra-block copy, piecewise mapping can attain the maximum bit-rate reductions of 11.48% for a camera-captured material.

Exposing latent knowledge in geospatial trajectories has the potential to provide a better understanding of the movements of individuals and groups. Motivated by such a desire, this work presents the context tree , a new hierarchical data structure that summarises the context behind user actions in a single model. We propose a method for context tree construction that augments geospatial trajectories with land usage data to identify such contexts. Through evaluation of the construction method and analysis of the properties of generated context trees, we demonstrate the foundation for understanding and modelling behaviour afforded. Summarising user contexts into a single data structure gives easy access to information that would otherwise remain latent, providing the basis for better understanding and predicting the actions and behaviours of individuals and groups. Finally, we also present a method for pruning context trees for use in applications where it is desirable to reduce the size of the tree while retaining useful information.

Recent years have seen the upraising of a new type of processors strongly relying on the Single Instruction, Multiple Data (SIMD) architectural principle. The main idea behind SIMD computing is to apply a flow of instructions to multiple pieces of data in parallel and synchronously. This permits the execution of thousands of operations in parallel, achieving higher computational performance than with traditional Multiple Instruction, Multiple Data (MIMD) architectures. The level of parallelism required in SIMD computing can only be achieved in image coding systems via microscopic parallel strategies that code multiple coefficients in parallel. Until now, the only way to achieve microscopic parallelism in bit plane coding engines was by executing multiple coding passes in parallel. Such a strategy does not suit well SIMD computing because each thread executes different instructions. This paper introduces the first bit plane coding engine devised for the fine grain of parallelism required in SIMD computing. Its main insight is to allow parallel coefficient processing in a coding pass. Experimental tests show coding performance results similar to those of JPEG2000.

This paper proposes a rate control algorithm for lossless region of interest (RoI) coding in HEVC intra-coding. The algorithm is developed for digital pathology images and allows for random access to the data. Based on an input RoI mask, the algorithm first encodes the RoI losslessly. According to the bit rate spent on the RoI, it then encodes the background by using rate control in order to meet an overall target bit rate. In order to increase rate control accuracy, the algorithm uses an R-λ model to approximate the slope of the rate-distortion curve, and updates any related model parameters during the encoding process. Random access is attained by coding the data using independent tiles. Experimental results show that the proposed algorithm attains the overall bit rate very accurately while providing lossless reconstruction of the RoI.

The generation of screen content (SC) videos has considerably increased with the popularity of multimedia applications such as video conferencing, remote desktop sharing and remote education. SC videos, as opposed to natural video sequences, are challenging to compress due to the high number of sharp edges. This paper proposes the sample-based angular prediction with median and edge prediction (SAP-ME), a novel intra-prediction method for lossless coding of SC videos. SAP-ME incorporates the advantages of three types of sample-wise intra-prediction; namely, DPCM-based angular intra-prediction, median prediction and edge prediction. By using rate distortion optimization, SAP-ME selects among these three types of predictions to accurately predict directional patterns, smooth regions and sharp edges, with a low computational complexity. Evaluation results on a variety of SC videos show that SAP-ME attains average bit-rate reductions of 16.13% over current HEVC intra-prediction coding for lossless coding.

The High Efficiency Video Coding (HEVC) reference encoder includes a Uniform Reconstruction Quantization (URQ) method. This block level quantization technique does not take into account the importance of transform coefficients in terms of signal reconstruction. In this paper, we present a transform coefficient level quantization technique based on a soft thresholding approach, in which the quantization parameter (QP) of a transform coefficient is altered according to its importance in the signal reconstruction process and according to the overall energy of the corresponding Transform Block (TB). The proposed method attains important coding efficiency performance gains as measured by the Bjøntegaard Delta Bitrate metric (BD-Rate). In comparison with reference encoder HM 16, our technique produces both luma and chroma BD-Rate improvements in all JCT-VC test sequences, particularly for sequences in Class E using the Low Delay B and Low Delay P configurations, with BD-Rate reductions of up to 5.5% (Y), 13.2% (Cb) and 11.7% (Cr).

The amount of generated screen content sequences has recently increased due to the high popularity of multimedia services such as videoconferencing and cloud video gaming. Screen content sequences are characterized by a high number of sharp edges and repeating patterns, which are difficult to compress using current video coding tools. This paper proposes improvements to lossless compression of these sequences using HEVC intra-prediction. Specifically, the paper proposes SAP-G (sample-based angular intra-prediction with gradient-based edge prediction), which is a DPCM-based intra-prediction method that combines angular prediction with a low-complexity edge predictor and a DPCM-based DC predictor. The edge predictor, which replaces the current PLANAR mode, is based on the computation of gradients along four main directions using the adjacent pixels to the pixel to be predicted. Simulation results show that SAP-G can attain average bit-rate reductions of 15.44% over current HEVC intra-prediction for a variety of screen content sequences. Results also show that SAP-G outperforms other state-of-the-art DPCM-based intra-prediction methods without increasing encoding/decoding times.

This work defines an entropy-based measure aimed to establish the compression limits of wavelet-based image coding. This measure serves to appraise the efficiency of current codecs, determining whether there is margin for their improvement or not. Also, it may help to design new compression schemes that target a particular type of images and/or compression rates.

This works focuses on the High Efficiency Video Coding (HEVC) standard as a compression method to be potentially adopted by the Digital Imaging and Communications in Medicine (DICOM) standard. We are particularly interested in improving the lossless compression efficiency of the intra coding process for grayscale anatomical medical images. We focus on intra coding due to its low complexity and outstanding compression results, as well as the fact that it allows coding high-dimensional medical images on a slice-by-slice basis. This is especially advantageous for cases when only a small set of slices needs to be accessed without the need to decode the entire data set. Based on the characteristics of grayscale anatomical medical images, specifically their large amount of edge information and frequent number of patterns depicted on various directions, we propose improvements to HEVC intra coding based on sample-by-sample (SbS) differential pulse code modulation (DPCM) with equal displacements so the density of prediction modes is constant in all directions. Performance evaluations over MRI, CT and X-ray angiography sequences show that the proposed improvements outperform current HEVC lossless intra coding, achieving average coding gains of 6%.

The recent introduction of the High Efficiency Video Coding (HEVC) standard provides opportunities to improve medical image compression in picture archiving and communications systems. In this paper, we propose improvements to the HEVC intra coding process for lossless compression of grayscale anatomical medical images, which are characterized by their large amount of edges. Specifically, we propose alternative angular and planar prediction modes that are based on sample-wise differential pulse code modulation (DPCM) with an increased range of directionalities. We also propose an implementation of the DPCM decoding process that maintains the block-wise coding structure of HEVC. Evaluation results on various medical images show that the proposed DPCM modes efficiently predict the large amount of edges in these images achieving bit-rate savings of up to 15%.

Active contours, or snakes, have been widely used for image segmentation purposes. However, high noise sensitivity and poor performance over weak edges are the most acute issues that hinder the segmentation accuracy of these curves, particularly in medical images. In order to overcome these issues, we propose a novel external force that integrates gradient vector flow (GVF) field forces and balloon forces based on a weighting factor computed according to local image features. The proposed external force reduces noise sensitivity, improves performance over weak edges and allows initialization with a single manually selected point. We evaluate the proposed external force for segmentation of various regions on real MRI and CT slices. Evaluation results show that the proposed approach leads to more accurate segmentation than snakes using traditional external forces.

Context modeling is key in wavelet-based image coding schemes to achieve competitive coding performance. Commonly, context models are devised for a particular coding system and are employed for many different types of images. The aim of this work is to evaluate the suitability of three well-known context models for coding hyperspectral images, without focusing on a particular wavelet-based coding system. To do so, an entropy-based measure defined using the mechanisms utilized by modern image codecs is employed. The experimental results assess the appropriateness of the context models considering different coding rates and transform strategies. They reveal that some widely-used context models may not be as adequate as it is generally thought. The hints provided by this analysis may help to design simpler and more efficient wavelet-based codecs for hyperspectral images.

DNA micro arrays are state-of-the-art tools in biological and medical research. In this work, we discuss the suitability of lossy compression for DNA micro array images and highlight the necessity for a distortion metric to assess the loss of relevant information. We also propose one possible metric that considers the basic image features employed by most DNA micro array analysis techniques. Experimental results indicate that the proposed metric can identify and differentiate important and unimportant changes in DNA micro array images.

Summary form only given. X-ray angiographic (angio) images are widely used for identifying irregularities in the vascular system. Because of their high spatial resolution and the increasingly amount of X-ray angio images generated, compression of these images is becoming increasingly appealing. In this paper, we introduce a diagnostically lossless compression scheme for X-ray angio images. The coding scheme relies on a novel method based on ray casting and a-shapes for distinguishing the clinically relevant Region of Interest from the background. The background is then suppressed to increase data redundancy, allowing to achieve a higher coding performance. Experimental results suggest that the proposed scheme correctly identifies the Region of Interest in X-ray angio images and achieves more than 2 bits per pixel reduction in average as compared to the case of compression with no background suppression. Results are reported here for 20 out of 25 images compressed using various lossless compression methods.

X-ray angiography (angio) images are widely used to identify irregularities in the vascular system. Because of their high spatial resolution and the increasing amount of X-ray angio images generated, compression of these images is becoming paramount. In this paper, we propose a diagnostically lossless compression method based on automatic segmentation using ray-casting and α-shapes. The diagnostically relevant Region of Interest is separated from the background by exploiting the inherent symmetrical features of the image. The background-suppressed images are then losslessly encoded using DICOM-compliant lossless compression methods. Experimental results suggest that the proposed method correctly identifies the Region of Interest in X-ray angio images with an average segmentation accuracy of 98.4% and achieves more than 2 bits per pixel improvement on compression performance as compared to lossless compression with no background suppression.

We present an automatic system for planar 3D modeling of building interiors from point cloud data generated by range scanners. This is motivated by the observation that most building interiors may be modeled as a collection of planes representing ceilings, floors, walls and staircases. Our proposed system, which employs model-fitting and RANSAC, is capable of detecting large-scale architectural structures, such as ceilings and floors, as well as small-scale architectural structures, such as staircases. We experimentally validate our system on a number of challenging point clouds of real architectural scenes.

This paper presents a joint source/channel coding method in 3D-JPEG2000 for wireless transmission of 3D regions of interest (RoIs) in 3D medical images. The method provides unequal error protection by combining the error correction capabilities of channel codes with the 3D-JPEG2000 error resilience tools. It generates quality scalable and error-resilient code streams with 3D-RoI decoding capabilities. The robustness of the proposed method is evaluated over Rayleigh-fading channels with rate-compatible punctured turbo codes. Comparisons are made with the cases of equal error protection and unequal error protection. Evaluations results on real 3D medical images show a significant improvement in reconstruction quality of the received 3D-RoIs and 3D images, in terms of the average peak signal-to-noise ratio.

We propose a 3D medical image coding method with optimal channel protection for wireless transmission. The proposed method employs the 3D integer wavelet transform and a modified EBCOT with 3D contexts to create a scalable layered bit-stream. Optimal channel protection is attained by assigning protection bits to the different sections of the compressed bit-stream according to their mean energy content. The robustness of the proposed method is evaluated over a Rayleigh-fading channel with a concatenation of a cyclic redundancy check code and a rate-compatible convolutional code. Comparisons are made with the cases of equal channel protection and unequal channel protection. Simulation results show a significant improvement in reconstruction quality of the received 3D images.

We present a novel 3-D scalable compression method for medical images with optimized volume of interest (VOI) coding. The method is presented within the framework of interactive telemedicine applications, where different remote clients may access the compressed 3-D medical imaging data stored on a central server and request the transmission of different VOIs from an initial lossy to a final lossless representation. The method employs the 3-D integer wavelet transform and a modified EBCOT with 3-D contexts to create a scalable bit-stream. Optimized VOI coding is attained by an optimization technique that reorders the output bit-stream after encoding, so that those bits belonging to a VOI are decoded at the highest quality possible at any bit-rate, while allowing for the decoding of background information with peripherally increasing quality around the VOI. The bit-stream reordering procedure is based on a weighting model that incorporates the position of the VOI and the mean energy of the wavelet coefficients. The background information with peripherally increasing quality around the VOI allows for placement of the VOI into the context of the 3-D image. Performance evaluations based on real 3-D medical imaging data showed that the proposed method achieves a higher reconstruction quality, in terms of the peak signal-to-noise ratio, than that achieved by 3D-JPEG2000 with VOI coding, when using the MAXSHIFT and general scaling-based methods.

We recently proposed a symmetry-based scalable lossless compression method for 3D medical images using the 2D integer wavelet transform and the embedded block coder with optimized truncation (EBCOT). In this paper, we present two major contributions that enhance our early work: 1) a new block-based intra-band prediction method that exploits the global and local symmetries of the wavelet-transform sub-bands based on the main axis of symmetry as detected using the analytical Fourier-Mellin transform; and 2) a new inter-slice DPCM prediction method that exploits the correlation between slices. Performance evaluations on real 3D medical images show an average improvement of up to 17% in lossless compression ratios when compared to the state-of-the-art compression methods including 3D-JPEG2000, JPEG2000 and H.264 intra-coding.

We recently proposed a method for lossless compression of 4-D medical images based on the advanced video coding standard (H.264/AVC). In this paper, we present two major contributions that enhance our previous work for compression of functional MRI (fMRI) data: 1) a new multiframe motion compensation process that employs 4-D search, variable-size block matching, and bidirectional prediction; and 2) a new context-based adaptive binary arithmetic coder designed for lossless compression of the residual and motion vector data. We validate our method on real fMRI sequences of various resolutions and compare the performance to two state-of-the-art methods: 4D-JPEG2000 and H.264/AVC. Quantitative results demonstrate that our proposed technique significantly outperforms current state of the art with an average compression ratio improvement of 13%.

We propose a novel symmetry-based technique for scalable lossless compression of 3D medical image data. The proposed method employs the 2D integer wavelet transform to decorrelate the data and an intraband prediction method to reduce the energy of the sub-bands by exploiting the anatomical symmetries typically present in structural medical images. A modified version of the embedded block coder with optimized truncation (EBCOT), tailored according to the characteristics of the data, encodes the residual data generated after prediction to provide resolution and quality scalability. Performance evaluations on a wide range of real 3D medical images show an average improvement of 15% in lossless compression ratios when compared to other state-of-the art lossless compression methods that also provide resolution and quality scalability including 3D-JPEG2000, JPEG2000, and H.264/AVC intra-coding.

Dynamic volumetric (four dimensional- 4D) medical images are typically huge in file size and require a vast amount of resources for storage and transmission purposes. In this paper, we propose an efficient lossless compression method for 4D medical images that is based on a multi-frame motion compensation process employing a 4D search, variable block- sizes and bi-directional prediction. Data redundancies are reduced by recursively applying multi-frame motion compensation in the spatial and temporal dimensions. The proposed method also uses a novel differential coding algorithm to reduce redundancies in motion vectors and a new context-based adaptive binary arithmetic coder (CABAC) for compression of the residual data. Performance evaluations on real medical images of varying modality resulted in lossless compression ratios of up to 16:1.

This paper presents an efficient lossless compression method for 4-D medical images based on the advanced video coding scheme (H.264/AVC). The proposed method efficiently reduces data redundancies in all four dimensions by recursively applying multiframe motion compensation. Performance evaluations on real 4-D medical images of varying modalities including functional magnetic resonance show an improvement in compression efficiency of up to three times that of other state-of-the-art compression methods such as 3D-JPEG2000.

Four dimensional (4D) medical data are sequences of volumetric images captured in time. These data sets are typically very large in size and demand a great amount of resources for storage and transmission. In this paper, we present a lossless compression technique for 4D medical images which is based on the H.264/AVC video coding standard. Our lossless compression technique efficiently exploits spatial and temporal redundancies between 2D image slices and 3D images in 4D medical images and eliminates any concerns regarding the effects of compression on image quality for diagnostic purposes. Performance evaluations have shown that the proposed compression technique outperforms current 4D compression methods by 70%

Navigation of mobile robot in the configuration space requires the robot pose measurement for control algorithms implementation. Calibrated fixed cameras may be utilized to such measurements. A promising approach where calibration can be avoided is navigation in the image space. This leads to the concept of direct image based control where absolute measurement of the robot pose in the configuration space is obviated; only information extracted from visual information — and partial camera calibration— is utilized for guidance of the robot. This paper adopt this concept and the application of the velocity field control philosophy to visual guidance of wheeled mobile robots. The main feature of this approach is that the robot task is encoded by means of a specified velocity field in image space.