Dependency Graph (DG) structural pruning for enabling edge deployment of optimized Deep Learning (DL) models within the context of Non-Intrusive Load Monitoring (NILM) is introduced. DG pruning groups the DL model’s interdependencies between pruned and adjacent layers, enforcing structural consistency while simultaneously pruning all affected parameters. This addresses challenges in conventional structured pruning, related to maintaining model architecture integrity and performance when entire units are removed. By doing so, the DG structural pruning allows for a larger trade-off between pruning threshold and performance degradation when compared to its conventional counterpart. Experimental results using the Plegma dataset containing Mediterranean-based appliances, show that the DG structural pruning can achieve up to 90% model size, and up to 10× computational efficiency, calculated in terms of Multiply-and-Accumulate (MAC) units, all while exhibiting minimal performance degradation. The findings underscore the applicability of DG structural pruning to enable robust edge NILM solutions that enhance flexibility and energy efficiency, thereby facilitating broader adoption in real-world applications.

Abstract This document proposes a spatiotemporal deep learning model for ionospheric total electron content (TEC) modeling using global navigation satellite systems (GNSSs) observables. Data from dual-frequency GNSS receivers are used to compute the daily GNSS TEC timeseries. Usually, these timeseries are computed independently per GNSS permanent station, and the state-of-the art models proposed in literature exploit temporal characteristics of the timeseries and neglect any spatial dependencies and information from different stations. In our approach, we propose a practical solution for parallel processing of TEC timeseries and additional indicators from various adjacent stations to predict future VTEC values. We face the problem in both spatial and temporal dimensions adopting a graph neural network-based approach from the broader family of geometric deep learning. According to our proposed scheme, the different adjacent GNSS stations are structured in a graph and then, we apply the proposed temporal graph convolutional network called ION_TGNN. Our model predicts future vertical TEC (VTEC) values for all stations in a single run with mae error better than 1.0 TECU. Comparisons with state-of-the art models show the superiority of the proposed method in terms of performance but also in terms of computational cost during training and test phases.

Segmentation of remote sensing data is a critical task in various environmental and geospatial applications. However, the presence of missing modalities, such as the Digital Elevation Model (DEM), poses significant challenges to achieving high segmentation accuracy. In this paper, we propose a novel approach for addressing this issue using Prototype Knowledge Distillation. Our methodology involves training a teacher model with access to all available modalities, including DEM, to generate high-quality segmentation results. Subsequently, we train a student model that performs segmentation without access to the DEM modality. The teacher model distills its learned knowledge into the student model through prototype representations, ensuring that the student model can achieve comparable or better performance despite the missing modality. Experimental results demonstrate the effectiveness of our approach, showing significant improvements in segmentation accuracy over baseline methods that do not utilize knowledge distillation. This work paves the way for robust segmentation of remote sensing data in scenarios where certain modalities are unavailable, enhancing the applicability and reliability of remote sensing analyses.

Climate change and extreme weather events diversly affect riverine areas. Since decades river monitoring techniques in combination with satellite images, are used to draw meaningful conclusions about rivers e.g., river banklines, islets, flooded areas etc. However, the majority of the existing methods rely on manual techniques especially for the analysis of river islets. In this paper we propose an automatic method to find the islets of a river and analyze them. Firstly, the proposed method uses deep learning and GIS algorithms, applied on satellite images, to find the islets. Then, the detected islets are stored as GIS layers in multiple geometries i.e., polygons, lines and points. Finally, the created layers are used to classify the islets into the "missing", "existing" and "new" ones resulting in an informative output for the users. In general, the proposed approach gives promising results and especially for the vectorization part which achieves an mIoU 94.9%.

Predicting channel excess attenuation is of utmost importance for the current satellite networks design. However, the adoption of data-driven models is hampered by the absence of reliable satellite propagation measurements in real conditions. In this framework, ESA initiated in 2015 a dedicated project for a large-scale measurement campaign using the Alphasat SCIEX Ka/Q band signals (ASALASCA). Here, we propose a data-driven model to predict in excess attenuation in the next future time-steps, by exploiting the real measurements (excess attenuation and rainfall rate) from the two experimental stations in Greece by NTUA operating at Ka band (19.704GHz).

An explainable AI framework for flood detection in SAR images is proposed. Compact encoder-decoder CNNs are used within the framework to achieve flood segmentation, with their output results fed to a Grad-CAM explainer so as to introduce trustworthiness to a stakeholder from naive thresholding selection during post-processing steps. The proposed framework is evaluated on the ETCi 2021 dataset using three different CNNs, resulting in more than 97% accuracy, while descriptive statistics on the Jaccard score are used to indicate the CNNs improper generalization towards the dataset. Edge cases highlight the importance of using Grad-CAM in complement with the CNN when the latter struggles to segment small regions due to thresholding.

EV charging optimization with utilization of decentralized renewable energy resources can be seen as a promising tool towards domestic EV fleet decarbonization. However, optimization is usually conducted on an individual household level, and community-driven approaches with shared resources are heavily underexplored. In this work, a community-driven smart EV charging optimization scheme with Multi-Agent Deep Reinforcement Learning is proposed. Compared to existing single-agent approaches, the employment of a multi-agent one allows for concurrent EV charging optimization for each household within a community, deployed in a centralized, shared, energy management system, while utilizing a community-owned solar photovoltaic (PV) panel. Our approach results in reduced cost barriers for domestic EV owners that desire to optimize their charging profile, as it eliminates the investment on an individual PV panel and energy management system. Experimental results on the Pecan Street dataset validate the effectiveness of our approach compared to individual household optimization, resulting in cost savings up to 17.65%, increase in PV power utilization of up to 133.10%, as well as network stress reduction of up to 18.75%.

Non-intrusive load monitoring (NILM) is considered an efficient approach to infer the consumption pattern of household appliances from the aggregate consumption signal. Continual adaptability is an important aspect of practical NILM applications, as they usually require frequent post-deployment maintenance to deal with non-stationary appliances’ data distributions. However, in most approaches, the trained deep learning model weights remain static, potentially neglecting valuable information that can be used for further model training. This work alleviates the aforementioned limitation by introducing ContiNILM, a continual learning scheme for NILM to build robust models that track environmental/seasonal alterations with direct impact on several appliances’ operation. In our approach, model weights do not remain static, but utilize additional training data to further improve the disaggregation performance. A novel mechanism is proposed that determines whether new incoming samples would be beneficial for model training and alleviates the risk of "forgetting" previously learned knowledge. Experimental results demonstrate the efficiency of the proposed approach.

This work proposes an interpretable Deep Learning framework utilizing Vision Transformers (ViT) for the classification of remote sensing images into land use and land cover (LULC) classes. It uses the Shapley Additive Explanations (SHAP) values to achieve two-stage explanations: 1) bandwise feature importance per class, showing which band assists the prediction of each class and 2) spatial-wise feature understanding, explaining which embedded patches per band affected the network's performance. Experimental results on the EuroSAT dataset demonstrate the ViT's accurate classification with an overall accuracy 96.86 %, offering improved results when compared to popular CNN models. Heatmaps in each one of the dataset's existing classes highlight the effectiveness of the proposed framework in the band explanation and the feature importance.

An interpretable deep learning framework for land use and land cover classification (LULC) in remote sensing using SHAP is introduced. It utilizes a compact CNN model for the classification of satellite images and then feeds the results to a SHAP deep explainer so as to strengthen the classification results. The proposed framework is applied to Sentinel-2 satellite images containing 27000 images of pixel size 64 × 64 and operates on three-band combinations, reducing the model’s input data by 77% considering that 13 channels are available, while at the same time investigating on how different spectrum bands affect predictions on the dataset’s classes. Experimental results on the EuroSAT dataset demonstrate the CNN’s accurate classification with an overall accuracy of 94.72%, whereas the classification accuracy on three-band combinations on each of the dataset’s classes highlights its improvement when compared to standard approaches with larger number of trainable parameters. The SHAP explainable results of the proposed framework shield the network’s predictions by showing correlation values that are relevant to the predicted class, thereby improving the classifications occurring in urban and rural areas with different land uses in the same scene.

Understanding the dynamics of deforestation and land uses of neighboring areas is of vital importance for the design and development of appropriate forest conservation and management policies. In this article, we approach deforestation as a multilabel classification (MLC) problem in an endeavor to capture the various relevant land uses from satellite images. To this end, we propose a multilabel vision transformer model, ForestViT, which leverages the benefits of the self-attention mechanism, obviating any convolution operations involved in commonly used deep learning models utilized for deforestation detection. Experimental evaluation in open satellite imagery datasets yields promising results in the case of MLC, particularly for imbalanced classes, and indicates ForestViT's superiority compared with well-established convolutional structures (ResNET, VGG, DenseNet, and ModileNet neural networks). This superiority is more evident for minority classes.

In Cultural Heritage, hyperspectral images are commonly used since they provide extended information regarding the optical properties of materials. Thus, the processing of such high-dimensional data becomes challenging from the perspective of machine learning techniques to be applied. In this paper, we propose a Rank-R tensor-based learning model to identify and classify material defects on Cultural Heritage monuments. In contrast to conventional deep learning approaches, the proposed high order tensor-based learning demonstrates greater accuracy and robustness against over-fitting. Experimental results on real-world data from UNESCO protected areas indicate the superiority of the proposed scheme compared to conventional deep learning models.

Links between environmental conditions (e.g., meteo-rological factors and air quality) and COVID-19 infection/mortality have been reported worldwide. However, the existing statistical frameworks are insufficient to investigate the factors that increase the risk for COVID-19 in urban areas. In this paper, we extend the concept of machine learning-based predictive modelling for COVID-19 spread, proposing an explainable AI approach in order to i) prioritize the risk factors, ii) define the interconnections between them and iii) detect positive or negative influence of the factors with respect to COVID-19 morbidity and mortality.

Modeling ionospheric variability throughout a proper total electron content (TEC) parameter estimation is a demanding, however, crucial, process for achieving better accuracy and rapid convergence in precise point positioning (PPP). In particular, the single-frequency PPP (SF-PPP) method lacks accuracy due to the difficulty of dealing adequately with the ionospheric error sources. In order to apply ionosphere corrections in techniques, such as SF-PPP, external information of global ionosphere maps (GIMs) is crucial. In this article, we propose a deep learning model to efficiently predict TEC values and to replace the GIM-derived data that inherently have a global character, with equal or better in accuracy regional ones. The proposed model is suitable for predicting the ionosphere delay at different locations of receiver stations. The model is tested during different periods of time, under different solar and geomagnetic conditions and for stations in various latitudes, providing robust estimations of the ionospheric activity at the regional level. Our proposed model is a hybrid model comprising of a 1-D convolutional layer used for the optimal feature extraction and stacked recurrent layers used for temporal time series modeling. Thus, the model achieves good performance in TEC modeling compared to other state-of-the-art methods.

The problem of separating the household aggregated power signal into its additive sub-components, called energy (power) disaggregation or Non-Intrusive Load Monitoring (NILM) can play an instrumental role as a driver towards consumer energy consumption awareness and behavioral change. In this paper, we propose EnerGAN++, an adversarially trained model for robust energy disaggregation. We propose a unified autoencoder (AE) and GAN architecture, in which the AE achieves a non-linear power signal source separation. The discriminator performs sequence classification, using a recurrent CNN to handle the temporal dynamics of an appliance energy consumption time series. Experimental results indicate the proposed method’s superiority compared to the state of the art.

Ionospheric variability and disturbances can affect technologies in space and on Earth, disrupting satellite operations, communications networks, and navigation systems. The availability of numerous satellites deployed by GPS, GLONASS, Galileo, BeiDou navigation systems allows continuous monitoring of the Earth's ionosphere using measurements from these satellites. Here, we scrutinize the effectiveness and efficiency of a convolutional enriched recurrent neural network for spatio-temporal VTEC prediction. In our analysis, we have chosen different years under different solar and geomagnetic activity. We test our models for different days and at various latitudes to see model's response in cases of high ionosphere activity. Our experiments indicate that the proposed combined deep CNN-GRU model is capable of providing an accurate prediction of TEC values even in intense conditions.

An efficient, appliance-level approach for energy disaggregation, exploiting the benefits of Generative Adversarial Networks, is presented. The concept of adversarial training supports the creation of fine tuned dissagregators, which produce more detailed load estimations for a specific appliance, compared to state of the art deep learning models. The Generator and Discriminator of the model are appropriately adapted to fit the particularities of NILM problem, whereas a Seeder component is added to provide encoded compact input vectors to the Generator. The experimental evaluation against state of the art techniques indicates promising results.

In this paper, we present a deep learning scheme for classification of choreographic primitives from RGB images. The proposed framework combines the representational power of feature maps, extracted by Convolutional Neural Networks, with the long-term dependency modeling capabilities of Long Short-Term Memory recurrent neural networks. In addition, it uses AutoRegressive and Moving Average (ARMA) filter into the convolutionally enriched LSTM filter to face dance dynamic characteristics. Finally, an adaptive weight updating strategy is introduced for improving classification modeling performance The framework is used for the recognition of dance primitives (basic dance postures) and is experimentally validated with real-world sequences of traditional Greek folk dances.

Recent advances in sensing technologies require the design and development of pattern recognition models capable of processing spatiotemporal data efficiently. In this study, we propose a spatially and temporally aware tensor-based neural network for human pose classification using three-dimensional skeleton data. Our model employs three novel components. First, an input layer capable of constructing highly discriminative spatiotemporal features. Second, a tensor fusion operation that produces compact yet rich representations of the data, and third, a tensor-based neural network that processes data representations in their original tensor form. Our model is end-to-end trainable and characterized by a small number of trainable parameters making it suitable for problems where the annotated data is limited. Experimental evaluation of the proposed model indicates that it can achieve state-of-the-art performance.

Detection of unusual behavior is an important topic in signal and image processing. Because of the topic's complexity, addressing it as a solely RGB video analysis problem raises significant challenges. This has resulted in approaches that aim at exploiting different data modalities that can overcome the inherent restrictions of unimodal techniques. Moreover, the classification outcome of such approaches is affected not only by the input data, but also by previous classification history. To this end, this paper introduces a novel deep-NARMA filter that extends a typical CNN architecture, and endows it with autoregressive moving average behavior. In addition, it incorporates a data fusion framework that supplements RGB video streams, with thermal capturing and information about the distortion of WiFi signal reflectance. Experimental results indicate a better performance compared to conventional as well as deep learning approaches.

In this paper, a Bayesian-optimized bidirectional Long Short -Term Memory (LSTM) method for energy disaggregation, is introduced. Energy disaggregation, or Non-Intrusive Load Monitoring (NILM), is a process aiming to identify the individual contribution of appliances in the aggregate electricity load. The proposed model, Bayes-BiLSTM, is structured in a modular way to address multi-dimensionality issues that arise when the number of appliances increase. In addition, a non-causal model is introduced in order to tackle with inherent structure, characterizing the operation of multi-state appliances. Furthermore, a Bayesian-optimized framework is introduced to select the best configuration of the proposed regression model, thus increasing performance. Experimental results indicate the proposed method's superiority, compared to the current state-of-the-art.

In this work we propose a method for reducing the dimensionality of tensor objects in a binary classification framework. The proposed Common Mode Patterns method takes into consideration the labels' information, and ensures that tensor objects that belong to different classes do not share common features after the reduction of their dimensionality. We experimentally validate the proposed supervised subspace learning technique and compared it against Multilinear Principal Component Analysis using a publicly available hyper-spectral imaging dataset. Experimental results indicate that the proposed CMP method can efficiently reduce the dimensionality of tensor objects, while, at the same time, increasing the inter-class separability.

In this paper, we present a general tensor-based nonlinear classifier, the Rank-R Feedforward Neural Network (FNN). In the proposed model, which is an extension of the Rank-1 FNN classifier, the network weights are constrained to satisfy a rank-R Canonical Polyadic Decomposition. By allowing a rank-R, instead of a rank-1, Canonical Polyadic Decomposition of the weights, the learning capacity of the model can be increased, which contributes to avoiding underfitting problems. The effectiveness of the proposed model is scrutinized on a hyperspectral image classification experimental setting, since hyperspectral data can naturally be represented as tensor objects. Performance evaluation results indicate that the proposed model outperforms other state-of-the-art models, including deep learning ones, especially in cases where the number of available training samples is small.

Performing arts is an essential aspect of Intangible Cultural Heritage (ICH), requiring tools for its modelling. In this paper, we introduce a Bayesian Optimized Bi-directional LSTM model, called BOBi-LSTM, that automatically estimates dancers' poses through 3D skeleton data processing. Bi-directionality models non-causal relationships occurred in a dance performance, in the sense that future dancer's steps depend on previous/current steps. Additionally, long-range dependence correlates choreographic primitives on a long time (memory) window. To model the aforementioned principles, we modify the conventional LSTM networks under a Bayesian Optimized framework in order to define the best network structure. Experimental results and comparisons for different types of dances are given to showcase how the proposed BOBi-LSTM out-performs traditional classifiers.

Deep learning paradigm has been shown to be a very efficient classification framework for many application scenarios, including the analysis of Light Detection and Ranging (LiDAR) data for building detection. In fact, deep learning acts as a set of mathematical transformations, encoding the raw input data into appropriate forms of representations that maximize the classification performance. However, it is clear that mathematical computations alone, even highly nonlinear, are not adequate to model the physical properties of a problem, distinguishing, for example, the building structures from vegetation. In this letter, we address this difficulty by augmenting the raw LiDAR data with features coming from a physical interpretation of the information. Then, we exploit a deep learning paradigm based on a convolutional neural network model to find out the best input representations suitable for the classification. As test sites, three complex urban study areas with various kinds of building structures through the LiDAR data set of Vaihingen, Germany were selected. Our method has been evaluated in the context of “ISPRS Test Project on Urban Classification and 3-D Building Reconstruction.” Comparisons with traditional methods, such as artificial neural networks and support vector machine-based classifiers, indicate the outperformance of the proposed approach in terms of robustness and efficiency.

In this paper we propose a tensor-based nonlinear model for high-order data classification. The advantages of the proposed scheme are that (i) it significantly reduces the number of weight parameters, and hence of required training samples, and (ii) it retains the spatial structure of the input samples. The proposed model, called Rank-1 FNN, is based on a modification of a feedforward neural network (FNN), such that its weights satisfy the rank-1 canonical decomposition. We also introduce a new learning algorithm to train the model, and we evaluate the Rank-1 FNN on third-order hyperspectral data. Experimental results and comparisons indicate that the proposed model outperforms state of the art classification methods, including deep learning based ones, especially in cases with small numbers of available training samples.

A computer vision module is proposed for crack detection in tunnels, a challenging process due to the low visibility, the curvature from, and the structures of the cracks which, though being very narrow in width, they are very deep. Our system is embedded on a robot which surveys tunnels in real-time as it is moving in the infrastructure. Initially, a Convolutional Neural Network is employed to detect the cracks which, however, yields only approximate regions due to the great complexity of the scene. Then, a combined fuzzy spectral clustering is then introduced to refine the detected crack regions exploiting spatial and orientation coherency. The algorithms have been tested in real-life tunnels in Egnatia Highway. Our scheme yields high detection accuracy than existing methods and the capacity of the robot to touch the crack to allow in-situ measurements within a precision of 2-3cm in a tunnel of 7m height.

In this paper, we propose a novel method to extract representative instances from choreographic sequences of 3D human motion data. The proposed key-frame extraction method implements a hierarchical scheme that exploits spatio-temporal variations of the dance movement features. The method is based on a hierarchical adaptation of the sparse modeling for representative selection algorithm (SMRS). Leveraging a joint -centric distance metric, summaries are provided at variable levels of granularity depending on the richness and complexity of the visual content at different sequence segments. The proposed method can contribute to addressing the need of organizing, indexing, archiving, retrieving and analyzing intangible (in this case, dance-related) cultural content in a tractable fashion and with lower computational and storage resource requirements. The approach is evaluated on real-world dance sequences, as well as on theatrical kinesiology datasets (available by Carnegie Mellon University). Comparisons with traditional video summarization methods show that the proposed hierarchical spatio- temporal decomposition scheme achieves promising results.

Capturing, documenting and storing Intangible Cultural Heritage content has been recently enabled at unprecedented volume and quality levels through a variety of sensors and devices. When it comes to the performing arts, and mainly dance and kinesiology, the massive amounts of RGB-D and 3D skeleton data produced by video and motion capture devices the huge number of different types of existing dances and variations thereof, dictate the need for organizing, indexing, archiving, retrieving and analyzing dance-related cultural content in a tractable fashion and with lower computational and storage resource requirements. In this context, we present a novel framework based on kinematics modeling for the extraction of salient 3D human motion data from real-world choreographic sequences. Two approaches are proposed: a clustering-based method for the selection of the basic primitives of a choreography, and a kinematics-based method that generates meaningful summaries at hierarchical levels of granularity. The dance summarization framework has been successfully validated and evaluated with two real-world datasets and with the participation of dance professionals and domain experts.

Detection of moving objects in videos is a crucial step toward successful surveillance and monitoring applications. A key component for such tasks is called background subtraction and tries to extract regions of interest from the image background for further processing or action. For this reason, its accuracy and real-time performance are of great significance. Although effective background subtraction methods have been proposed, only a few of them take into consideration the special characteristics of thermal imagery. In this paper, we propose a background subtraction scheme, which models the thermal responses of each pixel as a mixture of Gaussians with unknown number of components. Following a Bayesian approach, our method automatically estimates the mixture structure, while simultaneously it avoids over-/underfitting. The pixel density estimate is followed by an efficient and highly accurate updating mechanism, which permits our system to be automatically adapted to dynamically changing operation conditions. We propose a reference implementation of our method in reconfigurable hardware achieving both adequate performance and low-power consumption. Adopting a high-level synthesis design and demanding floating point arithmetic operations are mapped in reconfigurable hardware, demonstrating fast prototyping and on-field customization at the same time.

In this paper, we present tensor-based linear and nonlinear models for hyperspectral data classification and analysis. By exploiting the principles of tensor algebra, we introduce new classification architectures, the weight parameters of which satisfy the rank-1 canonical decomposition property. Then, we propose learning algorithms to train both linear and nonlinear classifiers. The advantages of the proposed classification approach are that: 1) it significantly reduces the number of weight parameters required to train the model (and thus the respective number of training samples); 2) it provides a physical interpretation of model coefficients on the classification output; and 3) it retains the spatial and spectral coherency of the input samples. The linear tensor-based model exploits the principles of logistic regression, assuming the rank-1 canonical decomposition property among its weights. For the nonlinear classifier, we propose a modification of a feedforward neural network (FNN), called rank-1 FNN, since its weights satisfy again the rank-1 canonical decomposition property. An appropriate learning algorithm is also proposed to train the network. Experimental results and comparisons with state-of-the-art classification methods, either linear (e.g., linear support vector machine) or nonlinear (e.g., deep learning), indicate the outperformance of the proposed scheme, especially in the cases where a small number of training samples is available.

We consider the fully automated behavior understanding through visual cues in industrial environments. In contrast to most existing work, which relies on domain knowledge to construct complex handcrafted features from inputs, we exploit a Convolutional Neural Network (CNN), which is a type of deep model and can act directly on the raw inputs, to automate the process of feature construction. Although such models are limited to handle still 2D inputs, in this paper we appropriately transform video input to incorporate temporal information into each frame. This way our model hierarchically constructs features from both spatial and temporal dimensions. We apply our model in real-world environment, on data taken from Nissan factory, and it achieves superior performance without relying on handcrafted features.

The millions of tweets submitted daily overwhelm users who find it difficult to identify content of interest revealing the need for event detection algorithms in Twitter. Such algorithms are proposed in this paper covering both short (identifying what is currently happening) and long term periods (reviewing the most salient recently submitted events). For both scenarios, we propose fuzzy represented and timely evolved tweet-based theoretic information metrics to model Twitter dynamics. The Riemannian distance is also exploited with respect to words' signatures to minimize temporal effects due to submission delays. Events are detected through a multiassignment graph partitioning algorithm that: 1) optimally retains maximum coherence within a cluster and 2) while allowing a word to belong to several clusters (events). Experimental results on real-life data demonstrate that our approach outperforms other methods.

The Feed In Tariff policy (FIT) used for accelerating renewable energy investments cannot be retained as a sustainable business model for the future smart energy grid. It is also evident that the current centralized electricity market prevents small or very small energy producers, who usually generate energy by renewable means, to participate. In this paper, addressing the aforementioned problems, at first, we present a decentralized architecture (Virtual DER Clusters), where small Distributed Energy Sources (DERs) are united in coalitions, each participating in the market as a single entity. Then, efficient clustering algorithms are proposed based on a min-max optimization policy in order to dynamically derive the cluster that best satisfies coalition's goals. Maximization in the sense of increasing as much as possible the profits of small-scale energy producers. Minimization in the sense of creating dynamically most competitive clusters. In this paper, three clustering policy schemes are discussed, each presenting different advantages with respect to the contradictory benefits between DERs and power utilities. From the examined policies, a fair sharing allocation scheme seems to be a good compensator between the electricity market and the small-scale players.

Spectral observations along the spectrum in many narrow spectral bands through hyperspectral imaging provides valuable information towards material and object recognition, which can be consider as a classification task. Most of the existing studies and research efforts are following the conventional pattern recognition paradigm, which is based on the construction of complex handcrafted features. However, it is rarely known which features are important for the problem at hand. In contrast to these approaches, we propose a deep learning based classification method that hierarchically constructs high-level features in an automated way. Our method exploits a Convolutional Neural Network to encode pixels' spectral and spatial information and a Multi-Layer Perceptron to conduct the classification task. Experimental results and quantitative validation on widely used datasets showcasing the potential of the developed approach for accurate hyperspectral data classification.

The electricity market for Renewable Energy (RE) Sources (RES) has to be transformed into a market that is more competitive and decentralized than the current one, given the failure of subsidy policies, like the Feed-In-Tariff (FIT) policy, and the increase in the number of small producers but also in the number of power utilities. Clearly, each utility must follow regulation rules regarding the proportion of RES units it must have in its energy mix, in order to avoid emission penalties. Following a decentralized market scheme, we address the problem of allocating a total amount of RE demanded by a set of utilities in a local market to individual RES microgrids (MGs) that can cover the demands, considering two supply policies. In the first policy, a RES producer is assumed to be able to split its production into smaller parts so that it can supply multiple utilities, and a simple allocation algorithm is presented. In the second policy, a RES producer, due to market or technical constraints, cannot split its production and share it among utilities. In this case, we provide an algorithm that solves the problem effectively by viewing it as a knapsack problem. If the cost functions of MGs are independent of the utilities to which they sell (e.g., negligible transportation costs), the results show that the non-divisible policy slightly benefits the MGs. However, in a decentralized market, it is natural to assume that the cost functions of the producers depend on the location of the utilities. To account for this, we provide two more allocation algorithms under both examined supply policies. In this case, the non-divisible policy is much more profitable for MGs and, moreover, the entrance of new utilities in the market clearly benefits them over the divisible case.

The constant expansion of Renewable Energy Source (RES) installations results in increasing the price of electricity in an unsustainable way due to the Feed-In-Tariff (FIT) policy currently being used. The challenge is to create a more liberalized market mechanism without, however, deterring further small scale RES investments, which remain costly. In the current paper, we consider the local electricity market in a given country or geographical area (rural, island, or other), where a certain portion of demand is asked to be covered from DERs. As DERs tend to be to some extend isolated from the main grid, it is possible and desirable for a group of DERs in a geographical area to be organized in a static local association that acts as a multi-plant organization. If the DERs (being small and many, thus "price takers") negotiated as individual units with the market operator, they would achieve a price that is close to their marginal cost, which would be very small, much smaller than their average total cost. In this paper, we provide an algorithm that allows the coalition to offer its electricity production units at a higher price than normal market price without endangering their market share. The price they achieve is higher than if they participated in a complete liberalized market, but less than the tariff of the FIT policy. Interestingly, we find that the coalition has an optimal price at which its profits are maximized. Finally, we observed that if there are limited participants (coalitions) in the local electricity market, an upper bound needs to be set by a regulator, otherwise, the coalition will set the price at will.

A semi-supervised deep learning paradigm is proposed for object classification/tracking. The method addresses the main difficulties of deep learning, by allowing unsupervised data to initially configure the network and then a gradient descent optimization scheme is triggered to fine tune the data. Unsupervised learning transforms the input data into smaller and more abstract forms of representations and therefore improves the stability, convergence and performance of the model. Additionally, an adaptive approach is presented in a way to allow dynamic modification of the model to the current visual conditions. Adaptation is performed by exploiting both unsupervised and supervised samples, coming by the application of a combined motion/deep learning tracker activating only at frames a decision mechanisms ascertains retraining.

Resource selection and task assignment are basic operations in distributed computing environments, like the grid and the cloud, where tasks compete for resources. The decisions made by the corresponding algorithms should be judged based not only on metrics related to user satisfaction, such as the percentage of tasks served without violating their quality-of-service (QoS) requirements, but also based on resource-related performance metrics, such as the number of resources used to serve the tasks and their utilization efficiency. In our work, we focus on the case of tasks with fixed but not strict time requirements, given in the form of a requested start and finish time. We propose an algorithm for assigning tasks to resources that minimizes the violations of the tasks' time requirements while simultaneously maximizing the resources' utilization efficiency for a given number of resources. The exact time scheduling of the tasks on the resources is then decided by taking into account the time constraints. The proposed scheme exploits concepts derived from graph partitioning, and groups together tasks so as to 1) minimize the time overlapping of the tasks assigned to a given resource and 2) maximize the time overlapping among tasks assigned to different resources. The partitioning is performed using a spectral clustering methodology through normalized cuts. Experimental results show that the proposed algorithm outperforms other scheduling algorithms for different values of the granularity and the load of the task requests.

Social networks need to manage and control the drift of huge amounts of information by filtering and summarizing everything, in order to ensure they satisfy users' viewing pleasure. Until now social media content has already been used in a variety of applications such as for ranking of news stories, for profiling of user preferences, even for products' recommendations. However, this type of conversational, user-generated content might be used to add value to more traditional event media, such as video. In this paper we examine the capability of automatically producing meaningful summaries of generic videos. To do so we consider EdgeRank's affinity, weight and time decay parameters and implement a CLARANS-based key-frames extraction scheme. This paper forms an initial study of a social media video abstraction service and experiments indicate its promising performance.

In this paper, we give a short summary of the papers proposed in ACMARTEMIS 2013 which is held in Barcelona Spain in conjunction with ACM Multimedia. The workshop handles the areas of features analysis both at low and high level for efficient events detection, retrieval of multimedia events and objects and video synchronization issues and also events and behavior recognition from visual data. All papers were classified into three session of a single track workshop. The first session named "Video Features and Scene Analysis" includes articles that handle low level and high level visual analysis appropriate for event detection. The second session entitled "Retrieval of Multimedia Objects/Events" applies schemes for media data retrieval and video synchronization. Finally the third session "Analysis of Visual Events" describes algorithms for detecting actions, behaviors and events in complex visual scenes.

Google Maps and Google Earth were milestones in the evolution of Geographic Information Systems (GIS). A new era of 3D virtual geo-referenced worlds began. The need of exploration and creation personalized virtual tours is imperative. This work implements a best viewing algorithm for presenting a 3D model by considering both semantic and geometric features of the 3D model. The camera trajectory is estimated by respecting the user's preferences using a personalized entropy-based measurement. Moreover the camera speed is adjusted properly based on the complexity of the projected view of the model.

Nowadays, a paradigm shift is under way in the world of Digital Broadcast TV. This change, similar to that of the mobile market, promises a future where modern TV sets and set top boxes will become the merging point of TV and computers. The `Connected TV' will allow users to access content available either from the broadcast channels or the Internet. There are already some independent attempts in realizing this concept as well as standardization efforts that aim at closing the gap between different implementations. However, more focus is placed on the integration of Internet content while the traditional broadcast part is neglected. In this work, we outline the current status and propose a framework for creating a true hybrid solution, where end users can enrich broadcast content with Internet-based enhancements so that they can enjoy improved and personalized viewing experience. In fact, this approach allows for content composition, by merging the base (broadcast) content with add on (Internet) content. This will further facilitate the opening of the TV market, the emergence of new business models and the offering of more advanced and personalized services.

In an endeavour to speak and prevail over some of the open problems that obstruct efficient networked media, this workshop will fetch together folks from a number of research communities, including but not limited to Multimedia Distribution and Access, Social Network Analysis, Multimedia Content Analysis, Behavioral Analysis, User Modelling Adaptation and Personalization. It is our credence that a synergetic approach involving the above mentioned research areas can surpass their individual potentials, leading to improved networked media access. The main objective of this workshop is to provide a forum to disseminate work that explicitly exploit the synergy between multimedia content analysis, behavioral modelling, personalisation, and next generation networking and community aspects of social networks. This synergetic methodology could produce high quality of experience for personalized multimedia access in networking environment.

Resource allocation and scheduling are fundamental issues in a Workflow Management System (WfMS). Effective resource management in WfMS should examine resource allocation together with task scheduling since these problems impose mutual constraints. Optimization of the one factor is subject to the other constraints and vice versa. Thus, an ideal algorithm should take into account not only performance metrics of the infrastructure, such as the number of resources and their utilization, but also quality criteria such as the percentage of tasks undergone violation in their temporal restrictions. In this paper, we propose an innovative algorithm which jointly optimizes the two aforementioned contradictory criteria. The algorithm, called Resource Conflicts Joint Optimization (Re.Co.Jo.Op.), minimizes resource conflicts subject to temporal constraints and simultaneously optimizes throughput or utilization subject to resources constraints. To achieve the optimization, the two factors are formulated in a matrix form and the optimal solution is found by applying concepts of the generalized eigenvalue analysis. A rough outline of an agent-based architecture is proposed to achieve runtime integration of our algorithm into a functional WfMS, while experimental results under different load environments and tasks assumption reveal the superiority of the proposed strategy than the other conventional approaches.

In this paper we present a multicost algorithm for the joint time scheduling of the communication and computation resources that will be used by a task. The proposed algorithm selects the computation resource to execute the task, determines the path to route the input data, and finds the starting times for the data transmission and the task execution, performing advance reservations. We initially present an optimal scheme of non-polynomial complexity and by appropriately pruning the set of candidate paths we also give a heuristic algorithm of polynomial complexity. We evaluate the performance of our algorithm and compare it to that of algorithms that handle only the computation or communication part of the problem separately. We show that in a Grid network where the tasks are CPU- and data- intensive important performance benefits can be obtained by jointly optimizing the use of the communication and computation resources.

AREA 2008 is the first ACM international workshop on analysis and retrieval of events, actions and workflows in video streams. Such research is nowadays critical for many real-life applications, such as area supervision, semantic characterization and annotation of video streams, quality assurance, and security. This workshop consists of 16 high quality papers organized in four thematic sessions. More specifically, the first session is dedicated to new objects tracking algorithms under complex environments and to object labeling techniques. The second session deals with methods, tools and architectures for detecting high level semantics (events, actions, and workflows) in video sequences. The third session presents new algorithms for analyzing video sequences oriented to detecting humans' actions or implicitly annotating multimedia content. Finally, the fourth includes a special session of the recent advantages of the ongoing research projects in the field of multimedia analysis, cognitive video supervision, personalized video annotation, fast retrieval of multimedia content in compressed domain and scheduling tools for interactive multimedia services. We hope that these proceedings will serve as a valuable reference for analysis of events in video streams.

Future grid networks should be able to provide quality of service (QoS) guarantees to their users. In this work we propose a framework for grid networks that provides deterministic delay guarantees to its guaranteed service (GS) users and best effort service to its best effort (BE) users. The proposed framework is theoretically and experimentally analyzed. We also define four types of computational resources based on the type of users (GS, BE) these resources serve and the priority they give them. We implement the proposed QoS framework for grids and verify that it not only satisfies the delay guarantees given to GS users, but also improves performance in terms of deadlines missed and resource use. In our simulations, data from a real grid network are used, validating in this way the appropriateness and usefulness of the proposed framework.

In general a design process is complex and requires the collaboration of several designers on the same product to improve its reliability, performance and efficiency. Though the increase of the Internet as a communication means that supports the sharing and transferring of knowledge, the collaborative declarative design phase lacks for a) imprecision in declaring the statements (ambiguity) and b) subjective interpretation of a scene with respect to the current designer's profile. For this reason, on-line learning strategies should be applied, which models the actual user's preferences. In this paper, we propose an efficient and adaptable learning strategy for dynamic modeling of a designer profile based an adaptable neural network architecture. The scheme optimally updates the network weights in a way that the current designers' preferences are trusted as much as possible, while simultaneously a minimal degradation of the already obtained network knowledge is minimized. The algorithm requires low computational complexity and guarantees stable performance instead of conventional neural network training schemes, whose the solution is often trapped to local minima.

In this paper, we provide the theoretical evaluation of Icebreaker, a social networking service designed in the scope of IST project Magnet Beyond. We built on previous results of our work that indicate spectral clustering's applicability in regard to the service requirements imposed by the specific application; we extend our approach by provisioning for an online adaptive algorithm that places users into social groups that have previously been assessed through the application of spectral clustering. Experimental results indicate that our approach is able to adhere to the service requirements as new users join the system, without the need of iterative spectral clustering application that is computationally demanding.

In this paper, we introduce and evaluate a framework for a user profile-based socializing application. We model the user profile as an unordered set composed of n keywords that represent users' preferences. We evaluate the effectiveness of two clustering algorithms, k-means and spectral clustering in the scope of social groups' assessment. Through experimental results we substantiate the applicability of spectral clustering in the examined service and we evaluate the impact of profile size in terms of the quality of partitions yielded by spectral clustering. The above are performed using case studies and scenarios developed in the context of 1ST project's MAGNET Beyond pilot services definition.

In this paper, we propose a new algorithm for fair scheduling, and we compare it to other scheduling schemes such as the Earliest Deadline First and the First Come First Serve schemes. Our algorithm uses a max-min fair sharing approach for providing fair access to users. When there is no shortage of resources, the algorithm assigns to each task enough computational power for it to finish within its deadline. When there is congestion, the main idea is to fairly reduce the CPU rates assigned to the tasks, so that the share of resources that each user gets is proportional to the user’s weight. The weight of a user may be defined as the user’s contribution to the infrastructure or the price he is willing to pay for services or any other socioeconomic consideration. In our algorithms, all tasks whose requirements are lower than their fair share CPU rate are served at their demanded CPU rates. However, the CPU rates of tasks whose requirements are larger than their fair share CPU rate are reduced to fit the total available computational capacity in a fair manner.Three different versions of fair scheduling are adopted in this paper; the Simple Fair Task Order (SFTO), which schedules the tasks according to their respective fair completion times, the Adjusted Fair Task Order (AFTO), that refines the SFTO policy by ordering the tasks using the adjusted fair completion times, and the Max-min Fair Share (MMFS) scheduling policy, which simultaneously addresses the problem of finding a fair task order and assigning a processor to each task based on a Max-Min fair sharing policy. Experimental results and comparisons with traditional scheduling schemes, such as the Earliest Deadline First (EDF) and the First Come First Served (FCFS) are presented using three different error criteria. Validation of the simulations using real experiments of tasks generated from 3D image rendering processes is also provided. The three proposed scheduling schemes can be inte

In this paper, we proposed a context-adaptive and user-centric emotion classification scheme of low complexity. Different people express their feelings in a different way under different circumstances (different context). Therefore, an adaptable architecture is proposed in this paper able to automatically update its performance to a particular individual (user-centric) and context environment (context-adaptive). As a result, the same expressions may lead to different emotional states in accordance to the specific environment to these feelings are expressed. The adaptation is performed using concepts derived from functional analysis. The presented adaptable architecture requires low memory and processing capabilities and thus it can be embedded in smart pervasive devices of low processing requirements. Experimental results on real-life databases illustrate the efficiency of the proposed scheme in recognizing the emotion of different people or even the same under different circumstances.

A novel method for summarizing videos of gestures is presented. The gestures performed by the hands and the head are extracted through skin color segmentation and represented through Zernike moments. The gesture energy is calculated using the norms of the Zernike moments and monitored through time for local minima and maxima that indicate distinctive visual events and thus key-frames. The proposed scheme is not threshold-dependent and therefore the number of extracted key-frames varies according to the complexity of gesture energy variation. The applicability of the method is verified experimentally in sign language videos.

Since multimedia information is characterized by motley types of media with different properties, multimedia content retrieval in digital libraries requires dynamic reconfigurable architectures. In this paper, mobile agents are being used as the basis of such a dynamic reconfigurable architecture. The proposed architecture is further enhanced with innovative relevance feedback algorithms for interactive multimedia retrieval. In particular, an optimal recursive relevance feedback mechanism is examined by adaptively estimating a parametric correlation-based similarity measure.

An integrated, reconfigurable, adaptable and open system for mining, indexing and retrieving multimedia information based on a mobile agent technology scheme is presented. The system consists of three integral subsystems, namely the acquisition, the transformation and the distribution modules. Innovative algorithms are used to extract information from Web sources, transform it and deliver it to the end users. The system supports efficient content adaptation mechanisms, textual and visual summarization schemes (both sequential and hierarchical), automatic language translation, ontological representation, visual processing and Web-based data mining. Experimental analysis on real-life Web sites has been performed to test the efficiency of the proposed scheme and compare it with other approaches presented in the literature

In this paper, a novel relevance feedback algorithm is proposed for improving the performance of interactive content-based retrieval systems. The algorithm recursively estimates the similarity measure, which is used for data ranking in description environments where similarity-based queries are applied, using a set of relevant/irrelevant samples feedback by the user to the system so that the adjusted response is a better approximation of the current user's information needs and preferences. In particular, using concepts of functional analysis, the similarity measure is expressed as a parametric form of known monotone increasing functional components. Then, the contribution of each functional component to the similarity measure is estimated through a recursive and efficient on-line learning algorithm so that: 1) the current user's needs and preferences, as indicated by a set of selected relevant/irrelevant samples, are satisfied as much as possible, while simultaneously 2) a minimal modification of the already estimated similarity measure is accomplished. Experimental results on a large real-life database using objective evaluation criteria, such as the precision-recall curve and the average normalized modified retrieval rank (ANMRR), indicate that the proposed scheme outperforms the compared ones. In addition, the proposed algorithm requires low computational complexity and it can be implemented in a recursive way.

In this paper, an interactive framework for navigating video sequences is presented using an optimal content-based video decomposition scheme. In particular, each video sequence is analyzed at different content resolution levels, creating a hierarchy from the lowest (coarse) to the highest (fine) resolution. This content hierarchy is represented as a tree structure, each level of which corresponds to a particular content resolution, while the tree nodes indicate the temporal video segments that the sequence content is partitioned at a given resolution. A criterion is introduced to measure the efficiency of the proposed scheme in organizing the video visual content and to compare it with other hierarchical video content representations and navigation schemes. The efficiency is measured as the difficulty for a user to locate a video segment of interest, while moving through different levels of hierarchy. In our case, video is decomposed so that the best efficiency is accomplished. However, the efficiency of a nonlinear video decomposition scheme depends on: 1) the number of paths required for a user to locate a relevant video segment and 2) the number of shot/frame classes (i.e., content representatives) extracted to represent the visual content. Both issues are addressed in this paper. In the first case, the probability of selecting a relevant video segment in the first path is maximized by extracting optimal content representatives through a minimization of a cross-correlation criterion. For the minimization, a genetic algorithm (GA) is adopted, since application of an exhaustive search to obtain the minimum value is too large to be implemented. The cross-correlation criterion is evaluated on the feature domain by extracting appropriate global and object-based descriptors for each video frame so that a better representation of the visual content is achieved. The second aspect (e.g., the number of content representatives) is addressed by minimizing the average transmitted information and simultaneously taking into consideration the temporal video segment complexity. More content representatives are extracted for video segments of high complexity, whereas a low number is required for low-complexity segments. In addition, a degree of interest is assigned to each video shot (or frame) to address the fact that, from the user's perception, the visual content of a set of shots (frames) satisfies his/her information needs. Finally, a computationally efficient algorithm is proposed to regulate the degree of detail (i.e., the number of shot/frames representatives) in case the visual content is not efficiently represented from the user's perceptive view. Experimental results on real-life video sequences indicate the performance of the proposed GA-based video decomposition scheme compared to other hierarchical video organization methods.

Implementation of a commercial application to a grid infrastructure introduces new challenges in managing the quality-of-service (QoS) requirements, most stem from the fact that negotiation on QoS between the user and the service provider should strictly be satisfied. An interesting commercial application with a wide impact on a variety of fields, which can benefit from the computational grid technologies, is three-dimensional (3-D) rendering. In order to implement, however, 3-D rendering to a grid infrastructure, we should develop appropriate scheduling and resource allocation mechanisms so that the negotiated (QoS) requirements are met. Efficient scheduling schemes require modeling and prediction of rendering workload. In this paper workload prediction is addressed based on a combined fuzzy classification and neural network model. Initially, appropriate descriptors are extracted to represent the synthetic world. The descriptors are obtained by parsing RIB formatted files, which provides a general structure for describing computer-generated images. Fuzzy classification is used for organizing rendering descriptor so that a reliable representation is accomplished which increases the prediction accuracy. Neural network performs workload prediction by modeling the nonlinear input-output relationship between rendering descriptors and the respective computational complexity. To increase prediction accuracy, a constructive algorithm is adopted in this paper to train the neural network so that network weights and size are simultaneously estimated. Then, a grid scheduler scheme is proposed to estimate the queuing order that the tasks should be executed and the most appopriate processor assignment so that the demanded QoS are satisfied as much as possible. A fair scheduling policy is considered as the most appropriate. Experimental results on a real grid infrastructure are presented to illustrate the efficiency of the proposed workload prediction--scheduling algorithm compared to other approaches presented in the literature.

In this paper, an adaptive neural network architecture is proposed for efficient video object segmentation and tracking of stereoscopic sequences. The scheme includes (a) a retraining algorithm for adapting network weights to current conditions, (b) a semantically meaningful object extraction module for creating a retraining set and (c) a decision mechanism, which detects the time instances that a new network retraining is required. The retraining algorithm optimally adapts network weights by exploiting information of the current condition with a minimal deviation of the network weights. Description of the current conditions is provided by a segmentation fusion scheme, which appropriately combines color and depth information.

In this paper, we evaluate and investigate two main types of relevance feedback algorithms; the Euclidean and the correlation-based approaches. In the first case, we examine heuristic and optimal techniques, which exploit either on the weighted or the generalized Euclidean distance. In the second type, two different ways for parametrizing the cross-correlation similarity metric are proposed. The first scales only the elements of the query feature vector, while the second scales both the query and the selected samples. All the examined algorithms are evaluated using objective criteria, such as the precision-recall curve and the average normalized modified retrieval rank (ANMRR). Discussions and comparisons of all the aforementioned relevance feedback algorithms are presented.

Multimedia services and especially digital video is expected to be the major traffic component transmitted over communication networks [such as internet protocol (IP)-based networks]. For this reason, traffic characterization and modeling of such services are required for an efficient network operation. The generated models can be used as traffic rate predictors, during the network operation phase (online traffic modeling), or as video generators for estimating the network resources, during the network design phase (offline traffic modeling). In this paper, an adaptable neural-network architecture is proposed covering both cases. The scheme is based on an efficient recursive weight estimation algorithm, which adapts the network response to current conditions. In particular, the algorithm updates the network weights so that 1) the network output, after the adaptation, is approximately equal to current bit rates (current traffic statistics) and 2) a minimal degradation over the obtained network knowledge is provided. It can be shown that the proposed adaptable neural-network architecture simulates a recursive nonlinear autoregressive model (RNAR) similar to the notation used in the linear case. The algorithm presents low computational complexity and high efficiency in tracking traffic rates in contrast to conventional retraining schemes. Furthermore, for the problem of offline traffic modeling, a novel correlation mechanism is proposed for capturing the burstness of the actual MPEG video traffic. The performance of the model is evaluated using several real-life MPEG coded video sources of long duration and compared with other linear/nonlinear techniques used for both cases. The results indicate that the proposed adaptable neural-network architecture presents better performance than other examined techniques.

In this paper, an unsupervised video object (VO) segmentation and tracking algorithm is proposed based on an adaptable neural-network architecture. The proposed scheme comprises: 1) a VO tracking module and 2) an initial VO estimation module. Object tracking is handled as a classification problem and implemented through an adaptive network classifier, which provides better results compared to conventional motion-based tracking algorithms. Network adaptation is accomplished through an efficient and cost effective weight updating algorithm, providing a minimum degradation of the previous network knowledge and taking into account the current content conditions. A retraining set is constructed and used for this purpose based on initial VO estimation results. Two different scenarios are investigated. The first concerns extraction of human entities in video conferencing applications, while the second exploits depth information to identify generic VOs in stereoscopic video sequences. Human face/ body detection based on Gaussian distributions is accomplished in the first scenario, while segmentation fusion is obtained using color and depth information in the second scenario. A decision mechanism is also incorporated to detect time instances for weight updating. Experimental results and comparisons indicate the good performance of the proposed scheme even in sequences with complicated content (object bending, occlusion).

A new recursive algorithm is proposed for optimal estimation of similarity measures used in a content-based retrieval system. This is performed through a relevance feedback mechanism, which adjusts the similarity distance using information fed back to the user according to the relevance of the previously retrieved images. In contrast to conventional relevance feedback schemes to which a degree of importance is assigned to each element of the feature vector describing the image content, the proposed algorithm optimally adapts the similarity measure at each feedback iteration. This is performed by modeling the similarity distance using functional analysis. The algorithm assumes that a small modification of the similarity measure parameters is adequate to adapt the system response to the new user's requirements. In this case, a first-order Taylor series expansion can be applied and a computationally efficient scheme can be implemented to estimate the optimal similarity measure.

In this paper, an interactive framework for navigating video sequences over Internet protocol (IP)-based networks is presented using an optimal content-based video decomposition scheme. In particular, each video sequence is analyzed at different "content resolution levels", creating a hierarchy from the lowest (coarse) to the highest (fine) resolution. This content hierarchy is represented as a tree structure, each level of which corresponds to a particular content resolution, while the tree-nodes indicate the regions that the sequence content is partitioned at a given resolution. Optimal content-based video decomposition is performed by minimizing a cross correlation criterion so that the most representative shots from a video sequence or frames from a video shot are extracted. Experimental results on real-life video sequences show a sufficient representation of the visual content at each resolution level and a significant reduction of the transmission information, compared to the traditional sequential video scanning.

Two efficient unsupervised video object segmentation approaches are proposed and then extensively compared in terms of computational cost and quality of segmentation results. Both methods exploit depth information. In particular a depth segments map is initially estimated by analyzing a stereoscopic pair of frames and applying a segmentation algorithm. In the first, a "constrained fusion of color segments" (CFCS), video object segmentation is performed by fusion of color segments according to a depth similarity criterion. In the second approach, first, a dilated version of the boundary of each depth segment is produced and several feature points are estimated on this dilated boundary. Then, for each initial point, a normalized motion geometric space (MGS) is created which determines the only allowed path on to which the point can move. In the last step, each initial point moves on to its MGS and stops according to a weighted stop-function. Experiments on real life stereoscopic sequences are presented to exhibit the speed and accuracy of the proposed schemes.

A fully automatic system for embedding visually recognizable watermark patterns to video objects is proposed. The architecture consists of 3 main modules. In the first module, unsupervised video object extraction is performed, by analyzing stereoscopic pairs of frames. Then each video object is decomposed into three levels with ten subbands, using the discrete wavelet transform (DWT) and three pairs of subbands are formed (HL/sub 3/, HL/sub 2/), (LH/sub 3/, LH/sub 2/) and (HH/sub 3/, HH/sub 2/). Next qualified significant wavelet trees (QSWTs) are estimated for the specific pair of subbands that contains the highest energy content compared to the other two pairs. QSWTs are derived from the embedded zerotree wavelet (EZW) algorithm and they are high-energy coefficient paths within the selected pair of subbands. Finally, in the third module, visually recognizable watermark patterns are redundantly embedded to the coefficients of the highest energy QSWTs and the inverse DWT is applied to provide the watermarked video object. The performance of the proposed video object watermarking system is tested under various signal distortions such as JPEG lossy compression, sharpening, blurring and adding different types of noise. Experimental results on real life stereoscopic images are presented to indicate the efficiency and robustness of the proposed scheme.

In this paper, an optimal and efficient algorithm for video summarization is proposed by exploiting temporal variations of video visual content. In particular, the most characteristic frames/shots (key-frames/shots) are extracted by estimating appropriate points on the feature vector curve, which represent in an optimal way the corresponding trajectory. This is performed by minimizing the approximation error of the feature vector curve and the respective curve formed by the estimated points using an interpolation scheme. A genetic algorithm is used for the minimization task, since the complexity of an exhaustive search is too large to be implemented. Furthermore, a fast technique for increasing the number of extracted key-frames/shots is presented.

A wavelet-based steganographic method is proposed for robust message hiding. The message is embedded into the most significant wavelet coefficients of a cover image to provide invisibility and resistance against lossy transmission, compression or other distortion. The architecture consists of three modules. In the first module, the initial message is enciphered by an encryption algorithm. The enciphered message is imprinted onto a white-background image to construct the message-image to be hidden. In the second module, the cover image is decomposed into two levels with seven subbands, using the DWT. Next, qualified significant wavelet trees (QSWTs), which are paths of significant wavelet coefficients, are estimated for the highest energy pair of subbands. In the third module, the message-image is redundantly embedded to the coefficients of the best QSWTs and the IDWT is applied to provide the stego-image. The robustness and efficiency of the proposed steganographic system is evaluated under various loss rates, combined with different JPEG compression ratios.

An optimal relevance algorithm is proposed, which adapts the response of a content based image retrieval (CBIR) system to the user's information needs. In particular, the importance of each descriptor to the similarity measure of the system is estimated so that the correlation between the query image and all images marked by the user as relevant is maximized while simultaneously the correlation over all irrelevant images is minimized. Other degree of relevance can be also included in the proposed scheme. In case the user applies more than one feedback iteration, a recursive algorithm is introduced for increasing the system efficiency. Convergence of the proposed scheme is achieved if "consistent" relevant and irrelevant images are selected by the user.

In this paper constrained gradient vector flow (GVF) field generation is performed, for fast and accurate unsupervised stereoscopic semantic segmentation. The scheme utilizes the information provided by a depth segments map, produced by stereo analysis methods and incorporation of a segmentation algorithm. Then a Canny edge detector is applied to the depth region and produces an edge map. The edge map is used for tube estimation inside which the GVF field evolves. After generation of the GVF field an active contour is unsupervisedly initialized onto the outer bound of the tube. Finally a greedy approach is adopted and the active contour, guided by the GVF field, extracts the VOP. Experimental results on real life stereoscopic video sequences indicate the efficiency of the proposed scheme.

In this paper, generalized granulometric size distributions and size histograms (a.k.a 'pattern spectra') are developed using generalized multiscale lattice operators of the opening and closing type. The generalized size histograms are applied to granulometric analysis of soil-section images. An interesting structure is obtained when the histogram is based on area openings. Furthermore, a fast implementation of the generalized size histograms is presented using threshold analysis-synthesis. Comparisons with size distributions based on conventional morphological operators indicate that the generalized histograms provide a more direct and informative description of the image content in objects with scale-dependent geometric attributes. Applications are also developed for studying the structure of soilsection images.

An efficient system for unsupervised structuring of stereoscopic sequences is presented. It generates links between similar VOPs (video object planes) of different shots. Particularly after shot cut detection, for each shot, a fast, unsupervised VOP detection and tracking algorithm is applied. Then for each of the foreground VOPs of a frame, a feature vector is constructed using low level features of the VOP, such as color and size. Afterwards, for a given shot, key-VOP poses are extracted for each VOP, using an optimization method for locating VOP poses of minimally correlated feature vectors. Finally an iterative process is performed to link each key VOP of a shot to another, according to a correlation measure. Experimental results indicate the promising performance of the proposed system on real life stereoscopic video sequences.

In this paper, an interactive framework for navigating video sequences over Internet Protocol (IP)-based networks is presented using an optimal content-based video decomposition scheme. In particular, each video sequence is analyzed at different resolution levels, creating a hierarchy from the lowest (coarse) to the highest (fine) resolution. This content hierarchy is represented as a tree structure, each level of which corresponds to a particular content resolution, while the tree-nodes indicate the regions that the sequence content is partitioned at a given resolution. Optimal content-based video decomposition is performed by minimizing a cross correlation criterion so that the most representative shots from a video sequence or frames from a video shot are extracted. The number of shot and frame representatives is estimated by taking into account both the average transmitted information, required for a user to access any video frame or segment of interest, and the complexity of the visual content.

The efficiency of a content-based image retrieval (CBIR) system depends on the efficiency of the image visual content representation Usually, the extracted descriptors are organized in a binary framework, which, apart from the fact that it is sensitive to noise, it cannot also provide a physical interpretation of the image content. This problem is faced in this paper by introducing fuzzy histograms In particular, in the proposed scheme each image descriptor is allowed to belong to several (or all) classes but with a different degree of membership. Such a scheme removes possible noise existing in the extracted descriptors and simultaneously provides a physical interpretation of the image visual content. Experimental results are also presented, which explain the theoretical developments and illustrate the good performance of the proposed scheme to real-life image databases.

In this paper an efficient scheme for automatic organization of stereo-captured video sequences is presented, which exploits foreground VOP information of frames. More specifically after shot cut detection, for each frame of a shot, a fast, unsupervised foreground VOP extraction algorithm is applied, based on depth information and normalized Motion Geometric Spaces. Then for each frame, a feature vector is constructed, containing color and size characteristics of the foreground VOPs within the frame. Afterwards, for a given shot, key frames are extracted using an optimization method for locating minimally correlated feature vectors. Finally correlation links are generated between the extracted key frames to produce a graph-like structure of the video sequence. Experimental results on real life stereoscopic sequences indicate the promising performance of the proposed scheme.

In this paper, an efficient scheme for video object segmentation is proposed. The scheme is based on a multiresolution Gradient Vector Flow field (M-GVF) and a Motion Geometric Space (MGS) formulation. In particular the proposed scheme is initialized from an object approximation which can be provided either (a) automatically (unsupervised case) based on a depth map estimation method or (b) semi-automatically by user interaction. In the following, several feature points are estimated on the initial object contour (i.e. depth object) and an M-GVF adapted MGS is created to determine the direction that a feature point is allowed to move to. In this framework, each feature point moves onto its MGS in order to locate the contour of the physical video object. Experimental results are presented to indicate the reliable performance of the proposed scheme on real life stereoscopic and monocular video sequences.

A multiscale video content organization scheme is proposed for fast browsing and efficient transmission of video sequences. The scheme leads to construction of a five-layer tree structure. At layer 0 the root-node is located, connected to all nodes of layer 1, each corresponding to a class of shots. Then every class of shot is expanded at layer 2. The nodes of this layer represent shots. At the next resolution level (layer 3) nodes represent key-frames of shots. Finally at layer 4 the full resolution level is reached, where nodes correspond to frames of the sequence. Each node contains a viewing element and we focus on the extraction of these elements for layers 1, 2 and 3. Viewing elements of layers 1 and 3 are optimally extracted by minimizing a cross correlation criterion. Additionally viewing elements of layer 2 are selected according to a correlation measure between the mean vector of a shot and each of the frames within this shot. The resulting tree-structure enables a user to quickly and easily detect content of interest, by selecting the viewing element of his/her liking. Experimental results on real-life video sequences indicate the promising performance of the proposed scheme.

The paper presents an effective method to reduce the iron losses of wound core distribution transformers based on a combined neural network/genetic algorithm approach. The originality of the work presented is that it tackles the iron loss reduction problem during the transformer production phase, while previous works concentrated on the design phase. More specifically, neural networks effectively use measurements taken at the first stages of core construction in order to predict the iron losses of the assembled transformers, while genetic algorithms are used to improve the grouping process of the individual cores by reducing iron losses of assembled transformers. The proposed method has been tested on a transformer manufacturing industry. The results demonstrate the feasibility and practicality of this approach. Significant reduction of transformer iron losses is observed in comparison to the current practice leading to important economic savings for the transformer manufacturer.

Any performance evaluation of broadband networks requires modeling of the actual network traffic. Since multimedia services and especially MPEG coded video streams are expected to be a major traffic component over these networks, modeling of such services and traffic prediction are useful for the reliable operation of the broadband based an Asynchronous Transfer Mode (ATM) networks. In this paper, a recursive implementation of a Non linear AutoRegressive model (RNAR) is presented for on line traffic prediction of Variable Bit Rate (VBR) MPEG-2 video sources. This is accomplished by using an efficient weight adaptation algorithm so that the network provide good performance even in case of highly fluctuated traffic rates. In particular, the network weights are adapted so that the output is approximately equal to the current data while preserving the former knowledge of the network. Experimental results are presented to show the good performance of the proposed scheme. Furthermore, comparison with other linear or non linear techniques is presented to show that the adopted method yields better results than the other ones.

A fuzzy representation of visual content is proposed, which is useful for video summarization. In particular, a multidimensional fuzzy histogram is constructed for each video frame based on a collection of appropriate features, extracted using video sequence analysis techniques. Then, key frames are selected optimally by minimizing a cross correlation criterion. Experimental results and comparison with other known methods are presented to indicate the good performance of the proposed scheme on real life video recordings.

An efficient technique for summarization of stereoscopic video sequences is presented, which extracts a small but meaningful set of video frames using a content-based sampling algorithm. The proposed video-content representation provides the capability of browsing digital stereoscopic video sequences and performing more efficient content-based queries and indexing. Each stereoscopic video sequence is first partitioned into shots by applying a shot-cut detection algorithm so that frames (or stereo pairs) of similar visual characteristics are gathered together. Each shot is then analyzed using stereo-imaging techniques, and the disparity field, occluded areas, and depth map are estimated. A multiresolution implementation of the recursive shortest spanning tree (RSST) algorithm is applied for color and depth segmentation, while fusion of color and depth segments is employed for reliable video object extraction. In particular, color segments are projected onto depth segments so that video objects on the same depth plane are retained, while at the same time accurate object boundaries are extracted. Feature vectors are then constructed using multidimensional fuzzy classification of segment features including size, location, color, and depth. Shot selection is accomplished by clustering similar shots based on the generalized Lloyd-Max algorithm, while for a given shot, key frames are extracted using an optimization method for locating frames of minimally correlated feature vectors. For efficient implementation of the latter method, a genetic algorithm is used. Experimental results are presented, which indicate the reliable performance of the proposed scheme on real-life stereoscopic video sequences.

The performance evaluation of broadband networks requires statistical analysis and modeling of the actual network traffic. Since multimedia services, and especially variable bit rate (VBR) MPEG-coded video streams are expected to be a major traffic component carried by these networks, modeling of such services and accurate estimation of network resources are crucial for proper network design and congestion-control mechanisms that can guarantee the negotiated quality of service at a minimum cost. The layer modeling of MPEG-1 coded video streams and statistical analysis of their traffic characteristics at each layer is proposed, along with traffic models capable of estimating the network resources over asynchronous transfer mode (ATM) links. First, based on the properties of the entire MPEG-1 sequence (frame layer signal), a model (Model A) is presented by correlating three stochastic processes in discrete time (autoregressive models), each of which corresponds to the three types of frames of the MPEG encoder (I, P, and B frames). To simplify the traffic Model A and to reduce the required number of parameters, we study the MPEG stream at a higher layer by considering a signal, which expresses the average properties of I, P, and B frames over a group of picture (GOP) period. However, models on this layer cannot accurately estimate the network resources, especially in multiplexing schemes. For this reason, an intermediate layer is introduced, which exploits and efficiently combines information of both the aforementioned layers, producing a model (Model B), which requires much smaller number of parameters than Model A and simultaneously provides satisfactory results as far as the network resources are concerned. Evaluation of the validity of the proposed models is performed through experimental studies and computer simulations, using several long duration VBR MPEG-1 coded sequences, different from that used in modeling. The results indicate that both Models A and B are good estimators of video traffic behavior over ATM links at a wide range of utilization.

A novel approach is presented in this paper for improving the performance of neural-network classifiers in image recognition, segmentation, or coding applications, based on a retraining procedure at the user level. The procedure includes: 1) a training algorithm for adapting the network weights to the current condition; 2) a maximum a posteriori (MAP) estimation procedure for optimally selecting the most representative data of the current environment as retraining data; and 3) a decision mechanism for determining when network retraining should be activated. The training algorithm takes into consideration both the former and the current network knowledge in order to achieve good generalization. The MAP estimation procedure models the network output as a Markov random field (MRF) and optimally selects the set of training inputs and corresponding desired outputs. Results are presented which illustrate the theoretical developments as well as the performance of the proposed approach in real-life experiments.

In this paper efficient modeling of VBR MPEG coded video sources is proposed by appropriately combining properties of frame and GOP layer signals. In particular a Markov chain is presented for modeling the video activity, the states of which correspond to correlated AR models responsible for generating the I, P and B frames. Furthermore, an adaptive implementation of the AR coefficients is accomplished in cases that we are interested in video traffic prediction. Experimental results using long duration sequences care provided to indicate the good performance of the proposed modeling scheme.

A neural network scheme is presented in this paper for adaptive video indexing and retrieval. First, a limited but characteristic amount of frames are extracted from each video scene, by minimizing a cross-correlation criterion. Low level features are extracted to indicate the frame characteristics, such as color and motion segments. This is due to the fact that extraction of high-level, semantic, features from any kind of images is too hard to be implemented. After the key frame extraction, the video queries are implemented directly on this small number of frames. To reduce, however, the limitation of low-level features the human is considered as a part of the process, meaning that he/she is able to assign a degree of appropriateness for each retrieved image of the system and then restart the searching. A feedforward neural network structure is proposed as a parametric distance for the retrieval, mainly due to the highly non linear capabilities. An adaptation mechanism is also proposed for updating the network weights, each time a new image selection is performed by the user. This mechanism can modify the network weights so that the output of the network, after the adaptation, is as much as close to the user's selection while simultaneously performing a minimal degradation of the previous learned data.

A novel object dependent coding scheme based on optimization upon wavelet packet trees is proposed. The method may be employed in the context of object oriented video standard MPEG-4 for constraint distortion minimization for each video object (VO). Optimal transmission rates are evaluated for each VO which guarantee that important VO (like the foreground) are transmitted with distortions which are equal or less than preselected distortion values. The proposed algorithm is applied to object dependent coding of video frames of the Akiyo sequence with peak-signal-to-noise (PSNR) values, which are higher for the foreground than the background. Experimental results, which favor the proposed method, are provided as well.

A framework for video content representation is proposed in this paper for extracting limited, but meaningful, information of video data directly from MPEG compressed domain. First, the traditional frame-based representation is transformed to a feature-based one. Then, all features are gathered together using a fuzzy formulation and extraction of several key frames is performed for each shot in a content-based rate sampling framework. In particular, our approach is based on minimization of a cross-correlation criterion among video frames of a given shot so as to be located a set of minimally correlated feature vectors. Experimental results indicating the good performance of the proposed scheme are also presented.

In this paper, efficient modeling of variable bit-rate (VBR) MPEG-coded video sources is proposed by appropriately combining properties of frame and GOP (group of pictures) layer signals. In particular, a Markov chain is presented for modeling the video activity and the states of which correspond to correlated autoregressive (AR) models that are responsible for generating the I (intra-frame), P (predictive) and B (bidirectionally predictive) frames. Furthermore, an adaptive implementation of the AR coefficients is accomplished in cases where we are interested in video traffic prediction.

In this paper, an efficient image content representation is proposed, which is appropriate for content-based indexing and retrieval. In particular, the image content is represented by a hierarchical graph, constructed in a tree or pyramid structure framework. The basic concept is to represent the spatial relations of the objects in the image with an image-graph and then, the object details with an object-graph. Further decomposition of the objects regions is permitted in a hierarchical framework.

A complete face recognition system is proposed in this paper by introducing the concepts of foreground objects, which are currently used in the MPEG-4 standardization phase, to human identification. The system automatically detects and extracts the human face from the background, even if is not uniform, based on a combination of a retrainable neural network structure and the morphological size distribution technique. In order to combine face images of high quality and low computational complexity, the recognition stage is performed in the compressed domain. Thus, in contrast to existing recognition schemes, the face images are available in their original quality and not only in their transformed representation.

An efficient video content representation is proposed using optimal extraction of characteristic frames and scenes. This representation, apart from providing browsing capabilities to digital video databases, also allows more efficient content-based queries and indexing. For performing the frame/scene extraction, a feature vector formulation of the images is proposed based on color and motion segmentation. Then, the scene selection is accomplished by clustering similar scenes based on a distortion criterion. Frame selection is performed using an optimization method for locating a set of minimally correlated feature vectors.

We propose a neural network based scheme for performing unsupervised video object segmentation, especially for videophone or videoconferencing applications. The procedure includes (a) a training algorithm for adapting the network weights to the current condition, (b) a maximum a posteriori (MAP) estimation procedure for optimally selecting the most representative data of the current environment as retraining data and (c) a decision mechanism for determining when network retraining should be activated. The training algorithm takes into consideration both the former and the current network knowledge in order to achieve good generalization. The MAP estimation procedure models the network output as a Markov random field (MRF) and optimally selects the set of training inputs and corresponding desired outputs, using initial estimates of the human face and body. Finally, a verification mechanism is introduced which augments the training data, exploiting information of the previous and current environment.

An adaptive algorithm for extracting foreground objects from background in videophone or videoconference applications is presented. The algorithm uses a neural network architecture that classifies the video frames in regions of interest (ROI) and non-ROI areas, also being able to automatically adapt its performance to scene changes. The algorithm is incorporated in motion-compensated discrete cosine transform (MC-DCT)-based coding schemes, allocating more bits to ROI than to non-ROI areas. Simulation results are presented, using the Claire and Trevor sequences, which show reconstructed images of better quality, as well as signal-to-noise ratio improvements of about 1.4 dB, compared to those achieved by standard MC-DCT encoders.

This paper performs issues relative to modeling of VBR MPEG coded video sources over ATM B-ISDN networks. Firstly, we analyse the statistical characteristics of the three types of frames, which the MPEG algorithm generates, and we study their autocovariance and probability density functions. Based on the data analysis, we propose two new source models which approximate both the statistical properties and the traffic characterization of MPEG sequences. The second source model (model B) has been introduced in order to reduce significantly the number of parameters which are necessary for our modeling. To verify the good fitness of the proposed new models, a statistical multiplexing configuration over ATM B-ISDN networks has been implemented and cell as well as frame loss probability have been estimated when the bit stream of video sources is generated by the real data and the two proposed models.

On line retraining of neural network is introduced for extracting foreground/background objects in video sequences. The scheme is applied together with a modification of the rate control of MPEG-1 algorithm. The proposed method is compatible to MPEG-1/2 standard but also can be used as a pre-coding stage for the forthcoming MPEG-4 algorithm. Simulation studies have shown an improvement of about 1.5 dB on average as far the PSNR is concerned compared with the conventional MPEG-1 encoder.