Iris motion tracking is critical for discriminating the iris stiffness and developmental stage of primary angle-closure disease (PACD). Anterior segment optical coherence tomography (AS-OCT) video is a highly efficient approach to observe the morphological determinant in iris motion. However, the iris exhibits inconsistent elastic changes during movement, accompanied by changes in local features after long-term frames. Currently, iris tracking methods have not yet been studied in AS-OCT videos. In this paper, we propose a Temporal Constraint-based Tracking Morph (TCTMorph) for estimating iris trajectory in long-term AS-OCT videos. We first estimate the deformation fields between three interrelated frames by a multi-frame diffeomorphic registration network. Then, we estimate iris trajectory from these results in long-term AS-OCT video sequences by leveraging temporal constraints among the consecutive flows. Our experiments on multi-center AS-OCT glaucoma datasets demonstrate that our method outperforms conventional motion tracking methods for long-term iris trajectory tracking.

In the context of incremental learning, a network is sequentially trained on a stream of tasks, where data from previous tasks are particularly assumed to be inaccessible. The major challenge is how to overcome the stability-plasticity dilemma, i.e., learning knowledge from new tasks without forgetting the knowledge of previous tasks. To this end, we propose two mathematical conditions for guaranteeing network stability and plasticity with theoretical analysis. The conditions demonstrate that we can restrict the parameter update in the null space of uncentered feature covariance at each linear layer to overcome the stability-plasticity dilemma, which can be realized by layerwise projecting gradient into the null space. Inspired by it, we develop two algorithms, dubbed Adam-NSCL and Adam-SFCL respectively, for incremental learning. Adam-NSCL and Adam-SFCL provide different ways to compute the projection matrix. The projection matrix in Adam-NSCL is constructed by singular vectors associated with the smallest singular values of the uncentered feature covariance matrix, while the projection matrix in Adam-SFCL is constructed by all singular vectors associated with adaptive scaling factors. Additionally, we explore adopting self-supervised techniques, including self-supervised label augmentation and a newly proposed contrastive loss, to improve the performance of incremental learning. These self-supervised techniques are orthogonal to Adam-NSCL and Adam-SFCL and can be incorporated with them seamlessly, leading to Adam-NSCL-SSL and Adam-SFCL-SSL respectively. The proposed algorithms are applied to task-incremental and class-incremental learning on various benchmark datasets with multiple backbones, and the results show that they outperform the compared incremental learning methods.

Deep neural networks suffer from catastrophic forgetting when trained on sequential tasks in continual learning. Various methods rely on storing data of previous tasks to mitigate catastrophic forgetting, which is prohibited in real-world applications considering privacy and security issues. In this paper, we consider a realistic setting of continual learning, where training data of previous tasks are unavailable and memory resources are limited. We contribute a novel knowledge distillation-based method in an information-theoretic framework by maximizing mutual information between outputs of previously learned and current networks. Due to the intractability of computation of mutual information, we instead maximize its variational lower bound, where the covariance of variational distribution is modeled by a graph convolutional network. The inaccessibility of data of previous tasks is tackled by Taylor expansion, yielding a novel regularizer in network training loss for continual learning. The regularizer relies on compressed gradients of network parameters. It avoids storing previous task data and previously learned networks. Additionally, we employ self-supervised learning technique for learning effective features, which improves the performance of continual learning. We conduct extensive experiments including image classification and semantic segmentation, and the results show that our method achieves state-of-the-art performance on continual learning benchmarks.

In the setting of continual learning, a network is trained on a sequence of tasks, and suffers from catastrophic forgetting. To balance plasticity and stability of network in continual learning, in this paper, we propose a novel network training algorithm Adam-NSCL which sequentially optimizes network parameters in the null space of all previous tasks. We first propose two mathematical conditions respectively for achieving network stability and plasticity in continual learning. Based on them, the network training for sequential tasks without forgetting can be simply achieved by projecting the candidate parameter update into the approximate null space of all previous tasks in the network training process, where the candidate parameter update can be generated by Adam. The approximate null space can be derived by applying singular value decomposition to the un-centered covariance matrix of all input features of previous tasks for each linear layer. For efficiency, the uncentered covariance matrix can be incrementally computed after learning each task. We also empirically verify the rationality of the approximate null space at each linear layer. We apply our approach to training networks for continual learning on benchmark datasets of CIFAR-100 and TinyImageNet, and the results suggest that the proposed approach outperforms or matches the state-ot-the-art continual learning approaches.

Providing scalable video streaming services for heterogeneous users in dynamic networked environments requires efficient and adaptive quality management mechanisms which deliver quality-customized services according to the client's preferences and adapt the services to cope with various network conditions. In this paper, we address the issue of quality adaptation for providing personalized scalable media streaming services in dynamic network environments. We propose a differentiated adaptive quality optimization algorithm, called Scalable Video Coding Quality Adaptation algorithm (SVC-QA), which adapts streaming quality based on both system-level and client-level optimization to optimize streaming quality according to network bandwidth conditions, content characteristics, a user's quality preferences, and buffering capacities of different client devices (e.g., mobile phones, PCs, HDTVs, etc.). Comparative studies are conducted to compare our proposed algorithms with other adaptive methods. We show that two-level SVC quality adaptation method can achieve better SVC streaming quality with both high peak signal-to-noise ratio (PSNR) and low quality variance under dynamic resource constraints. Moreover, the proposed distributed method reduces the computational complexities at the server side substantially, making it practical and flexible for providing scalable streaming services.

As scientific applications like Computational Fluid Dynamics (CFD) simulations generate more and more data, co-processing becomes the most cost effective way to process the vast amount of data generated by these simulation. In a co-processing environment, analysis and/or visualization of intermediate results occur concurrently to the simulation itself. Improved efficiency and early insight into the simulation process and results are potential advantages in comparison to postprocessing, where analysis and/or visualization are performed after the completion of the simulation. To enable co-processing, however, intermediate data needs to be shared between simulation and data analysis, and some degree of coordination may be required to maintain the correctness of both simulation and data analysis. The overhead incurred to facilitate data sharing and coordination may well offset benefits gained, particularly where distributed, large-scale systems are involved as workload sharing, processor affinity and data locality introduce significant effects to the overall performance. In this paper, we propose a co-processing framework to address these issues. The empirical benchmarking results suggest that co-processing overhead tasks scale well with the system size, the overall gain of about 20% in turnaround time compared to post-processing and that the coprocessing framework allows simulation and data analysis task to scale up to their individual limits.

Approaching a comprehensive performance benchmark for on-line transaction processing (OLTP) applications in a cloud environment is a challenging task. Fundamental features of clouds, such as the pay-as-you-go pricing model and unknown underlying configuration of the system, are contrary to the basic assumptions of available benchmarks such as TPC-W or RUBiS. In this paper, we introduce a systematic performance benchmark approach for OLTP applications on public clouds that use virtual machines(VMs). We propose WPress benchmark, which is based on the widespread blogging software, WordPress, as a representative OLTP application and implement an open source workload generator. Furthermore, we utilize a CPU micro-benchmark to investigate CPU performance of cloud-based VMs in greater detail. Average response time and total VM cost are the performance metrics measured by WPress. We evaluate small and large instance types of three real-life cloud providers, Amazon EC2, Microsoft Azure and Rackspace cloud. Results imply that Rackspace cloud has better average response times and total VM cost on small instances. However, Microsoft Azure is preferable for large instance type.

In this paper, an intelligent model for analyzing and predicting cloud computing resource utilization is proposed to enhance on-demand services in cloud computing systems. The model is with the capability to discover active users and mine the system storage utilization patterns. This model is also with learning capabilities to adapt the dynamics in the cloud computing platform by capturing changing patterns of system storage utilization, and it employs data mining means for computing the practical model to be used for prediction and providing inputs for intelligent management in the on-demand cloud computing system. We have evaluated the proposed analysis and prediction model in a cloud computing platform. High prediction accuracies of 95% and 86% have been achieved in 1-day ahead and 7-day ahead system utilization prediction, respectively.

More and more interests have been shown to move large-scale simulations on modern data centers composed of a large number of virtualized multi-core computers. However, the simulation components (Federates) consolidated in the same computer may have imbalanced simulation workloads. Similarly, the computers involved in the same simulation execution (Federation) may also have imbalanced simulation workloads. Hence, federates may waste a lot of computer resources on time synchronization with each other. In this paper, a hierarchical resource management system is proposed to enhance simulation execution performance. Federates in the federation are enraptured in their individual Virtual Machines (VMs), which are consolidated on a group of virtualized multi-core computers. On the computer level, multiple VMs share the resource of the computer according to the simulation workloads of their corresponding federates. On the federation level, some VMs are migrated for workload balance purpose. Therefore, computer resources are fully utilized to conduct useful simulation workloads, avoiding the synchronization overheads. Experiments using synthetic and real simulation workloads have verified that the hierarchical resource management system enhances simulation performance significantly.

Scheduling multiple large-scale parallel workflow applications on heterogeneous computing systems like hybrid clouds is a fundamental NP-complete problem that is critical to meeting various types of QoS (Quality of Service) requirements. This paper addresses the scheduling problem of large-scale applications inspired from real-world, characterized by a huge number of homogeneous and concurrent bags-of-tasks that are the main sources of bottlenecks but open great potential for optimization. The scheduling problem is formulated as a new sequential cooperative game and propose a communication and storage-aware multi-objective algorithm that optimizes two user objectives (execution time and economic cost) while fulfilling two constraints (network bandwidth and storage requirements). We present comprehensive experiments using both simulation and real-world applications that demonstrate the efficiency and effectiveness of our approach in terms of algorithm complexity, makespan, cost, system-level efficiency, fairness, and other aspects compared with other related algorithms.

While hybrid computing environments provide good potential for achieving high performance and low economic cost, it also introduces a broad set of unpredictable overheads especially for running data-intensive applications. This paper describes a novel approach which refines workflow structures and optimizes intermediate data transfers for large-scale scientific workflows containing thousands (or even millions) of tasks. The proposed method includes pre- and post-partitioning of workflows and data-flow optimization. Firstly, it partitions a workflow by identifying the critical path of the task graph. Secondly, it controls the granularity of partitions to reduce the complexity of task graph in order to process large-scale workflows. Thirdly, it optimizes the data-flow based on the scheduling to minimize its communication overheads. Our proposed approach is able to handle complex data flows and significantly reduce data transfer by replacing individual tasks according to data dependencies. We conducted experiments using real applications such as Montage and Broadband, and the results demonstrated the effectiveness of our methods in achieving low execution time with low communication overhead in a hybrid computing environments.

Scientific workflow applications generally require various levels of computing power over the course of execution. The applications then often take advantage of Cloud computing due to its cost-effective, pay-as-you-go pricing model. However, the scientific workflow executions must be planned wisely in order to minimize total cost of the resource usage. In addition, lateness of completing some workflows may result in high penalty cost. In this paper, the scheduling algorithm based on GA and PSO is proposed for optimizing the workflow execution. The experiment to evaluate the scheduling efficiency is performed on the simple workflow engine developed by the authors. The result is then compared to the existing algorithms including HEFT, GA, PSO, and PSO-SA. The result shows that the proposed GAPSO algorithm has a good potential to give the minimum cost when execution time is restricted.

High Level Architecture (HLA)-based simulations employing optimistic synchronization allows federates to process event and to advance simulation time freely at the risk of over-optimistic execution and execution rollbacks. In this paper, an adaptive resource provisioning system is proposed to accelerate optimistic HLA-based simulations in Virtual Execution Environment (VEE). A performance monitor is introduced using a middleware approach to measure the performance of individual federates transparently to the simulation application. Based on the performance measurements, a resource manager distributes the available computational resources to the federates, making them advance simulation time with comparable speeds. Our proposed approach is evaluated using a real-world simulation model with various workload inputs and different parameter settings. The experimental results show that, compared with distributing resources evenly among federates, our proposed approach can accelerate the simulation execution significantly using the same amount of computational resources.

Traditionally, the "best effort, cost free" model of Supercomputers/Grids does not consider pricing. Clouds have progressed towards a service-oriented paradigm that enables a new way of service provisioning based on "pay-as-you-go" model. Large scale many-task workflow (MTW) may be suited for execution on Clouds due to its scale-* requirement (scale up, scale out, and scale down). In the context of scheduling, MTW execution cost must be considered based on users' budget constraints. In this paper, we address the problem of scheduling MTW on Clouds and present a budget-conscious scheduling algorithm, referred to as ScaleStar (or Scale-*). ScaleStar assigns the selected task to a virtual machine with higher comparative advantage which effectively balances the execution time-and-monetary cost goals. In addition, according to the actual charging model, an adjustment policy, refer to as DeSlack, is proposed to remove part of slack without adversely affecting the overall makespan and the total monetary cost. We evaluate ScaleStar with an extensive set of simulations and compare with the most popular HEFT-based LOSS3 algorithm and demonstrate the superior performance of ScaleStar.

Recently, application service providers have been employing Infrastructure-as-a-Service (IaaS) clouds such as Amazon EC2 to scale their computing resources on-demand to adapt to dynamic workloads. Existing research has been focusing more on cloud resource scaling in batch processing, non latency-sensitive applications. In this paper, we consider the problem of revenue-cost optimization in cloud-based application service providers with stringent QoS requirements, e.g., online gaming services. We propose an integrated approach which combines resource provisioning algorithms and request scheduling disciplines. The main goal is to maximize the service provider's revenue via satisfying pre-defined QoS requirements, and at the same time, to minimize cloud resource cost. We have implemented the proposed resource provisioning algorithms and scheduling disciplines into a cloud scaling framework developed in our previous work. Extensive experiments have been conducted with a fully functional implementation and realistic workloads modeled after real traces of popular online game servers. The results demonstrated the effectiveness of our proposed approach.

In this paper, we presented the design, implementation, and evaluation of a distributed system to manage the parallelized analytics for Aircraft Spare parts Management and Optimizations (SMO), which is a well-known problem in logistics industry. Our proposed solution is able to solve the resource-intensive SMO problem using distributed computing infrastructures (e.g., private or public clouds) in a scalable manner. We designed and fine-tuned a parallel met heuristics based on a fine-grained flow control workflow model which enables flow controls of running parallel meta-heuristics in multiple processors and achieved significant performance gains. Together with priority based scheduling, the proposed system effectively dispatches submitted SMO jobs over the set of distributed resources to accommodate different classes of users. Extensive experimental studies were conducted to analyze the performance of parallelized SMO job executions in term of execution time, computation and data transmission time, waiting time, memory usage, etc. Insightful lessons have been drawn from the obtained results, and potential areas for further improvements have also been identified.

The concept of Data Value Chain (DVC) involves the chain of activities to collect, manage, share, integrate, harmonize and analyze data for scientific or enterprise insight. For some applications, it also entails the leverage of visualization and simulation. However, the curse of big data (volume, velocity, variety) makes it difficult to efficiently handle and understand the data in near real-time. To address these challenges, this paper proposed the Data Value Chain as a Service (DVCaaS) framework, a data-oriented approach for data handling, data security and analytics in the cloud environment.

Cloud computing is a suitable platform for execution of complex computational tasks and scientific simulations that are described in the form of workflows. Such applications are managed by Workflow Management System (WfMS). Because existing WfMSs are not able to autonomically provision resources to real-time applications and schedule them while supporting fault tolerance and data privacy, we present a highly-scalable workflow-enabled analytics system that manages inter-dependable analytics tasks adaptively with varying operational requirements on a common platform and enables visualization of multidimensional datasets of real world phenomena. In this paper, we present the architecture of such a WfMS and evaluate it in terms of performance for execution of workflows in Clouds. A real world application of climate-associated dengue fever prediction was evaluated on public, private, and hybrid Clouds and experienced effective speedup in all the environments.

Cloud has been the main technology utilized as a high performance computing (HPC) platform. The characteristics of cloud can satisfy a large scale processing required by scientific applications, which are mostly compute-intensive with big data. Cloud can also reduce the computing cost through sharing and virtualizing of resources. In the cloud, a large number of virtual machines (VM) can be generated on demands. In order to obtain the optimal cost and high efficiency in the task execution on the public cloud, the suitable amount of virtual machines should be properly determined prior to the start of the computation. Moreover, the application should be effectively partitioned and distributed onto the virtual machines. In this paper, we propose an automatic mechanism to allocate the optimal numbers of resources in the cloud. The novel resource estimation model and scheduling algorithm are presented. We select an analytic application with high level of computations in the field of epidemic forecast to demonstrate the use of the designed mechanism. Experimental studies have been conducted to examine the resource prediction accuracy and the scalability of running the application on the cloud.

Infrastructure-as-a-Service (IaaS) cloud computing provides the ability to dynamically acquire extra or release existing computing resources on-demand to adapt to dynamic application workloads. In this paper, we propose an extensible framework for on-demand cloud resource provisioning and adaptation. The core of the framework is a set of resource adaptation algorithms that are capable of making informed provisioning decisions to adapt to workload fluctuations. The framework is designed to manage multiple sets of resources acquired from different cloud providers, and to interact with different local resource managers. We have developed a fully functional web-service based prototype of this framework, and used it for performance evaluation of various resource adaptation algorithms under different realistic settings, e.g. when input data such as jobs' wall times are inaccurate. Extensive experiments have been conducted with both synthetic and real workload traces obtained from the Grid Workload Archives, more specifically the traces from the Large Hadron Collider Computing Grid. The results demonstrate the effectiveness and robustness of our proposed algorithms.

In this paper, we present a cloud framework to provide cloud clustering, workflow scheduling and management, fault tolerance and distributed data storage, data analytics and visualisation services. Using a practical case study, we show that in the process of analyzing multiscale climate data, typical problems plaguing data analysts are faced. These include large datasets and limited computational resources, data complexity and limited knowledge, and varying data structures/formats and the need to integrate different tools. The implementation of our framework to climate studies was a success. This can be seen in its ability to perform spatio-temporal data analysis and visualization of a large multi-dimensional climate dataset with reduced processing time. The framework demonstrates great flexibility and simplicity for end users intending to perform data analysis by aiding the integration of data and tools and enabling interactive visualization on-the-fly. This is coupled with effective utilization of computational resources and data storage systems.

In this paper, we study how resources within a large High Performance Computing (HPC) cluster can be dynamically partitioned to optimize client utility for multiple service classes. We model service effectiveness using both perceived service quality and resources required. Using empirical data obtained from A*STAR Computational Resource Center (A*CRC), we analyze how quality metrics and statistical characteristics of HPC jobs affect user satisfaction. We derive the optimal number of processors required to achieve the maximal overall client utility in M/G/1 based clusters. Based on measured job characteristics, we propose a Statistics-based Client Utility Optimization (SCUO) algorithm, which dynamically partitions the cluster into resource groups serving different service classes. Simulations show that our proposed algorithm is able to achieve better performance with both higher client utility and higher job admission rates.

Due to the advance of technologies in multimedia compression and network communications, scalable media streaming services have been availed to provide QoS differentiated services for heterogeneous users. However, it is still a big challenge to support consistent end-to-end Quality of Services (QoS) for the users due to the dynamic feature of the Internet, and abrupt variability of the network resources may severally affect the client perceived QoS. In this paper, we address the issues of smooth QoS adaptation for scalable streaming services. We propose an Optimal Smooth QoS Adaptation (OS-QA) strategy which allocates the server resource adaptively to cope with the variability of network bandwidth and protects the service quality of different quality classes under dynamic resource constraints. We analyze the quality variation caused by resource fluctuation and proposed OS-QA to minimize the average QoS variance under the resource constraints. Simulations are conducted to compare our proposed method with other QoS adaptation methods, and performance is analyzed in terms of QoS variance and PSNR. Results show that our proposed method is able to gracefully adapt the QoS and protect the client perceived QoS by minimizing the QoS variance under dynamic network resource constrains.

10.1109/CCNC.2007.71

The convergence of grid computing technologies and Web services offers many opportunities to utilize resources distributed across the Internet and solves many issues of interoperability. As a result, enabling applications as Web services are required intensively. Hence, a framework for "gridifying" and running applications on service-oriented grids (GRASG) was built to offer developers a flexible and effective tool for "gridifying" applications and making use of distributed resources on grid environment without much effort from the developers. It allows users to quickly enable an application as a Web service and access this service in a simple fashion. Further, in order to make use of distributed resources, GRASG provides a metascheduling mechanism that is able to schedule jobs to grid resources using Web services protocol. These features reduce the time taken for application development and execution.

In this paper, we proposed to setup a distributed multimedia system which aggregates the capacity of multiple servers to provide customized multimedia services in a cost-effective way. Such a system enables clients to customize their services by specifying the service delay or the viewing times. We developed an experimental prototype in which media servers can cooperate in streams caching, replication and distribution. We applied a variety of stream distribution algorithms to the system and studied their performance under the real-life situations with limited network resources and varying request arrival pattern. The results show such a system can provide cost-effective services and be applied to practical environments.

In this paper, we propose a novel strategy, referred to as window-assisted video partitioning (WAVP), for efficiently rendering network-based multimedia services within delay constraints. To minimize the service cost and maximize the number of requests that can be successfully served under resources constraints (cache capacity and link bandwidth), our WAVP strategy partitions videos into multiple portions and delivers them by adaptive schedule windows. Based on a mathematical analysis, it is shown that the service cost can be optimized by managing video portions with the schedule windows and it can improve resource utilization to partition video streams into multiple portions. We analyze the performance under several influencing parameters such as link availability, cache capacity, delay bound and partition gradients. Simulation results show that our proposed method can significantly reduce the service cost and achieve a high acceptance ratio under the constraints of network resources.

We address the issue of minimizing the per user service cost and at the same time maximizing the number of requests that can be served by distributed video-on-reservation (VOR) systems. We propose a heuristic algorithm, destination-based stream scheduling (DBS) algorithm, which combines the concept of multicast routing and network caching with end-to-end delay constraints. Our simulation results show that our algorithm can reduce the service cost, balance the network load and achieve a high acceptance ratio.