In recent years, artificial intelligence(AI) technologies represented by foundation models have experienced rapid development. Concurrently, On-device AI inference has become the primary approach for intelligent technology applications, offering advantages such as low latency, high security, and personalization. However, due to the limited resources of on-device systems, on-device AI inference faces new challenges, including improving computational efficiency, optimizing task parallelism, and model optimization. This survey addresses these challenges from a software and algorithmic perspective, focusing on three key areas: Operator Computation: Explores methods to accelerate matrix multiplication and convolution, as well as techniques like operator fusion and vectorized computation. Task Inference: Analyzes heterogeneous and distributed computing, memory allocation, and energy-efficient tuning to improve the parallel execution and energy efficiency of inference tasks. AI Models: Covers model compression, lookup table quantization, and model architecture design to reduce computational complexity and storage requirements. By analyzing these areas, the survey aims to improve inference speed, reduce resource dependency, and provide insights into the future trends of on-device AI technology.

The multicloud environments (MCE) represent a novel paradigm encompassing multiple infrastructure as a service (IaaS) providers, enabling users to tailor and optimize cloud services according to their specific requirements. This approach effectively addresses the limitations of a single cloud environment (SCE) regarding technical constraints, geographical coverage deficiencies, and cost-effectiveness concerns while catering to the increasingly diverse and expanding user demands. In MCE, users must employ appropriate strategies to efficiently allocate diverse tasks across multiple cloud service providers (CSPs) by leveraging the best available resources. Traditional scheduling algorithms are inadequate for addressing the complexities of such MCE. This study introduces a framework for the task scheduling procedure in MCE, treating independent task scheduling as a Markov decision process (MDP). We propose a novel agent environment framework that is designed based on the distinctive characteristics of MCE and enables independent task scheduling. Furthermore, we propose a task scheduling algorithm for MCE based on deep reinforcement learning (DRL) to optimize cost and makespan according to diverse user requirements. The simulation experiments are conducted using both simulated datasets and real-world datasets, demonstrating that our proposed algorithm surpasses the other five algorithms in terms of cost minimization and makespan optimization.

Link prediction in directed graphs offers a solution for uncovering detailed and accurate relationships among distinct entities. Unlike conventional link prediction in undirected graphs, the task becomes more intricate in directed graphs as it involves predicting both associations and orientations. Existing methods simply apply classic graph embedding techniques to learn node representations, followed by mapping representations of corresponding node pairs into probabilities indicating potential links. However, the inadequate capture of orientation information and sole reliance on node representations for prediction hinder the effective differentiation of orientation, thereby impeding the link prediction accuracy. In response, we introduce DGLP, an orientation-aware link prediction method tailored for directed graphs. DGLP utilizes the incidence matrix to learn both node and edge representations, effectively capturing structural and orientation information. By leveraging edge representations, DGLP achieves accurate link prediction in directed graphs without relying solely on implicit node representations. Experiments across six datasets demonstrate the effectiveness of DGLP, achieving a 1.2x improvement in prediction results.

Recently, the multi-cloud environment (MCE) has increasingly become the preferred choice of users. As with the cloud environment, efficient workflow scheduling in a MCE remains crucial for identifying the cost efficiency and overall performance of the MCE. In MCE, the resources exhibit heterogeneity, complexity, and dynamism. Simultaneously, the intricate inter-task dependencies among workflow tasks, diverse Quality of Service (QoS) metrics for users, and multiple cloud service providers’ (CSPs) billing mechanisms significantly amplify the workflow scheduling challenge. Motivated by the application of reinforcement learning (RL) in workflow scheduling in a cloud environment, this paper proposes a scheduling algorithm that takes advantage of the asynchronous advantage actor–critic algorithm (A3C) to balance cost, makespan and resource utilization in workflow scheduling in a MCE. By analyzing the elements in the MCE, we design and define multiple agents in the MCE, and each cloud service provider will have an agent to record the state and update the local parameters. For the workflow task submitted by the user, the action is selected according to the initialization policy and submitted to the scheduling action to allocate the task to a designated virtual machine in the MCE so that each agent can more clearly perceive the environment change and adapt to the MCE. In contrast to the traditional A3C algorithm, we design a new critic network according to the data characteristics of real-world scientific workflows so that each agent is more suitable for real-world scientific workflow data. Through multiple sets of simulation experiments, the workflow scheduling algorithm based on the A3C algorithm in the MCE (MCWS-A3C) was compared with three benchmark methods. The experimental results show that the proposed method has better advantages than other methods in terms of cost, makespan, and resource utilization . Specifically, on the Montage_100 dataset, the average cost was reduced by 55.12% compared to other methods. The pioneering introduction of the A3C algorithm that adapts to the dynamic environment into the MCE brings more possibilities to address the issue of workflow scheduling in the MCE.

Telemedicine for 3D images on mobile devices presents promising development opportunities. Being constrained by computing power and storage capacity on mobile devices, the processing performance of 3D medical images is insufficient for more demanding tasks. Using virtual mobile infrastructure technology to utilize cloud resources is a common solution. But it encounters the challenge of poor performance in data transmission, image rendering and image coding. This paper presents a GPU and VPU enabled Open Virtual Mobile Infrastructure (OpenVMI) for 3D image rendering to solve the challenge. It makes two improvements. First, a bespoke GPU driver is developed in the Android Docker, optimizing the transmission workflow for data transmission and image rendering. Second, a Video Process Unit (VPU) is added to the hardware layer to code rendered results in H.264 format, replacing CPU coding which consumes a large amount of CPU resources. By adopting the OpenVMI, the telemedicine training system proposed in this paper presents an easy-to-set up, cheap and low latency solution that is particularly helpful for telemedicine training in remote and underdeveloped areas. Performance experiments suggest that the OpenVMI delivers better performance than existing state-of-the-art systems, even in mobile devices with weaker hardware capabilities. Concurrency experiment suggests that a single host server can support up to 24 concurrent training sessions, which makes the OpenVMI very helpful for telemedicine training that demands high concurrency. The OpenVMI-based solution proposed in this paper is not restricted to the use of telemedicine training, but also suitable for other application areas such as Virtual Reality and Augmented Reality in mobile environments.

Adversarial attack is a serious threat to neural network-based natural language processing applications. Adversarial attack uses tiny well-crafted perturbations to mislead neural networks. While existing adversarial text attacks can achieve good attack effects, they still do not guarantee efficiency and robustness. The adversarial text attacks are more efficient if they use less perturbation to achieve a higher attack success rate. The attacks are more robust if they can achieve a higher success rate when defense strategies are applied. To improve the efficiency and robustness of the adversarial attack, we propose SMAL: Saliency Map Attack with Levenshtein-similarity. The proposed attack consists of two parts: (1) The saliency map measures the perturbation priority of each word. It considers not only the influence of each word on the classification result but also how to maintain the misled classification result to improve the robustness of the attack. (2) Levenshtein-similarity network embeds words into edit distance space. When perturbing sentences, some words are replaced by substitutions with less edit distance. This can reduce the amount of modification, which improves the efficiency of the attack. Since the words are embedded in edit distance space rather than semantic space, the semantic-based defense is not effective for this attack, which improves the robustness. The experiments show that SMAL achieves a higher attack success rate with fewer perturbations. Also, the proposed attack is better when attacking a classifier defended by adversarial training.

With astonishing speed, bandwidth, and scale, mobile-edge computing (MEC) has played an increasingly important role in the next generation of connectivity and service delivery. Yet, along with the massive deployment of MEC servers, the ensuing energy issue is now on an increasingly urgent agenda. In the current context, the large-scale deployment of renewable-energy-supplied MEC servers is perhaps the most promising solution for the incoming energy issue. Nonetheless, as a result of the intermittent nature of their power sources, these special design MEC servers must be more cautious about their energy usage, in a bid to maintain their service sustainability as well as service standard. Targeting optimization on a single-server MEC scenario, we, in this article, propose neural network-based adaptive frequency adjustment (NAFA), an adaptive processor frequency adjustment solution, to enable an effective plan of the server’s energy usage. By learning from the historical data revealing request arrival and energy harvest pattern, the deep reinforcement learning-based solution is capable of making intelligent schedules on the server’s processor frequency, so as to strike a good balance between service sustainability and service quality. The superior performance of NAFA is substantiated by real-data-based experiments, wherein NAFA demonstrates up to 20% increase in the average request acceptance ratio and up to 50% reduction in average request processing time.

Span-based joint extraction models have shown their efficiency on entity recognition and relation extraction.These models regard text spans as candidate entities and span tuples as candidate relation tuples.Span semantic representations are shared in both entity recognition and relation extraction, while existing models cannot well capture semantics of these candidate entities and relations.To address these problems, we introduce a span-based joint extraction framework with attention-based semantic representations.Specially, attentions are utilized to calculate semantic representations, including span-specific and contextual ones.We further investigate effects of four attention variants in generating contextual semantic representations.Experiments show that our model outperforms previous systems and achieves state-of-the-art results on ACE2005, CoNLL2004 and ADE.

Open source represents an important way in which today's software is developed. The adoption of open source software continues to accelerate because of the great potential it offers, such as productivity improvement, cost savings and quicker innovation. While the complexity and the size of software composition grow, it becomes difficult to effectively scan and track the code source, especially for software with tremendous scale of code, such as operating systems. So far, existing work on open source components mainly focus on how to mitigate potential license incompliance, to reduce potential security risks introduced by open source vulnerabilities, and to detect and match open source components in the code. To ensure code traceability and manageability for large scale mixed-source operating system, we believe it is beneficial to automatically distinguish sources of the system code in the granularity of software packages and manage them separately. However, according to the literature, there is a lack of relevant work in this area. In this paper, we first classify the packages into three categories in terms of code source from the perspective of OS developers and maintainers. Then we propose CLASC, an efficient code source classification algorithm. With the capability of package info extraction and analysis, CLASC can classify software packages into the defined categories according to their changelog info. And we design and implement KyAnalyzer, a Web-based package management and code source analysis platform. It provides automatic code source analyzing services and is capable of managing OS packages differentially according to their different categories of code source with CLASC incorporated as a component of it. Experimental results show the correctness and efficiency of the Web-enabled package source classifier.

Network I/O workloads are dominating as one of the leading costs for most of the virtualized clouds. One way to improve the inter virtual machine (VM) inefficiency is to build shared memory channels between VMs co-located on the same physical node to by-pass traditional TCP/IP network stack, so that the overhead is reduced by shorter communication path and fewer kernel interactions. However, it is a key challenge for existing work to achieve high performance inter-VM communication while keeping the capability of VM live migration, and most of existing work are neither seamlessly agile in the presence of VM live migration nor transparent to upper users as well as to operating system kernels, which limits the application of current co-location aware shared-memory based approaches. In this paper, we present the design and implementation of XenVMC, an adaptive and transparent inter-VM communication system for high performance network I/O in virtualized clouds. With proposed dynamic co-located VM membership update mechanism, XenVMC is applicable not only to intra-node VM communication, but also to cross-node communication. It also supports adaptive switching between shared-memory based channel and traditional network-based channel in case of VM live migration, with the aid of proposed VM migration perception and handling mechanisms. XenVMC enables efficient data transmission for both TCP and UDP workloads, with multilevel transparency guaranteed. Extensive experiments show that XenVMC achieves better performance for both TCP and UDP workloads with high transparency, compared with both native virtualized environment and representative existing work. Experimental results also show that it is capable of automatically handling VM migration correctly with acceptable latency.

Graph algorithms currently play increasingly important roles, especially in social networks and language modeling scenarios. Recently, accelerating graph algorithms by heterogeneous high performance computers with the integrated cores and expanded SIMD lanes has been becoming the mainstream. However, the existing methods, restricted by the low-efficiency grouping strategy and the non-optimized selection mechanism of tile size of a graph, are far below our expectations in many ways. Moreover, there are few convenient integrated tools provided for deploying the graph algorithms on MIC architecture. In this paper, we propose a high-efficiency framework MicRun, which is flexible to be used for graph algorithms on SIMD architecture of the Xeon Phi. There are two key components in MicRun, the Bucket Grouping module and Auto-tuning module. In the Grouping module, an optimization algorithm is designed for splitting graph tiles into conflict-free groups, which can be directly processed on SIMD parallelism. In the Auto-tuning module, a novel strategy is proposed for optimizing the tile size to boost execution efficiency of the graph computation. MicRun currently supports Bellman-Ford and PageRank algorithms, we also conduct extensive validation experiments on MicRun. Experimental results show that MicRun outperforms existing mechanisms in terms of storage and time overhead. As a consequence, both graph algorithms achieve an average speedup of 1.1× by MicRun, compared with the state-of-the-art.

Deploying cache has been generally adopted by Internet service providers (ISPs) to mitigate P2P traffic in recent years. Most traditional caching algorithms are designed for locality-unaware P2P networks, which mainly consider the requested frequency of contents as the principle of caching policies. However, in more prevalent locality-aware conditions with biased neighbor-selection policies, the existing caching schemes can hardly optimize the situation. In this paper we show that, what need to be cached in locality-aware conditions are the contents that can not be well provided by local neighbors, rather than the contents which are requested most frequently. Therefore, states of local neighbors should be taken into consideration in caching policies. We first present a new model in which P2P cache and locality-aware neighbor selection work together. We focus on inter-ISP traffic and available bandwidth of users in order to benefit both ISPs and users. Based on the mathematical model, a novel caching algorithm is proposed which considers replacement and allocation policies together. According to trace-driven simulations, the proposed algorithm outperforms other two representative caching algorithms in various scenarios.

The rapid rising of Linux users requires P2P, an efficient content transport method, to distribute packages. Different from traditional streaming P2P systems, a P2P package distribution system is hazarded by the special characteristics of small package size and hot packages. Due to the small package size, DHT search performance, which is rarely considered in traditional P2P system, comes to be an important factor in package distribution process. To improve the DHT search performance in this circumstance, we propose four kinds of optimizations as follow. Firstly, Fast-Response is proposed to eliminate useless searches after having found the target pair. Secondly, LRU Cache is proposed to reduce search hops on the same package. Thirdly, Leap Cache is proposed to reduce the cache redundancy. Finally, Probability Cache, gathering those optimizing above and considering hot packages in addition, is proposed to get increase of cache hit rate and overall efficiency improvement. We simulate our optimizations using PeerSim platform. The results show that all the four optimizations get considerable improvement in performance. With the best situation of Probability Cache, 86.22% delay time of original Kademlia is saved.

Complementary synthesis automatically generates an encoder's decoder with the assumption that the encoder's all input variables can always be uniquely determined by its output symbol sequence. However, to prevent the faster encoder from overwhelming the slower decoder, many encoders employ flow control mechanism that fails this assumption. Such encoders, when their output symbol sequences are too fast to be processed by the decoders, will stop transmitting data symbols, but instead transmitting idle symbols that can only uniquely determine a subset of the encoder's input variables. And the decoder should recognize and discard these idle symbols. This mechanism fails the assumption of all complementary synthesis algorithms, because some input variables can't be uniquely determined by the idle symbol. A novel algorithm is proposed to handle such encoders. First, it identifies all input variables that can be uniquely determined, and takes them as flow control variables. Second, it infers a predicate over these flow control variables that enables all other input variables to be uniquely determined. Third, it characterizes the decoder's Boolean function with Craig interpolant. Experimental results on several complex encoders indicate that this algorithm can always correctly identify the flow control variables, infer the predicates and generate the decoder's Boolean functions.

Network I/O workloads are dominating in many data centers and cloud computing environments today. One way to improve inter Virtual Machine (VM) communication efficiency is to support co-resident VM communication by using shared memory based approaches and to resort to the traditional TCP/IP for inter-VM communications between VMs that are located on different physical hosts. Although a number of independent efforts are dedicated to improving communication efficiency between co-resident VMs, they differ from one another in terms of how the inter-VM communication optimization is carried out and where in the software stack the shared memory channel is established. In this paper, we provide an in-depth overview of the design choices and techniques for optimizing the performance of the co-resident inter-VM communication, with dual objectives. First, we describe the core design guidelines and key issues for optimizing inter-VM communication by using shared memory based mechanisms. Typical issues include choices of implementation layer in the software stack, seamless agility for VM live migration and VM dynamic deployment support, multilevel transparency. Second, we conduct a comprehensive analysis of representative state-of-the-art research efforts and implementation techniques based on the core design guidelines. We also give an analysis of future requirements in advanced features such as reliability, security and stability. The research reported in this paper not only provides the reference for developing the next generation of inter-VM communication optimization mechanisms, but also offers opportunities for both cloud infrastructure providers and cloud service consumers to improve inter-VM communication efficiency in virtualized platforms.

Scale Invariant Feature Transform (SIFT) method plays a critical role in a wide variety of vision applications. But it is now facing the real-time computational challenge. Parallel computing is one of the most promising solutions to overcome the computational challenge. In this paper, we target at parallelizing SIFT by multi-core architecture with per-core SIMD support. We focus on the SIMDization of data parallel parts of SIFT to fully utilize per-core computing power. At Orientation Assignment and Key point Descriptor stages, we observe that load balance is an important factor. We also implement the optimized algorithm on multi-core system with SIMD support from Tianhe-2 Supercomputer and make comparison with the State-of-the-Art parallel SIFT algorithms.

Network has been widely reported as a bottleneck of data center applications. However, current researches of congestion control are unaware of multiple resources consuming and decrease all flows when congestion, ignoring some involved flows may not be the faults. In this paper, we propose a novel multi-resources aware congestion control framework MRTCP to provide a fine-grain control on flows when congestions appear. MRTCP exploits a multi-tuple vector model to measure multi-resources provision and consumption, and develops a novel metric RB (Resource Balance) to denote the heterogeneous amounts of resources employed by each flow. It analyzes which resources are being the bottlenecks that lead to congestions, calculates the responsibility of each flow to this congestion, and then adjusts their sending rates respectively. Our experiment results demonstrate that MRTCP is able to optimize network multi-resources utilization and improve network throughput without adding obvious packets delays.

Network I/O workloads are dominating in most of the Cloud datacenters today. One way to improve inter-VM communication efficiency is to support co-resident VM communication using a faster communication protocol than the traditional TCP/IP commonly used regardless whether VMs are located on the same

The stream processors represent a promising alternative to traditional cache-based general-purpose processors in achieving high performance in stream applications (media and some scientific applications). In a stream programming model for stream processors, an application is decomposed into a sequence of kernels operating on streams of data. During the execution of a kernel on a stream processor, all streams accessed must be communicated through a nonbypassing software-managed on-chip memory, the SRF (Stream Register File). Optimizing utilization of the scarce on-chip memory is crucial for good performance. The key insight is that the interference graphs (IGs) formed by the streams in stream applications tend to be comparability graphs or decomposable into a set of comparability graphs. We present a compiler algorithm for finding optimal or near-optimal colorings, that is, SRF allocations in stream IGs, by computing a maximum spanning forest of the sub-IG formed by long live ranges, if necessary. Our experimental results validate the optimality and near-optimality of our algorithm by comparing it with an ILP solver, and show that our algorithm yields improved SRF utilization over the First-Fit bin-packing algorithm, the best in the literature.

It is hard to execute parallel program efficiently on man-core platform because we could not divide program into appropriate granularity executed simultaneously. Based on virtual machine and binary translation technologies the article proposes the vapor profiling framework that uses SBIRP instruction in-place replacement method to collect program's run-time control flow and data flow information precisely. Moreover, it explains how to create control flow and data flow dependency graphs. Experiment results prove that vapor has better performance than traditional methods.

In this paper, we evaluate and compare the parallel scalability of three commodity operating systems (Linux, Solaris and FreeBSD) on an AMD 32-core platform. Measurements of microbenchmarks and a real-life application reveal that no operating system scales totally better than another for microbenchmarks; for the real-life application, Linux and Solaris are competitive in scalability and perform better than FreeBSD. Related kernel source analysis and performance data suggest that synchronization primitives protecting the shared data structure in kernels are the root cause of the poor scalability on multi-cores.

Multi-core architecture provides more on-chip parallelism and powerful computational capability. It helps virtualization achieve scalable performance. KVM (kernel based virtual machine) is different from other virtualization solutions which can make use of the Linux kernel components such as completely fair scheduler (CFS). However, CFS treats the KVM threads as normal tasks without considering about their unique features such as thread allocation mechanism and lock inside guest virtual machine, which may harm the KVM virtualization performance. In this paper, we analyze a phenomenon that some guest multi-threaded applications have very low performance when scheduled by CFS. As a solution to this problem, we introduce two kinds of optimizations in CFS: (1) configuration optimizations (2) lock optimizations. Our contributions are: (1) implement 5 original and 2 newest proposed optimizations in the newest Linux kernel. (2) Classify and compare them, a brief analysis is also given. They are all very simple and general to other virtual machine monitors such as Xen and schedulers as O(1). The performance of our CFS optimizations to KVM threads is measured by running some well-known benchmarks in two guest virtual machines on an 8-core server which models the real world applications. The results indicate our scheduling optimizations can improve the overall system performance. This paper can provide useful advices to KVM developers and virtualization data center administrators.

It is trendy that virtualization technology is adopted by server and desktop computers recently. Binary translation is an important method to implement full virtualization supporting any guest operating system without modification. Traditional methods use trap or interrupt to catch sensitive instruction's execution. Its performance is influenced by trap's context switch overhead. This article proposes a novel code scanning and replacing strategy, named as Block-based In-Place Replacement. BIPR tries to find a code block whose length is longer than 5 bytes and replaces the block with 5-bytes JMP instruction. The translated code block has same run-time mode as original code. As a result, BIPR's cost is lower than traditional trap methods. Moreover, it gives an optimize strategy, i.e. Super Block-based In-Place Replacement, to reduce unnecessary translation overhead of BIPR and get better performances. Experiment results prove that SBIPR performs pretty.

Current commodity operating systems allow a privileged user to run some programs in kernel mode by installing a kernel module or a device driver, but there isnpsilat an available method to verify the reliability of these programs. As a result, malware leverages this way to corrupt system services, defeat anti-malware and even get control of the whole system. It makes operating-system-based security tools undergo all kinds of hardships in face of various attacks. Virtualization technology brings a new chance to protect operating systems out-of-box in hypervisors. This article proposes a system monitoring and controlling framework, named as BMCS, which monitors low-level states and hardware accessing events of operating system in a hypervisor. BMCS abstracts low-level states and events and profiles OS-level semantics with the help of a hosted daemon. Moreover, it can control the execution of operating system software through the hypervisor. Our out-of box method can collect real run-time operating system profile and control the execution of the OS according some defined rules. As a result, it makes the operating system stronger. We implement BMCS based on KVM and experiment results prove that BMCS performs well.