With the rapid development of the Internet of Things (IoT), IoT devices find applications in various domains. The data generated by these devices is utilized for analysis and services, especially in the field of Artificial Intelligence (AI) applied to IoT, known as Artificial Intelligence of Things (AIoT). The enhancement of edge device computing power in the IoT has led to the emergence of research areas like edge-cloud synergy AI theories and application services. In the context of lifelong learning and real-time processes in AIoT edge-cloud synergy services, addressing unseen tasks becomes crucial. Unseen tasks arise when inference requests from edge devices involve models not present in the cloud’s model repository. Addressing these challenges involves generating data to either augment small sample problems or alter the data distribution for heterogeneous sample issues. As the application of large language models (LLMs) for data generation gains traction, challenges emerge in the context of AIoT edge-cloud synergy services. Firstly, fine-tuning LLMs with heterogeneous data exacerbates model bias issues. Secondly, the substantial data requirements for training LLMs pose a contradiction. Lastly, the involvement of manual annotation in LLM-based data generation introduces complexity and cost. This paper proposes a framework Seafarer to these challenges using Generative Adversarial Networks and Self-taught Learning. Seafarer avoids model bias, reduces data requirements, and eliminates the need for manual annotation. The design demonstrates effectiveness theoretically and is validated on the Cityscapes dataset, achieving an 80% reduction in training loss and improved validation loss stability.

Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices,unfamiliartest samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose a light-weight scheme, calledAnole, to cope with the local DNN model inference on mobile devices. The core idea of Anole is to first establish an army of compact DNN models, and then adaptively select the model fitting the current test sample best for online inference. The key is to automatically identifymodel-friendlyscenes for training scene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combining both human heuristics and feature similarity in separating scenes. Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implement Anole on different types of mobile devices and conduct extensive trace-driven and real-world experiments based on unmanned aerial vehicles (UAVs). The results demonstrate that Anole outwits the method of using a versatile large DNN in terms of prediction accuracy (4.5% higher), response time (33.1% faster) and power consumption (45.1% lower).

As computational hardware advance, integrating deep learning (DL) models into mobile devices has become ubiquitous for visual tasks. However, “data distribution shift” in live sensory data can lead to a degradation in the accuracy of mobile DL models. Conventional domain adaptation methods, constrained by their dependence on pre-compiled static datasets for offline adaptation, exhibit fundamental limitations in real-time practicality. While modern online adaptation methodologies enable incremental model evolution, they remain plagued by two critical shortcomings: computational latency from excessive resource demands on mobile devices that compromise temporal responsiveness, and accuracy collapse stemming from error accumulation through unreliable pseudo-labeling processes. To address these challenges, we introduce AdaShift, an innovative cloud-assisted framework enabling real-time online model adaptation for vision-based mobile systems operating under non-stationary data distributions. Specifically, to ensure real-time performance, the adaptation trigger and plug-and-play adaptation mechanisms are proposed to minimize redundant adaptation requests and reduce per-request costs. To prevent accuracy collapse, AdaShift introduces a novel anti-collapse parameter restoration mechanism that explicitly recovers knowledge, ensuring stable accuracy improvements during model evolution. Through extensive experiments across various vision tasks and model architectures, AdaShift demonstrates superior accuracy and 100ms-level adaptation latency, achieving an optimal balance between accuracy and real-time performance compared to baselines.

With the rapid proliferation of large Low Earth Orbit (LEO) satellite constellations, a huge amount of in-orbit data is generated and needs to be transmitted to the ground for processing. However, traditional LEO satellite constellations, which downlink raw data to the ground, are significantly restricted in transmission capability. Orbital edge computing (OEC), which exploits the computation capacities of LEO satellites and processes the raw data in orbit, is envisioned as a promising solution to relieve the downlink burden. Yet, with OEC, the bottleneck is shifted to the inelastic computation capacities. The computational bottleneck arises from two primary challenges that existing satellite systems have not adequately addressed: the inability to process all captured images and the limited energy supply available for satellite operations. In this work, we seek to fully exploit the scarce satellite computation and communication resources to achieve satellite-ground collaboration and present a satellite-ground collaborative system named TargetFuse for onboard object detection. TargetFuse incorporates a combination of techniques to minimize detection errors under energy and bandwidth constraints. Extensive experiments show that TargetFuse can reduce detection errors by 3.4× on average, compared to onboard computing. TargetFuse achieves a 9.6× improvement in bandwidth efficiency compared to the vanilla baseline under the limited bandwidth budget constraint.

The cost efficiency of model inference is critical to real-world machine learning (ML) applications, especially for delay-sensitive tasks and resource-limited devices. A typical dilemma is: in order to provide complex intelligent services (e.g., smart city), we need inference results of multiple ML models, but the cost budget (e.g., GPU memory) is not enough to run all of them. In this work, we study underlying relationships among black-box ML models and propose a novel learning task: model linking, which aims to bridge the knowledge of different black-box models by learning mappings (dubbed model links) between their output spaces. We propose the design of model links which supports linking heterogeneous black-box ML models. Also, in order to address the distribution discrepancy challenge, we present adaptation and aggregation methods of model links. Based on our proposed model links, we developed a scheduling algorithm, named MLink. Through collaborative multi-model inference enabled by model links, MLink can improve the accuracy of obtained inference results under the cost budget. We evaluated MLink on a multi-modal dataset with seven different ML models and two real-world video analytics systems with six ML models and 3,264 hours of video. Experimental results show that our proposed model links can be effectively built among various black-box models. Under the budget of GPU memory, MLink can save 66.7% inference computations while preserving 94% inference accuracy, which outperforms multi-task learning, deep reinforcement learning-based scheduler and frame filtering baselines.

Edge-cloud collaborative tasks with real-world services emerge in recent years and attract worldwide attention. Unfortunately, state-of-the-art edge-cloud collaborative machine-learning services are still not that reliable due to the data heterogeneity on the edge, where we usually have access to a mixed-up training set, which is intrinsically collected from various distributions of underlying tasks. Finding such hidden tasks that need to be revealed from given datasets is called the Task Partition problem. Manual task partition is usually expensive, unscalable, and biased. Accordingly, we propose Quality-aware Task Partition (QTP) problem, in which final tasks are partitioned by the performance of task models. To the best of our knowledge, this work is the first one to study the QTP problem with an emphasis on task quality. We also implement a public service, HiLens on Huawei Cloud, to support the whole process. We develop a polynomial-time algorithm namely the Task-Forest algorithm (TForest). TForest shows its superiority based on a case study with 57 real-world cameras. Compared with STOA baselines, TForest has on average 9.2% higher F1-scores and requires 43.1% fewer samples when deploying new cameras. Partial code of the framework has been adopted and released to KubeEdge-Sedna.

It is well-known that the HVAC (heating, ventilation, and air conditioning) dominates electricity consumption in commercial buildings. In this paper, we focus on one of the core problems in building operation, namelychiller sequencingto reduce HVAC electricity consumption. Our contributions are threefold. First, we make a case for why it is important to quantify the performance profile of a chiller, namely coefficient of performance (COP), atrun-time, by developing a data-driven COP estimation methodology. Second, we show that predicting COP accurately is a non trivial problem, requiring considerable computation time. To overcome this barrier, we develop a data-driven COP prediction model and an edge-based chiller sequencing framework integrating the COP predictions, and show that they strike a good balance between electricity saving and ease of use for real-world deployment. Finally, we evaluate the performance of our scheme by applying it to real-world data, spanning four years, obtained from multiple chillers across three large commercial buildings in Hong Kong. The results show an electricity saving of over 30 percent compared to baselines. We offer our edge based data-driven chiller sequencing framework as a cost-effective and practical mechanism to reduce electricity consumption associated with HVAC operation in commercial buildings.

In this article, we study solder paste inspection (SPI), an important stage that is used in the semiconductor manufacturing industry, where abnormal boards should be detected. A highly accurate SPI can substantially reduce human expert involvement, as well as reduce the waste in disposing of the boards in good condition. A key difference today is that because of increasing demand in board customization, the number of board types increases substantially and quantity of the boards produced in each type decreases. Thus, the previous approaches where a fine-tuned model is developed for each board type are no longer viable. Intrinsically, our problem is an anomaly detection problem. A major specialty in today’s SPI is that the target tasks for prediction cannot be fully pre-determined due to context changes during the solder paste printing stage. Our experiences show that a conventional approach to first define a set of tasks and train these tasks offline will lead to low accuracy. Here, we propose a novel multi-task approach, where the performance of all target tasks is ensured simultaneously. We note that the SPI process is streamlined and automatic, allowing the SPI time for only a few seconds. We propose a fast clustering algorithm that reuses existing models to avoid retraining and fine tune in the inference phase. We evaluate our approach using 3-month data collected from production lines. We show that we can reduce 81.28% of false alarms. This can translate to annual savings of $11.3 million.

Urban Mobility Models (UMMs) are fundamental tools for estimating the population in urban sites and their spatial movements over time. Most existing UMMs were developed primarily in 2D. However, we argue that people’s movements and living patterns involve 3D space, i.e., buildings, which can heavily affect the accuracy of UMMs. In this article, we for the first time conduct a comprehensive study on the impacts of buildings on human movements and the effect on UMMs. We innovatively capture the impacts by developing a Semi-absorbing Urban Mobility model (SUM) and theoretically prove its properties on its difference from that of previous UMMs. We also show that calibrating our original SUM may need a large number of parameters. As such, we develop two SUM extensions with a substantially reduced number of parameters, making calibration practical. Our evaluation also demonstrates that, as a basis for supporting mobile applications in an intracity and hourly scale, the SUM is far superior to previous UMMs. In a case study, we also show that the performance of the resource allocation scheme in a cellular network substantially improves by using SUM, with a reduction in the packet loss probability of 3.19 times.

On edge devices, data scarcity occurs as a common problem where transfer learning serves as a widely-suggested remedy. Nevertheless, transfer learning imposes heavy computation burden to the resource-constrained edge devices. Existing task allocation works usually assume all submitted tasks are equally important, leading to inefficient resource allocation at a task level when directly applied in Multi-task Transfer Learning (MTL). To address these issues, we first reveal that it is crucial to measure the impact of tasks on overall decision performance improvement and quantify task importance. We then show that task allocation with task importance for MTL (TATIM) is a variant of NP-complete Knapsack problem, where the complicated computation to solve this problem needs to be conducted repeatedly under varying contexts. To solve TATIM with high computational efficiency, we propose a Data-driven Cooperative Task Allocation (DCTA) approach. Finally, we evaluate the performance of DCTA by not only a trace-driven simulation, but also a new comprehensive real-world AIOps case study which bridges model and practice via a new architecture and main components design within AIOps system. Extensive experiments show that our DCTA reduces 3.24 times of processing time, and saves 48.4 percent energy consumption compared with the state-of-the-art when solving TATIM.

Presents corrections to author information for the above named paper.

Edge computing for machine learning has become a heated research topic. On edge devices, data scarcity occurs as a common problem where transfer learning serves as a widely-suggested remedy. Nevertheless, one obstacle is that transfer learning imposes heavy computation burden to the resource-constrained edge devices. Motivated by the fact that only a few tasks of Multi-task Transfer Learning (MTL) have a higher potential for overall decision performance improvement, we design a novel task allocation scheme, which assigns more important tasks to more powerful edge devices to maximize the overall decision performance. In this paper, we focus on task allocation under multi-task scenarios by introducing task importance and make the following contributions. First, we reveal that it is important to measure the impact of tasks on overall decision performance improvement and quantify task importance. We also observe the long-tail property of task importance, i.e., only a few tasks are important, which facilitates more efficient task allocation. Second, we show that task allocation with task importance for MTL (TATIM) is in fact a variant of the NP-complete Knapsack problem, where the complicated computation to solve this problem needs to be conducted repeatedly under varying contexts. To solve TATIM with high computational efficiency, we innovatively propose a Data-driven Cooperative Task Allocation (DCTA) approach. Third, we evaluate the performance of our DCTA approach by applying it to a real-world industrial operation (e.g., AIOps) scenario. Experiments show that our DCTA approach can reduce 3.24 times of processing time compared with the state-of-the-art when solving TATIM. We offer our DCTA approach as an effective and practical mechanism for reducing the required resource associated with performing MTL on edge devices.

Task Relation Discovery (TRD), i.e., reveal the relation of tasks, has notable value: it is the key concept underlying Multi-task Learning (MTL) and provides a principled way for identifying redundancies across tasks. However, task relation is usually specifically determined by data scientist resulting in the additional human effort for TRD, while transfer based on brute-force methods or mere training samples may cause negative effects which degrade the learning performance. To avoid negative transfer in an automatic manner, our idea is to leverage commonly available context attributes in nowadays systems, i.e., the metadata. In this paper, we, for the first time, introduce metadata into TRD for MTL and propose a novel Metadata Clustering method, which jointly uses historical samples and additional metadata to automatically exploit the true relatedness. It also avoids the negative transfer by identifying reusable samples between related tasks. Experimental results on five real-world datasets demonstrate that the proposed method is effective for MTL with TRD, and particularly useful in complicated systems with diverse metadata but insufficient data samples. In general, this study helps in automatic relation discovery among partially related tasks and sheds new light on the development of TRD in MTL through the use of metadata as apriori information.

Traffic prediction, particularly in urban regions, is an important application of tremendous practical value. In this paper, we report a novel and interesting case study of urban traffic prediction in Central, Hong Kong, one of the densest urban areas in the world. The novelty of our study is that we make good second use of inexpensive big data collected from the Hong Kong International Commerce Centre (ICC), a 118-story building in Hong Kong where more than 10,000 people work. As building environment data are much cheaper to obtain than traffic data, we demonstrate that it is highly effective to estimate building occupancy information using building environment data, and then to further use the information on occupancy to provide traffic predictions in the proximate area. Scientifically, we investigate how and to what extent building data can complement traffic data in predicting traffic. In general, this study sheds new light on the development of accurate data mining applications through the second use of inexpensive big data.