The usage of heterogeneous resources provisioned by edge nodes can be co-optimized through re-scheduling microservices. Current (re-)scheduling approaches typically treat the task of co-optimization as a single-objective optimization problem, which cannot address the issue of imbalanced usage of heterogeneous resources (e.g., CPU, memory, bandwidth) on a single edge node. More importantly, these approaches are inadequate in handling: (i) the adaptive co-optimization of heterogeneous resources, (ii) the fine-grained construction of micro service dependencies, and (iii) multi-step online mi croservice re-scheduling. To address these challenges, this paper proposes a Dependency-aware Online Microservice re-Scheduling (DOMS) approach. DOMS formulates microservice re-scheduling as a multi-knapsack optimization problem and solves it using a Double Dueling Deep Q-Network (D3QN) with prioritized experience replay. Specifically, an adaptive heterogeneous resources balancing detection algorithm is developed, incorporating a dynamic detection threshold mechanism. A fine-grained microservice performance metrics dependency graph is constructed by capturing causal relationships to represent sequential execution dependency. Based on this graph, a microservice multi-step scheduling partition algorithm is devised. Extensive experiments are conducted upon publicly-available datasets, and evaluation results demonstrate that DOMS outperforms the state-of-the-art techniques with improvements of at least 1.85%, 6.45%, 0.56%, and 3.18% in terms of latency, energy consumption, balance degree, and throughput. These results highlight the effectiveness and superiority of DOMS in maintaining a balanced usage of heterogeneous resources and improving network throughput, while satisfying latency and energy consumption constraints.

Empowered by edge computing, resources and computation capabilities provided by edge devices can be encapsulated as containerized services. When burst requests are coming, there may have edge devices which are overloaded, since most requests are spatially and temporally constrained, and edge devices are resource-scarceness and capacity-limited. Overloaded devices should be relieved through optimally migrating one or more activated services to contiguous edge devices. Besides, sensory data gathered by original edge devices should be periodically transmitted to migrated devices for data analysis purpose. To mitigate this issue, this paper proposes an Energy-efficient Online Service Migration (EOSM) mechanism to conduct the migration of multiple services simultaneously. Extensive experimental results show that our EOSM mechanism outperforms the state of arts techniques in mitigating overloaded services in terms of access latency, energy consumption, and request success rate.

Empowered by edge computing, resources and computation capabilities provided by edge devices can be encapsulated as containerized services, and domain applications can be achieved through service compositions. When burst requests are coming to be satisfied, there may exist edge devices which are overloaded, since requests are mostly spatially and temporally constrained, and edge devices are resource-scarceness and capacity-limited. In this setting, overloaded devices should be relieved through optimally migrating one or more activated services to contiguous edge devices. Besides, sensory data gathered by original edge devices should be periodically transmitted to migrated devices for data analysis purpose. To mitigate this issue, this paper proposes an Energy-efficient Online Service Migration (EOSM) mechanism to conduct the migration of multiple services simultaneously. Specifically, a light service sharing strategy is developed to only transmit the top container layer, and a modified NSGA-II algorithm is adopted to generate one or multiple paths for the container layer and time-series sensory data migration of each migrated service. Extensive experimental results show that our EOSM strategy outperforms the state of arts techniques in mitigating overloading devices in terms of access latency, energy consumption, and request success rate.

Ground-penetrating radar (GPR) is a pivotal noninvasive tool that yields subsurface images critical to archeology, near-surface characterization, geotechnical studies, and disaster response. The antenna central frequency of the GPR system has a significant impact on penetration depth and resolution. Lower antenna frequencies penetrate deeper but at lower resolutions, while higher frequencies offer detailed images at reduced depths. Therefore, improving the resolution of low-frequency radar with increased detection depth is an essential research focus. Inspired by image super-resolution advancements, supervised deep learning methods that rely on strictly paired training data have achieved remarkable success. However, acquiring such paired samples in practical scenarios is often a formidable challenge. To tackle this, we propose a novel resolution enhancement technique through weakly supervised learning, effectively addressing the scarcity of strictly paired samples in real-world situations. We utilize two sets of antennas with different central frequencies to construct our training data, with a low-frequency antenna as input and a high-frequency antenna as the learning target. A cycle-consistent generative adversarial network (Cycle-GAN) is trained to discern the mapping between low-resolution inputs and unpaired high-resolution data. The refined network is then employed to improve low-frequency GPR data resolution. Our work is validated on synthetic and real-world datasets. The proposed method effectively strengthens critical high-frequency details for finer imaging and broadens the frequency bandwidth. Significantly, it enhances resolution without compromising the detection depth of low-resolution GPR data, marking a substantial advancement in subsurface imaging technology.

During seismic acquisition, the received seismic data typically comprise many components, such as effective reflections and various interferences. Some components, such as industrial electrical interference and traffic vibrations, manifest as the equidistant narrowband discrete spectra (ENBD-spectra) in the frequency domain. Morphological component analysis (MCA) is widely used for separating different component from complicated seismic data. Therefore, it has been successfully used to extract the narrowband components from seismic data. However, the conventional MCA method overlooks equidistant feature of ENBD-spectra component in seismic data separation. In this study, we propose an improved MCA method that uses the interval between neighboring spectrum peaks as a constraint to separating the data with ENBD-spectra component. Two types of seismic datasets are used to show the proposed MCA’s effectiveness. The first type of dataset contains industrial electrical interference, while another type of dataset contains high-speed train (HST)-induced seismic signals. Both synthetic data examples and real data examples show that the proposed method has better performance in separating the seismic data with ENBD-spectra component and keeping the fidelity of separation compared with the conventional MCA method.

With the explosive growth of the Internet of Things (IoT) devices deployed in edge networks, the functionalities of IoT devices are typically encapsulated as IoT services, and user requests can be achieved through the composition of data and/or computation-intensive IoT services. Considering the prediction-uncertainty of forthcoming requests, certain IoT services may (i) not be hosted currently by appropriate IoT devices, or (ii) such an IoT service exists, but its non-functional properties may hardly be satisfied with respect to certain constraints prescribed by requests. To address this challenge, this paper proposes an efficiency-aware service Migration Scheduling (denoted eMS) mechanism in edge networks, in order to migrate IoT services on-demand, and thus, to optimally settle non-satisfiable constraints. Specifically, IoT services are re-scheduled, such that certain IoT services are migrated from their hosting IoT devices to neighboring ones, while minimizing the energy consumption and average delay caused by this service re-scheduling operation. We formulate this service re-scheduling as a multi-objective and multi-constraint optimization problem, which is solved through integrating the greedy algorithm into the fast non-dominated sorting and crowded-comparison operators as the hybrid genetic algorithm (G-NSGA-II). Based on real-life datasets provided by an oil pipeline monitoring project, extensive experiments are conducted, and evaluation results show that our eMS is promising in reducing the energy consumption and average delay of service re-scheduling in comparison with the state-of-art’s techniques.

Temporal moment localization using natural language (TMLNL) is an emerging issue in computer vision for localizing a specific moment inside a long, untrimmed video. The goal of TMLNL is to obtain the video’s output moment, which is related to the input query in a substantial way. Previous research focused on the visual portion of TMLNL, such as objects, backdrops, and other visual attributes, but natural language processing (NLP) techniques were largely used for the textual portion. A long query requires sufficient context to properly localize moments within a long untrimmed video. Thus, as a consequence of not completely understanding how to handle queries, performances deteriorated, especially when the query was longer. In this paper, we treat the TMLNL challenge as a unique variation of VQA, which equally considers the visual elements by using our proposed VQA joint visual-textual framework (JVTF). However, we also manage complex and long input queries without employing natural language processing (NLP) by improving poorly graded to finely graded distinct granularity representations. Our suggested BCPN searches for insufficient context for long input queries using an approach called query handler (QH) and helps the JVTF find the most relevant moment. Previously, a recurrence of words was caused by increasing the number of encoding layers in transformers, LSTMs, and other NLP techniques; however, our QH ensured that repetition of word locations was reduced. The output of BCPN is combined with JVTF’s guided attention to further improve the end outcome. Therefore, we propose a novel bidirectional context predictor network (BCPN), in addition to a VQA joint visual-textual framework (JVTF), to address the equal importance of videos and queries. Through extensive experiments on three benchmark datasets, we show that the proposed BCPN outperforms the state-of-the-art methods by$IoU = 0.3 (2.65 \%) $,$IoU = 0.5 (2.49 \%)$, and$IoU = 0.7 (2.06 \%) $.

Multi-action video recognition is much more challenging due to the requirement to recognize multiple actions co-occurring simultaneously or sequentially. Modeling multi-action relations is beneficial and crucial to understand videos with multiple actions, and actions in a video are usually presented in multiple modalities. In this paper, we propose a novel multi-action relation model for videos, by leveraging both relational graph convolutional networks (GCNs) and video multi-modality. We first build multi-modal GCNs to explore modality-aware multi-action relations, fed by modality-specific action representation as node features, i.e., spatiotemporal features learned by 3D convolutional neural network (CNN), audio and textual embeddings queried from respective feature lexicons. We then joint both multi-modal CNN-GCN models and multi-modal feature representations for learning better relational action predictions. Ablation study, multi-action relation visualization, and boosts analysis, all show efficacy of our multi-modal multi-action relation modeling. Also our method achieves state-of-the-art performance on large-scale multi-action M-MiT benchmark. Our code is made publicly available at https://github.com/zhenglab/multi-action-video.

An energy-efficient anomaly detection is fundamental to maintain a healthy status of domain applications in edge-cloud collaboration networks. Generally, various kinds of multimodal sensory data capture heterogeneous attributes, where a certain attribute, called the primary one, may be more significant in detecting certain anomaly. This observation drives us to propose a novel energy-efficient anomaly detection mechanism, where attributes sensed by multimodal smart things (msts) are categorized as primary and secondary ones according to their relevance with the characteristic of this anomaly. This technique includes two steps: 1) an initial anomaly detection in single edge networks. Edge nodes associated with the primary attribute adopt a lightweight object detection model to initially detect the potential occurrence of this anomaly. Certain edge networks are determined where an anomaly is suspected and 2) an anomaly refinement with multimodal and multiattribute smart things in marginal edge networks. The cloud identifies and issues a specific query request to gather anomaly-aware sensory data from smart things with secondary attributes, for refining the detection accuracy of this anomaly, where an adaptive weighted fusion model is developed to analyze sensory data coupling of msts. The experimental results show that this technique performs better than the state of the art on the reduction of energy consumption and query time.

Learning spatiotemporal features is very effective but challenging for video understanding especially action recognition. In this paper, we propose Multi-Group Multi-Attention, dubbed MGMA, paying more attention to "where and when" the action happens, for learning discriminative spatiotemporal representation in videos. The contribution of MGMA is three-fold: First, by devising a new spatiotemporal separable attention mechanism, it can learn temporal attention and spatial attention separately for fine-grained spatiotemporal representation. Second, through designing a novel multi-group structure, it can capture multi-attention rendered spatiotemporal features better. Finally, our MGMA module is lightweight and flexible yet effective, so that can be easily embedded into any 3D Convolutional Neural Network (3D-CNN) architecture. We embed multiple MGMA modules into 3D-CNN to train an end-to-end, RGB-only model and evaluate on four popular benchmarks: UCF101 and HMDB51, Something-Something V1 and V2. Ablation study and experimental comparison demonstrate the strength of our MGMA, which achieves superior performance compared to state-of-the-arts. Our code is available at https://github.com/zhenglab/mgma.