In multivariate long-term time series forecasting, the success of self-attention is commonly attributed to the attention matrix that encodes token interactions.In this paper, we provide evidence that challenges this view.Through extensive experiments on three classic and three latest Transformer models, we find that dotproduct attention can be replaced by elementwise operations without token interaction, such as the addition and Hadamard product, while maintaining or even improving accuracy.This motivates our central hypothesis: the effectiveness of self-attention in this task arises not from the dynamic attention matrix, but from the multi-branch feature extraction enabled by the parallel Query, Key, and Value projections and their fusion.To validate this hypothesis, we construct a minimalist multi-branch MLP that isolates the 'multi-branch mapping with element-wise operation' structure from the Transformer and show that it achieves competitive performance.Our findings indicate that the source of performance in self-attention is often misinterpreted, as its actual advantage stems from the architectural principle of multi-branch mapping and fusion, rather than the attention matrix.

Time series forecasting is always important, with multivariate long-term series forecasting being its most challenging task. Here, the existing methods typically learn only in a single domain and focus on optimizing model structures, leading to incomplete information mining and imprecise predictions. To address this, we propose a generalized Mixture of Multi-Domain Experts (MoME) for multivariate long-term series forecasting. Unlike most existing methods, MoME focuses on multi-perspective information mining and fusing. To this end, MoME transforms time series into the frequency and spatial domains to learn their respective representations. MoME regards variates information as embedded features and applies fast Fourier transform to the time dimension. Then it learns embedded features in the frequency domain. In spatial domain learning, MoME applies self-attention mechanism on the variates dimension to efficiently capture dependencies among multiple variates. Finally, MoME fuses the outputs from all domains, reinterprets and integrates information across multiple domains, and predicts future time series. Extensive experiments prove that MoME outperforms state-of-the-art (SOTA) methods. Code is available at: https://github.com/lxy-PhD2022/MoME

In multi-target collaborative tracking, sensor resources allocation is a crucial yet complicated problem. A new sensor management algorithm using the Conditional Posterior Cramér-Rao Lower Bound (CPCRLB) as information metric and the Modified Closest Point of Approach (MCPA) algorithm to assess the threat of targets is proposed in this paper. To quantify the tracking performance, CPCRLB is computed based on Particle Filter and serves as the information metric for sensor management. The MCPA threat, which integrates the relative positions and velocities of the target with respect to the protected asset, is employed to guide the sequence of multi-target sensor allocation scheme in order to achieve high tracking accuracy in consideration of target threat. In order to meet the computation requirements, the Hierarchical Binary Particle Swarm Optimization (HBPSO) algorithm is used to search for the optimal sensor allocation scheme at each time step. By integrating a hierarchical architecture into the original Binary Particle Swarm Optimization algorithm, particles from different layers are used to search in different spaces to prevent the algorithm from getting stuck in locally optimal solutions. Simulations in both static and dynamic scenarios demonstrate that this new sensor management algorithm outperforms the existing algorithms, including the state-of-the-art algorithms based on Modified Particle Swarm Optimization (MPSO).

Cloud-of-clouds storage systems are widely used in online applications, where user data are encrypted, encoded, and stored in multiple clouds. When some cloud nodes fail, the storage systems can reconstruct the lost data and store it in the substitute nodes. It is a challenge to reduce the latency of data recovery to ensure data reliability. In this paper, we adopt a Reinforcement Learning-based Data Recovery (RLDR) approach to reduce the regeneration time. By employing the Monte-Carlo method, our approach can construct the tree-topology-based regeneration process, a.k.a. regeneration tree, to effectively reduce the regeneration time. Through rigorous analysis, we apply the information flow graph to optimize the inter-cloud traffic for a given regeneration tree. To verify the merit of RLDR, We conduct extensive experiments on real-world traces. Experiments demonstrate that RLDR can significantly accelerate the regeneration process. Specifically, RLDR can reduce the regeneration time by up to 92% and increase the throughput by up to twelve-fold, compared to the prior art.

Cloud-of-clouds storage can enhance the data security and reliability of online applications by encrypting, encoding, and distributing user data across multiple clouds. Fast transferring large volumes of data through networks with limited bandwidths remains a practical challenge, especially in the event of disaster backup. To address this, we model a data storage process using an information flow graph and estimate inter-cloud traffic. We propose a new Network Coding-based Cloud-of-Clouds Backup (NC3B) framework, which enables collaborative encoding and data exchange among backup clouds to utilize inter-cloud bandwidth efficiently. We analytically corroborate that NC3B effectively reduces write operation latency. We also demonstrate the NC3B framework by incorporating two cutting-edge Reed-Solomon (RS) based data storage techniques, namely All-Or-Nothing Transform-RS (AONT-RS) and Converge AONT-RS (CAONT-RS), referred to as Network coding-based Backup AONT-RS (NBAONT-RS) and Network coding-based Backup CAONT-RS (NBCAONT-RS), respectively. To validate our approach, we deploy a real-world prototype storage system on Amazon EC2 using a cluster trace set, and underscore the effectiveness of NC3B, showcasing reductions in latency of up to 50% compared to state-of-the-art approaches, alongside throughput improvements of up to 98%. These findings underscore the benefits of NC3B in real-world storage scenarios.

Multi-cloud storage is recently a viable approach to solve the vendor lock-in, reliability, and security issues in cloud storage systems. As a key concern, data placement influences the cost and performance of storage services. Yet, in practice it remains challenging to address the huge solution space. Previous studies typically focus on constructing efficient data placement schemes based on the predicted pattern of workloads or assuming fully a-priori known network conditions. They cannot be easily applied in multi-cloud storage scenarios, which typically involve dynamic network conditions and time-varying workloads. To this end, we formulate the data placement optimization in a combinatorial multi-arm bandit (CMAB) perspective and solve it by learning placement strategy online. In contrast to a stationary setting where reward distributions are unknown but identical over time, we consider a realistic multi-cloud environment with non-stationary conditions, i.e., reward distributions change over time. To swiftly accommodate this, we propose an adaptive window combinatorial upper confidence bound based data placement (AW-CUCB-DP) scheme to reduce latency and cost. In AW-CUCB-DP, a simple and efficient change detector, i.e.,Page-Hinkley testwith forgetting mechanism (FM-PHT), is employed to enable variable-size sliding windows to handle both gradual and abrupt variations in network conditions or workloads. We establish that AW-CUCB-DP is asymptotically optimal in the non-stationary multi-cloud environment. Trace-driven experiments further verify that our scheme outperforms alternatives, especially in highly dynamic environments.

IPFS has surged into popularity in recent years. It organizes user data as multiple objects where users can obtain the objects according to their Content IDentifiers (CIDs). As a storage system, it is of great importance to understand its data I/O performance. But existing work still lacks such a comprehensive study. In this work, we deploy an IPFS storage system with geographically-distributed storage nodes on Amazon EC2. We then conduct extensive experiments to evaluate the performance of data I/O operations from a client's perspective. We find that the access patterns of I/O operations (e.g., request size) severely affect the I/O performance, since IPFS typically uses multiple I/O strategies to perform different I/O requests. Moreover, for the read operations, IPFS requires to resolve remote nodes and downloading objects via the internet. Our experimental study reveals that both resolving and downloading operations can become bottlenecks. Our results can shed light to optimizing IPFS in avoiding high-latency I/O operations.

Cloud-of-clouds storage is a viable means to ensure security and reliability of distributed data storage, where data are encrypted, encoded, and stored in multiple clouds. However, it is a great challenge to adopt such a paradigm in mobile devices (e.g., smartphone). Mobile devices are generally incapable to perform the heavy-weight operations (i.e., data encryption, encoding, and transmission) required in such a paradigm, given the limited resources in such devices. This paper focuses on addressing this challenge, i.e., improving data storage performance in mobile cloud-of-clouds storage systems. The key of our proposal is to allow the low-capability mobile devices to offload the computational and transmission overhead to the clouds. In other words, we propose a Network Coding based Cloud-of-clouds Storage (NCCS) scheme, where the clouds can encode and exchange data collaboratively. We consider two state-of-the-art cloud-of-clouds storage approaches, i.e., AONT-RS and CAONT-RS, as example cases to deploy our scheme. Accordingly, we propose their network coding-based enhancements, namely NAONT-RS and NCAONT-RS. We implement a prototype cloud-of-clouds system to verify the efficiency of our proposal. We deploy the prototype on Microsoft Azure and conduct extensive experiments with real-world traces. The experimental results show that NAONT-RS and NCAONT-RS can reduce the time of data storage process by up to 50% and improve the throughput by up to 110% compared with their original versions, i.e., AONT-RS and CAONT-RS.

This paper considers the problem of how to reduce the I/O overhead of data update operations in erasure coding based storage systems. To this end, we first analyze the I/O overhead of update operations with current update approaches. We find the key to reduce such I/O overhead is designing a scheduling algorithm to construct the sequence of update operations. Such an algorithm needs to execute with a time limit, since update requests work under a stringent latency constraint. To quickly schedule the order of update operations, we propose an efficient algorithm, namely UCODR. Our theoretical analysis verifies that UCODR can effectively reduce the I/O overhead of update operations when multiple blocks are updated. To further confirm its effectiveness, we implement a prototype storage system to deploy UCODR with different erasure codes. Extensive experiments are conducted on the prototype storage system with real-world traces. The experimental results show that UCODR can reduce the time of update operations by up to 35 percent and improve the throughput of the storage system by up to 67 percent, compared with the state-of-the-art update approaches.

Cloud-of-clouds storage is a recent approach to improve the security and reliability of data storage for online applications. It encrypts and encodes the user data, and disperses the results to multiple clouds. Thus, the data can tolerate cloud failures, while cannot be inferred even when some clouds are compromised. However, efficiency is a well-known challenge to such a paradigm, since its data storing process (also known as the dispersal process) is time-consuming involving encryptions, encoding, and transmissions, posing a barrier to its wide application. How to speed up the dispersal process is yet to be well addressed. We observe that the dispersal process consists of two types of operations: calculation and transmission. We find that they can execute simultaneously. Hence, the process can be optimized with a pipelined architecture. To this end, we propose the pipelined versions of two state-of-the-art cloud-of-clouds storage approaches, i.e., AONT-RS and CAONT-RS. We implement both proposals and release them open-source online. To verify their effectiveness, extensive experiments are conducted on a prototype storage system with real-world traces. The results show that the pipelined architecture can improve the performance of the dispersal process.

In data storage systems, data are typically stored in redundant storage nodes to ensure storage reliability. When storage nodes fail, with the help of the redundant nodes, the lost data can be restored in new storage nodes. Such a regeneration process may be aborted, since storage nodes may fail during the process. Therefore, reducing the time of regeneration process is a well-known challenge to improve the reliability of storage systems. Delayed repair is a typical repair scheme in real-world storage systems. It reduces the overhead of the regeneration process by recovering multiple node failures simultaneously. How to reduce the regeneration time of delayed repair is yet to be well addressed. Since available bandwidth is flowing in storage systems and the regeneration time is seriously affected by the available bandwidth, we find the key to solve this problem is determining the start time of the regeneration process. Via modeling this problem with Lyaponuv optimization framework, we propose an OMFR scheme to reduce the regeneration time. The experimental results show that OMFR scheme can reduce cumulative regeneration time by up to 78% compared with traditional delayed repair schemes.

Automatic detection of regions of salient texture and objects is useful for analysis of remotely sensed imagery, such as for land cover classification, object detection, and change detection. Intuitively, the local edges on an image indicate spectral discontinuity and the existence of structure texture or objects. This letter explores a simple method for measuring the saliency of texture and objects based on the edge density and spatial evenness of the edge distribution in the local window of each pixel. This method generates a saliency map by computing the saliency index of each pixel. By segmenting the saliency map, the salient structure texture regions and the locations of objects can be extracted. The algorithm requires only the window size as the input parameter and is relatively simple to implement. Experiments using high-resolution images show its effectiveness and accuracy in the detection of salient structure texture regions, such as crops and residential areas, and man-made objects, such as airplanes, cars, etc.