Privacy-preserving Transformer inference has gained attention due to the potential leakage of private information. Despite recent progress, existing frameworks still fall short of practical model scales, with gaps up to a hundredfold. A possible way to close this gap is the Mixture of Experts (MoE) architecture, which has emerged as a promising technique to scale up model capacity with minimal overhead. However, given that the current secure two-party (2-PC) protocols allow the server to homomorphically compute the FFN layer with its plaintext model weight, under the MoE setting, this could reveal which expert is activated to the server, exposing token-level privacy about the client's input. While naively evaluating all the experts before selection could protect privacy, it nullifies MoE sparsity and incurs the heavy computational overhead that sparse MoE seeks to avoid. To address the privacy and efficiency limitations above, we propose a 2-PC privacy-preserving inference framework, SecMoE. Unifying per-entry circuits in both the MoE layer and piecewise polynomial functions, SecMoE obliviously selects the extracted parameters from circuits and only computes one encrypted entry, which we refer to as Select-Then-Compute. This makes the model for private inference scale to 63× larger while only having a 15.2× increase in end-to-end runtime. Extensive experiments show that, under 5 expert settings, SecMoE lowers the end-to-end private inference communication by 1.8~7.1× and achieves 1.3~3.8× speedup compared to the state-of-the-art (SOTA) protocols.

Large language models (LLMs) with the Mixture-of-Experts (MoE) architecture achieve high cost-efficiency by selectively activating a subset of the parameters. Despite the inference efficiency of MoE LLMs, the training of extensive experts from scratch incurs substantial overhead, whereas reconstructing a dense LLM into an MoE LLM significantly reduces the training budget. However, existing reconstruction methods often overlook the diversity among experts, leading to potential redundancy. In this paper, we come up with the observation that a specific LLM exhibits notable diversity after being pruned on different calibration datasets, based on which we present a Diversity-Enhanced reconstruction method named DIVE. The recipe of DIVE includes domain affinity mining, pruning-based expert reconstruction, and efficient retraining. Specifically, the reconstruction includes pruning and reassembly of the feed-forward network (FFN) module. After reconstruction, we efficiently retrain the model on routers, experts and normalization modules. We implement DIVE on Llama-style LLMs with open-source training corpora. Experiments show that DIVE achieves training efficiency with minimal accuracy trade-offs, outperforming existing pruning and MoE reconstruction methods with the same number of activated parameters. Code is available at: https://github.com/yuchenblah/DIVE.

With the rapid advancement of wireless communication technologies, the scarcity of available spectrum resources has become increasingly pronounced. Dynamic Spectrum Access (DSA) emerges as a promising solution to address this challenge. Traditional DSA methods based on Q-learning emphasize autonomous learning by individual nodes, whereas more recent approaches incorporating Federated Learning (FL) introduce collaborative learning among nodes but remain reliant on a central server. In this paper, we propose a novel DSA scheme based on Swarm Learning (SL), which enables a fully decentralized, distributed machine learning paradigm by establishing a blockchain-based peer-to-peer network. This approach capitalizes on the strengths of SL, facilitating cooperative learning among multiple nodes to enhance DSA performance. By allowing IoT terminals to share model parameters within a blockchain framework, the proposed scheme mitigates the vulnerabilities associated with centralized servers. Simulation results demonstrate that the SL-based DSA scheme not only surpasses the access efficiency of FL-based methods but also obviates the necessity of a central aggregation server. Furthermore, the fully decentralized architecture enhances the auditability of system data, thereby bolstering user privacy protection.

With the advancement of wireless communication technologies, available spectrum resources are becoming increasingly scarce. Dynamic Spectrum Access (DSA) is one of the effective approaches to address the challenge. Traditional Q-learning DSA relies on node self-learning, while recent Federated Learning (FL) DSA introduces node collaboration but still depends on a central server. This paper proposes a DSA scheme based on Swarm Deep Reinforcement Learning (SDRL), achieving a fully decentralized distributed machine learning through the construction of a blockchain-based peer-to-peer network. This scheme leverages the advantages of swarm learning (SL), utilizing collaborative learning among multiple nodes to enhance DSA performance. IoT terminals share model parameters, utilizing the benefits of blockchain networks to mitigate the risks associated with centralized servers. Simulation results demonstrate that the SDRL scheme not only improves DSA access efficiency compared to FL-based schemes but also eliminates the need for a central aggregation server. The fully decentralization architecture enhances the auditablity of the data in the system which further preserves each user’s privacy.

Low Earth orbit (LEO) satellite systems play an important role in next generation communication networks due to their ability to provide extensive global coverage with guaranteed communications in remote areas and isolated areas where base stations cannot be cost-efficiently deployed. With the pervasive adoption of LEO satellite systems, especially in the LEO Internet-of-Things (IoT) scenarios, their spectrum resource management requirements have become more complex as a result of massive service requests and high bandwidth demand from terrestrial terminals. For instance, when leasing the spectrum to terrestrial users and controlling the uplink transmit power, satellites collect user data for machine learning purposes, which usually are sensitive information such as location, budget and quality of service (QoS) requirement. To facilitate model training in LEO IoT while preserving the privacy of data, blockchain-driven federated learning (FL) is widely used by leveraging on a fully decentralized architecture. In this paper, we propose a hybrid spectrum pricing and power control framework for LEO IoT by combining blockchain technology and FL. We first design a local deep reinforcement learning algorithm for LEO satellite systems to learn a revenue-maximizing pricing scheme. Then the agents collaborate to form an FL system. We also propose a reputation-based blockchain which is used in the global model aggregation phase of FL to optimize the power control. Based on the reputation mechanism, a node is selected for each global training round to perform model aggregation and block generation, which can further enhance the decentralization of the network and guarantee the trust. Simulation tests are conducted to evaluate the performances of the proposed scheme. Our results show the efficiency of finding the maximum revenue scheme for LEO satellite systems while preserving the privacy of each agent.

Low Earth orbit (LEO) satellite systems are receiving increasing attention as they provide extensive global coverage. Secure and efficient management of limited spectrum bands and power resources are crucial for controlling operational costs and ensuring reliable communication in LEO satellite systems. However, spectrum pricing and power control optimization are challenging tasks. First, dynamic pricing is needed for leasing idle satellite spectrum to terrestrial users, as it must consider user mobility and real-time demand changes. Additionally, there is a trust concern that when utilizing the leased spectrum, terrestrial users may maliciously exceed limited transmit power to improve the quality of service (QoS). Moreover, users' privacy should be protected because the data collected by satellites often contain sensitive information such as location, budget, and QoS needs. In this paper, we propose a hybrid spectrum pricing and power control framework for LEO satellite-enabled networks to mitigate the above concerns by combining blockchain technology and Federated Learning (FL). We first design a local deep reinforcement learning algorithm for LEO satellite systems to learn a revenue-maximizing pricing strategy and power control scheme. Subsequently, these individual agents collaborate to establish an FL system without sharing their sensitive raw data. We also propose a reputation-based blockchain used in the global model aggregation phase to further enhance the traceability of the network and guarantee the trust. We conduct simulation tests to evaluate the efficacy of the proposed scheme, and our results show its capability to efficiently find the maximum revenue scheme for LEO satellite systems while preserving the privacy of each participating agent in an auditable mode.

In collaborative software development, programmers create software branches to add features and fix bugs tentatively, and then merge branches to integrate edits. When edits from different branches textually overlap (i.e., textual conflicts ) or lead to compilation and runtime errors (i.e., build and test conflicts ), it is challenging for developers to remove such conflicts. Prior work proposed tools to detect and solve conflicts. They investigate how conflicts relate to code smells and the software development process. However, many questions are still not fully investigated, such as what types of conflicts exist in real-world applications and how developers or tools handle them. For this article, we used automated textual merge, compilation, and testing to reveal three types of conflicts in 208 open-source repositories: textual conflicts, build conflicts (i.e., conflicts causing build errors), and test conflicts (i.e., conflicts triggering test failures). We manually inspected 538 conflicts and their resolutions to characterize merge conflicts from different angles. Our analysis revealed three interesting phenomena. First, higher-order conflicts (i.e., build and test conflicts) are harder to detect and resolve, while existing tools mainly focus on textual conflicts. Second, developers manually resolved most higher-order conflicts by applying similar edits to multiple program locations; their conflict resolutions share common editing patterns implying great opportunities for future tool design. Third, developers resolved 64% of true textual conflicts by keeping complete edits from either a left or right branch. Unlike prior studies, our research for the first time thoroughly characterizes three types of conflicts, with a special focus on higher-order conflicts and limitations of existing tool design. Our work will shed light on future research of software merge.

Transformer-based pre-trained language models (PLMs) mostly suffer from excessive overhead despite their advanced capacity.For resource-constrained devices, there is an urgent need for a spatially and temporally efficient model which retains the major capacity of PLMs.However, existing statically compressed models are unaware of the diverse complexities between input instances, potentially resulting in redundancy and inadequacy for simple and complex inputs.Also, miniature models with early exiting encounter challenges in the trade-off between making predictions and serving the deeper layers.Motivated by such considerations, we propose a collaborative optimization for PLMs that integrates static model compression and dynamic inference acceleration.Specifically, the PLM is slenderized in width while the depth remains intact, complementing layer-wise early exiting to speed up inference dynamically.To address the trade-off of early exiting, we propose a joint training approach that calibrates slenderization and preserves contributive structures to each exit instead of only the final layer.Experiments are conducted on GLUE benchmark and the results verify the Pareto optimality of our approach at high compression and acceleration rate with 1/8 parameters and 1/19 FLOPs of BERT.

Emergence of massive dynamic objects will diversify spatial structures when robots navigate in urban environments. Therefore, the online removal of dynamic objects is critical. In this paper, we introduce a novel online removal framework for highly dynamic urban environments. The framework consists of the scan-to-map front-end and the map-to-map back-end modules. Both the front- and back-ends deeply integrate the visibility-based approach and map-based approach. The experiments validate the framework in highly dynamic simulation scenarios and real-world dataset.

In software merge, the edits from different branches can textually overlap (i.e., textual conflicts) or cause build and test errors (i.e., build and test conflicts), jeopardizing programmer productivity and software quality. Existing tools primarily focus on textual conflicts; few tools detect higher-order conflicts (i.e., build and test conflicts). However, existing detectors of build conflicts are limited. Due to their heavy usage of automatic build, current detectors (e.g., Crystal) only report build errors instead of identifying the root causes; developers have to manually locate conflicting edits. These detectors only help when the branches-to-merge have no textual conflict.

With the potential of implementing computing-intensive applications, edge computing is combined with digital twinning (DT)-empowered Internet of vehicles (IoV) to enhance intelligent transportation capabilities. By updating digital twins of vehicles and offloading services to edge computing devices (ECDs), the insufficiency in vehicles’ computational resources can be complemented. However, owing to the computational intensity of DT-empowered IoV, ECD would overload under excessive service requests, which deteriorates the quality of service (QoS). To address this problem, in this article, a multiuser offloading system is analyzed, where the QoS is reflected through the response time of services. Then, a service offloading (SOL) method with deep reinforcement learning, is proposed for DT-empowered IoV in edge computing. To obtain optimized offloading decisions, SOL leverages deep Q-network (DQN), which combines the value function approximation of deep learning and reinforcement learning. Eventually, experiments with comparative methods indicate that SOL is effective and adaptable in diverse environments.

The automotive industry, a key part of industrial Internet of Things, is now converging with cognitive computing (CC) and leading to industrial cognitive Internet of Vehicles (CIoV). As the major data source of industrial CIoV, social media has a significant impact on the quality of service (QoS) of the automotive industry. To provide vehicular social media services with low latency and high reliability, edge computing is adopted to complement cloud computing by offloading CC tasks to the edge of the network. Generally, task offloading is implemented based on the premise that edge servers (ESs) are appropriately quantified and located. However, the quantification of ESs is often offered according to empirical knowledge, lacking analysis on real condition of intelligent transportation system (ITS). To address the abovementioned problem, a collaborative method for the quantification and placement of ESs, named CQP, is developed for social media services in industrial CIoV. Technically, CQP begins with a population initializing strategy by Canopy and K-medoids clustering to estimate the approximate ES quantity. Then, nondominated sorting genetic algorithm III is adopted to achieve solutions with higher QoS. Finally, CQP is evaluated with a real-world ITS social media data set from China.

With the advent of the Internet of Vehicles (IoV), drivers are now provided with diverse time-sensitive vehicular services that usually require a large scale of computation. As civilian vehicles are generally insufficient in computational resources, their service requests are offloaded to cloud data centers and edge computing devices (ECDs) with ample computational resources to enhance the quality of service (QoS). However, ECDs are often overloaded with excessive service requests. In addition, as the network conditions and service compositions are complicated and dynamic, the centralized control of ECDs is hard to achieve. To tackle these challenges, a dynamic task offloading method with minority game (MG) in cloud-edge computing, named DOM, is proposed in this paper. Technically, MG is an effective tool with a distributed mechanism which can minimize the dependency on centralized control in resource allocation. In the MG, reinforcement learning (RL) is applied to optimize the distributed decision-making of participants. Finally, with a real-world dataset of IoV services, the effectiveness and adaptability of DOM are evaluated.