With the rapid expansion in the scale of large language models (LLMs), enabling efficient distributed inference across multiple computing units has become increasingly critical. However, communication overheads from popular distributed inference techniques such as Tensor Parallelism pose a significant challenge to achieve scalability and low latency. Therefore, we introduce a novel optimization technique, Sync-Point Drop (SPD), to reduce communication overheads in tensor parallelism by selectively dropping synchronization on attention outputs. In detail, we first propose a block design that allows execution to proceed without communication through SPD. Second, we apply different SPD strategies to attention blocks based on their sensitivity to the model accuracy. The proposed methods effectively alleviate communication bottlenecks while minimizing accuracy degradation during LLM inference, offering a scalable solution for diverse distributed environments: SPD offered about 20% overall inference latency reduction with $<$ 1% accuracy regression for LLaMA2-70B inference over 8 GPUs.

This paper studies the wireless scheduling design to coordinate the transmissions of (local) model parameters of federated learning (FL) for a swarm of unmanned aerial vehicles (UAVs). The overall goal of the proposed design is to realize the FL training and aggregation processes with a central aggregator exploiting the sensory data collected by the UAVs but it considers the multi-hop wireless network formed by the UAVs. Such transmissions of model parameters over the UAV-based wireless network potentially cause large transmission delays and overhead. Our proposed framework smartly aggregates local model parameters trained by the UAVs while efficiently transmitting the underlying parameters to the central aggregator in each FL global round. We theoretically show that the proposed scheme achieves minimal delay and communication overhead. Extensive numerical experiments demonstrate the superiority of the proposed scheme compared to other baselines.

Metaverse allows a 3D virtual mapping of the physical world to the digital world in which users interact with each other via digital avatars with a wide range of virtual activities. To realize this, the metaverse will inevitably employ numerous machine learning (ML) systems to enable the virtual-physical mapping process and offer intelligent virtual services to metaverse users (MUs). However, metaverse service providers (MSPs), who need ML models for their services (e.g., virtual events and healthcare services), may not have the expertise or resources required to build these underlying ML models. In addition, although ML models can be offered by a crowd of experienced ML workers (MLWs), the MLWs might not be able to collect the desired data for training their ML models due to privacy issues and the large-scale, distributed nature of the metaverse. In this paper, we propose MetaCrowd, a blockchain-based ML crowdsourcing framework that aims to overcome the mentioned issues and make ML accessible to a wide range of MUs and MSPs. Unlike traditional crowdsourcing systems which rely on central authorities, MetaCrowd is decentralized and automatic thanks to blockchain and smart contracts, thereby mitigating the single point of failure and trust issues. Experimental results illustrate the efficiency of MetaCrowd in both performance and cost. In addition, a decentralized application is also implemented and published widely to show its feasibility in practice.

Machine learning (ML)-based malware detection plays a crucial role in cyber-security by enabling the identification of potential malware threats without relying solely on predefined signatures or rules. Conventional ML approaches require a feature engineering step to analyze and convert collected data (e.g., captured network traffic and malware programs) into a format suitable for model training and prediction. However, this particular step typically requires a considerable depth of domain-specific expertise and also adds additional complexity to the learning process. To mitigate this limitation, we propose to perform malware detection directly based on the byte-level representation of malware data. We employ a byte embedding layer to convert byte sequences into higher-dimension representations. Then, we employ the multi-head attention technique to capture their correlation before forwarding the output to a fully connected deep neural network for malware detection. Extensive experiments on multiple datasets with diverse file formats demonstrated the superior performance of our proposed method. Additionally, we performed an ablation study on the role of the byte-embedding layer to show that our approach does not depend on a high embedding dimension for strong predictive performance, which helps reduce training complexity.

The emerging metaverse is envisioned as a virtual mapping of the real world, thus it would inevitably employ numerous Machine Learning (ML) frameworks to analyze and process massive data for the virtual-physical synchronization process. As a distributed ML paradigm, Federated Learning (FL) can naturally take advantage of numerous IoT, wearable devices, and edge, cloud servers under the metaverse infrastructure to train ML models with privacy guarantee. However, the large-scale and decentralized nature of the metaverse can pose significant challenges to traditional FL schemes, where there is a centralized server aggregating the local models received from local devices. It is not only vulnerable to Single Point of Failure (SPoF), but also lacks incentive mechanisms encouraging metaverse users to contribute their resources and data. In this paper, we propose BFLMeta, a blockchain-based FL scheme for the metaverse in which the aggregation process is performed in a decentralized manner, while the framework can estimate the non-IID degree of data to flexibly adjust blockchain committee size, thereby mitigating the impact of malicious aggregators. Security analysis shows that BFLMeta can resist SPoF, poisoning attack, privacy leakage, and sybil attack. Besides, our evaluation on computation, communication, and performance illustrates the efficiency of BFLMeta. Notably, BFLMeta can converge even with more than 50% poisoning nodes.