The emergence of Large Language Models (LLMs) has necessitated the adoption of distributed training techniques, involving the deployment of thousands of GPUs to train a single model. Unfortunately, the efficiency of large-scale distributed training systems is often suboptimal due to the increased likelihood of hardware errors in high-end GPU products and the heightened risk of network traffic collisions. Specifically, GPUs involved in the same job require periodic synchronization to exchange necessary data, such as gradients, parameters, or activations. As a result, any local hardware failure can disrupt training tasks, and the inability to swiftly identify faulty components leads to a significant waste of GPU resources. Moreover, prolonged communication due to traffic collisions can substantially increase GPU waiting times. To address these challenges, we propose a communication-driven solution, namely the C 4. The key insights of C 4 are twofold. First, the load in distributed training exhibits homogeneous characteristics and is divided into iterations through periodic synchronization, therefore hardware anomalies would incur certain syndrome in collective communication. By leveraging this feature, $\mathbf{C} 4$ can rapidly identify the faulty components, swiftly isolate the anomaly, and restart the task, thereby avoiding resource wastage caused by delays in anomaly detection. Second, the predictable communication model of collective communication, involving a limited number of long-lived flows, allows C 4 to efficiently execute traffic planning, substantially reducing bandwidth competition among these flows. The $\mathbf{C 4}$ has been extensively deployed across real-world production systems in a hyperscale cloud provider, yielding a significant improvement in system efficiency, from 30% to $\mathbf{4 5 \%}$. This enhancement is attributed to a $\mathbf{3 0 \%}$ reduction in error-induced overhead and a 15% reduction in communication costs.

Scene graph generation (SGG) is a structured approach to understanding real-world scenes with complex relations, which can enhance UAV autonomy in unfamiliar environments. However, SGG typically has numerous model parameters that require considerable computational resources. This paper proposes a refined SGG model and reduces the model parameters for its UAV applications. First, we use subject-object query pairs to predict triplets directly, eliminating the need for separate entity predictions. Additionally, the cross-attention mechanism enhances the model’s ability to query triplets. We use a single decoder to process subject and object entities simultaneously, enhancing computational speed and reducing the number of parameters. Then, we map the entities to the relational semantic space before performing relations classification, which improves the model performance by adding a small number of parameters. Finally, the set prediction loss function is designed for relation prediction to strengthen the role of relation prediction in triplets. Real-world UAV experiments show that our model can extract more triplets per second with fewer parameters than the benchmarks. Github: https://github.com/SupersPig/myLGTR.

In this letter, we propose a novel multi-target association algorithm that fuses topological and visual features, and this algorithm improves target discrimination. The fused features are first constructed, and then a similarity calculation method is proposed to aggregate neighbor similarities to the central node based on topological similarity, which can be iterated several times to enhance the differentiation between targets. The experimental results demonstrate the advantages of the proposed algorithm. Github:https://github.com/SupersPig/FTVF

Bundle adjustment (BA) has been proven to improve the accuracy of the LiDAR mapping, but has not yet been properly employed in a dead-reckoning navigation system. In this paper, we present a frame-to-frame (F2F) BA for LiDAR-inertial navigation, named BA-LINS. Based on the direct F2F point-cloud association method, the same-plane points are associated among the LiDAR keyframes. Hence, the F2F plane-point BA measurement model can be constructed using the same-plane points. The LiDAR BA and the inertial measurement unit (IMU)-preintegration measurements are tightly coupled under the framework of factor graph optimization. Meanwhile, an effective adaptive covariance estimation algorithm for LiDAR BA measurements is proposed to improve the accuracy further. Exhaustive experiment results on public and private datasets demonstrate that BA-LINS yields superior accuracy to state-of-the-art methods. Compared to the baseline system FF-LINS, the absolute translation accuracy and state-estimation efficiency of BA-LINS are improved by 29.5% and 28.7%, respectively. Besides, the proposed adaptive covariance estimation algorithm exhibits notably improved accuracy and robustness.

Training data pre-processing pipelines are essential to deep learning (DL). As the performance of model training keeps increasing with both hardware advancements (e.g., faster GPUs) and various software optimizations, the data pre-processing on CPUs is becoming more resource-intensive and a severe bottleneck of the pipeline. This problem is even worse in the cloud, where training jobs exhibit diverse CPU-GPU demands that usually result in mismatches with fixed hardware configurations and resource fragmentation, degrading both training performance and cluster utilization. We introduce GoldMiner, an input data processing service for stateless operations used in pre-processing data for DL model training. GoldMiner decouples data pre-processing from model training into a new role called the data worker. Data workers facilitate scaling of data pre-processing to anywhere in a cluster, effectively pooling the resources across the cluster to satisfy the diverse requirements of training jobs. GoldMiner achieves this decoupling in a fully automatic and elastic manner. The key insight is that data pre-processing is inherently stateless, thus can be executed independently and elastically. This insight guides GoldMiner to automatically extract stateless computation out of a monolithic training program, efficiently disaggregate it across data workers, and elastically scale data workers to tune the resource allocations across jobs to optimize cluster efficiency. We have applied GoldMiner to industrial workloads, and our evaluation shows that GoldMiner can transform unmodified training programs to use data workers, accelerating individual training jobs by up to 12.1x. GoldMiner also improves average job completion time and aggregate GPU utilization by up to 2.5x and 2.1x in a 64-GPU cluster, respectively, by scheduling data workers with elasticity.

The development of personalized recommendation has significantly improved the accuracy of information matching and the revenue of e-commerce platforms. Recently, it has two trends: 1) recommender systems must be trained timely to cope with ever-growing new products and ever-changing user interests from online marketing and social network; 2) state-of-the-art recommendation models introduce deep neural network (DNN) modules to improve prediction accuracy. Traditional CPU-based recommender systems cannot meet these two trends, and GPU-centric training has become a trending approach. However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in Computer Vision (CV) and Neural Language Processing (NLP) areas. This issue can be explained by two characteristics of these recommendation models: First, they contain up to a thousand of input feature fields, introducing fragmentary and memory-intensive operations; Second, the multiple constituent feature interaction submodules introduce substantial small-sized compute kernels. To remove this roadblock to the development of recommender systems, we propose a novel framework named PICASSO to accelerate the training of recommendation models on commodity hardware. Specifically, we conduct a systematic analysis to reveal the bottlenecks encountered in training recommendation models. We leverage the model structure and data distribution to unleash the potential of hardware through our packing, interleaving, and caching optimization. Experiments show that PICASSO increases the hardware utilization by an order of magnitude on the basis of state-of-the-art baselines and brings up to 6× throughput improvement for a variety of industrial recommendation models. Using the same hardware budget in production, PICASSO on average shortens the walltime of daily training tasks by 7 hours, significantly reducing the delay of continuous delivery.

Wearable sensors have many potential applications in health and tactile touch monitoring. In particular, Liquid Metal (LM) based wearable sensors have attracted more and more attentions as they can measure large deformations without failure. However, most of the current LM-based wearable sensors are 2D flaky structure, which making them position unstable while wearing, and induce measurement errors. In this report, a novel finger bending angle monitoring sensor with 3D hollow design based on embedded LM microchannels is presented. Because of the hollow design, this sensor can be easily worn on the finger joints and can accurately distinguish the finger bending angle. And real-time monitoring can be achieved by measuring the inductance value. The results suggest that the studied wearable sensor has excellent stability and reliability. And this sensor has better practical applications in finger action health detection.

Firstly, this paper designs the process of personalized recommendation method based on knowledge graph, and constructs user interest model. Second, the traditional personalized recommendation algorithms are studied and their advantages and disadvantages are analyzed. Finally, this paper focuses on the combination of knowledge graph and collaborative filtering recommendation algorithm. They are effective to solve the problem where [Formula: see text] value is difficult to be determined in the clustering process of traditional collaborative filtering recommendation algorithm as well as data sparsity and cold start, utilizing the ample semantic relation in knowledge graph. If we use RDF data, which is distributed by the E and P (Exploration and Development) database based on the petroleum E and P, to verify the validity of the algorithm, the result shows that collaborative filtering algorithm based on knowledge graph can build the users’ potential intentions by knowledge graph. It is enlightening to query the information of users. In this way, it expands the mind of users to accomplish the goal of recommendation. In this paper, a collaborative filtering algorithm based on domain knowledge atlas is proposed. By using knowledge graph to effectively classify and describe domain knowledge, the problems are solved including clustering and the cold start in traditional collaborative filtering recommendation algorithm. The better recommendation effect has been achieved.

In the recognition of osteosarcoma magnetic resonance images (MRI), the probability of a pixel belonging to a class is not only related to its own features, but also closely correlated with the information distribution of the surrounding pixels. However, it is currently unable to recognize the osteosarcoma lesions and surrounding issues simultaneously. In order to solve the problem, we propose a fully automated approach to osteosarcoma MRI segmentation and recognition. It uses Conditional Random Field (CRF) model to incorporate multiple features, especially the texture context features, which are based on the relative position of pixels' texture and make a significant difference in more accurately determining which class a pixel belongs to. Further, we propose to model the mutual constraint relations between the targets (bone tumor, soft tissue, etc.) features in the osteosarcoma MRI, and train the tagging samples with the Joint-boost algorithm. Our experimental results show that the proposed method is effective and encouraging. It is especially superior for recognizing tumors with irregular shape and structure, which are identified with low accuracy in other methods.

In current research on mobile ad-hoc networks, test-beds are always needed. However, real test-beds are costly, and sometimes unfeasible, especially for large-scale mobile ad-hoc networks. Therefore, emulation systems are provided. Unfortunately, exiting emulation systems have obvious defects. They can hardly support scalability or real-time simultaneously, neither adding nor removing mobile node dynamically. In this paper, we propose a test system named ManTS. With distributed architecture and some novel approaches, such as direct transmittal and virtual application traffic, and others. ManTS is able to build test-beds for large-scale mobile ad-hoc networks and guarantee real-time emulation at the same time. Furthermore, IP or upper layer protocols and applications implemented on Linux can run on ManTS without any modification. And with ManTS, simulated mobile nodes can be added into and deleted from mobile ad-hoc networks freely, which is quite useable to test routing and transport protocols. In a word, ManTS is a more flexible test system, and really suitable to test IP and upper layer protocols and applications for large-scale mobile ad-hoc networks.

In the future, more and more services will be supplied over networks, which can bring end users great pleasure and convenience. It can become true if service providers can benefit from these services. For this purpose, authentication, authorization and accounting or AAA for short is needed. Unfortunately, current AAA architectures are designed without especially considering deploying new services for service providers and consuming various services for end users. In other words, these AAA architectures are not designed from aspects of service providers and service consumers both. To remedy this problem, a novel service-oriented AAA architecture is proposed in this paper. It introduces a novel AAA component named AAA agent, which resides in service equipments as a logical component. AAA agent provides a standard interface for the component that provides a specific service, namely service-providing server, and the interface has nothing to do with AAA details. It is AAA agent that deals with all AAA affairs for the service-proving server. Therefore all AAA details are hidden from service-proving server. Besides, a user credential mechanism is also proposed to enable users to roam among various services seamlessly only if they have rights. In a word, the proposed service-oriented AAA architecture is a novel AAA architecture that enables service providers to deploy new services easily and helps users to enjoy different services conveniently.