Recently, various strategies for distributed training of large language models (LLMs) have been proposed. By categorizing them into basic strategies and composite strategies, we have discovered that existing basic strategies provide limited options in specific scenarios, leaving considerable room for optimization in training speed. In this paper, we rethink the impact of memory and communication costs on the training speed of LLMs, taking into account the impact of intra- and inter-group communication performance disparities, and then propose a new set of basic strategies named the \textbf{Pa}rtial \textbf{R}edundancy \textbf{O}ptimizer (PaRO). PaRO Data Parallelism (PaRO-DP) accelerates LLM training through refined model state partitioning and tailored training procedures. At the same time, PaRO Collective Communications (PaRO-CC) speeds up collective communication operations by rearranging the topology. We also propose a guideline for choosing different DP strategies based on simple quantitative calculations, which yields minimal ranking errors. Our experiments demonstrate that PaRO improves the training speed of LLMs by up to 266\% that of ZeRO-3 as basic DP strategies. Moreover, employing PaRO-CC independently for model parallel strategies, such as Megatron, can also boost the training speed by 17\%.

In the millimeter wave band, the beam codebook based analog beamforming is adopted for its trade-off between cost and performance. The beam codebook is a matrix containing a set of beam weight vectors that can be utilized to generate different beams. These beams occupy the same system bandwidth by time division multiplexing. In conventional schemes, the beam codebook is fixed hence the time-frequency resources allocated to the users in these beams can not be flexibly adjusted. However, the user traffic is dynamically varying. When the traffic increases sharply, more resources are expected and when the traffic decreases sharply, the resources can be wasted. Thus the fixed beam codebook can not match the user traffic causing unbalanced resource allocation reducing the system throughput. To overcome this problem, the adaptive beam codebook management algorithm is proposed in this paper. Specifically, the low traffic beams are merged while the high traffic beams are divided into narrower beams with higher beam gain. Then the transmission time slots saved by merging beams can be transferred to the divided beams enabling the users to reuse frequency resources. Simulation results show that the adaptive beam codebook can achieve higher system throughput than the fixed beam codebook.

In the course of recommending locations for establishing new facilities on urban planning or commercial programming, the location prediction offers the optimal candidates, which maximizes the number of served customers or minimize customer inconvenience, therefore brings the maximum profits. In most existing studies, only the spatial-temporal features are recognized to evaluate the location popularity, where social relationships of customers, which are significant factors for popularity assessing, have been ignored. Additionally, current researches also fail to take capacities and categories of the facilities into consideration. To overcome the drawbacks, we introduce a novel model of Multi-characteristic Information based Top-k Location Prediction (MITLP), it captures the spatio-temporal behaviors of customers based on historical trajectories, exploits the social relevancy from their friend relationships, as well as examines the category competitiveness of specific facilities thoroughly. Subsequently, by drawing on the feature evaluation and popularity quantization, MITLP will be implemented within a hybrid B-tree-liked recommending framework, Constrained Location and Social-Trajectory Clustered forest (CLSTC-forest), which can not only produce better performance in practice but also address the facility service constraints. Finally, extensive experiments conducted on real-world datasets demonstrate the higher efficiency and effectiveness of the proposed model.

Abstract Online group buying is a burgeoning business model of Internet shopping, in which people with the same merchandise interests form a group and co-purchase goods with favorable prices. The buyer who launches the co-purchase is called the initiator, and other buyers are called the co-purchasers. Although recommending co-purchasers for a target buyer (co-purchase initiator) on the group buying is an interesting problem, existing studies have paid few attention to this topic. Different from the collaborator recommendation that only considers users with high similarity to the target user, co-purchaser recommendation takes both users with high and weak similarity into account, and the recommendation results can achieve high recall and diversity. However, the task turns out to be a challenging problem since it is hard to make a precise recommendation for buyers with weak similarity. To address the problem, we propose the following two methods. In the first one, we directly impose a penalty to the weak similar co-purchasers in the embedding space. To further improve the recommendation performance, in the second one, we smoothly increase the co-occurrence probability of the weak similar co-purchasers by truncated bias walk. Our experimental results on real datasets show that the proposed methods, particularly the latter, can effectively complete the co-purchaser recommendation and has high recommendation performance. In addition, considering that co-purchase may last longer, the total recommendation result can be generated in multiple stages and adjust the current recommendation list based on the feedback from the recommendation of previous stages. It is a trick for all co-purchaser recommendation methods to make the total result better.

To realize the on-board flood monitoring of SAR image, this paper proposed a SAR flood image change detection algorithm and its FPGA-based hardware implementation. The algorithm combines logarithm ratio and wavlet based multiscale analysis to generate the difference map,and the change region is obtained by CFAR thresholding method. The hardware implementation environment is (Xilinx XC7VH580Tflg1155-2). experimental results confirm the ectiveness of the proposed method in the respect of accuracy; in terms of running speed, the 512*512 image hardware implementation based on multiscale wavelet analysis takes only 50ms, which is ten times faster than PC implementation. hence the hardware implementation of the proposed method has good performance in terms of running speed and change detection accuracy.

As steganography is the main practice of hidden writing, many deep neural networks are proposed to conceal secret information into images, whose invisibility and security are unsatisfactory. In this paper, we present an encoder-decoder framework with an adversarial discriminator to conceal messages or images into natural images. The message is embedded into QR code first which significantly improves the fault-tolerance. Considering the mean squared error (MSE) is not conducive to perfectly learn the invisible perturbations of cover images, we introduce a texture-based loss that is helpful to hide information into the complex texture regions of an image, improving the invisibility of hidden information. In addition, we design a truncated layer to cope with stego image distortions caused by data type conversion and a moment layer to train our model with varisized images. Finally, our experiments demonstrate that the proposed model improves the security and visual quality of stego images.