In large-scale industrial recommendation systems, model checkpoints are instrumental in maintaining training goodput and numerical correctness during system failures and job preemptions. The increasing prevalence of multi-terabyte models has rendered frequent regular model checkpoints impractical, resulting in substantial lost progress when recovering from failures. As model sizes continue to grow, researchers and practitioners are compelled to investigate more efficient and scalable solutions. This paper presents DECK, a novel approach to delta model checkpointing designed for real-world industrial systems. Specifically, DECK focuses on extracting delta states with near-zero overhead, staging and streaming delta checkpoints without interrupting the training process, and merging delta checkpoints in an optimal and decoupled manner. Experimental results demonstrate that DECK achieves a 12-fold increase in checkpoint frequency while maintaining negligible impact on training throughput, thereby attaining state-of-the-art (SOTA) production performance.

Generative adversarial network (GAN)-based image compression approaches reconstruct images with highly realistic quality at low bit rates. However, currently there is no published GAN-based image compression approach that utilizes advanced GAN losses, such as the Wasserstein GAN with gradient penalty loss (WGAN-GP), to improve the quality of reconstructed images. Meanwhile, existing deep learning-based image compression approaches require extra convolution layers to estimate and constrain the entropy during training, which makes the network larger and may require extra bits to send information to the decoder. In this paper, we propose a new GAN for image compression with novel discriminator and generator loss functions and a simple entropy estimation approach. Our new loss functions outperform the current GAN loss for low bitrate image compression. Our entropy estimation approach does not require extra convolution layers but still works well to constrain the number of bits during training.

Deep learning has been successfully applied to image and video compression. Specifically, generative adversarial network (GAN) can compress images at low bit rates with sharp details and high perceptual quality. In this work, we propose a novel generative video compression (GVC) model with a transformer-based discriminator (TD), which learns non-local correlations within video frames to improve adversarial training. Besides, our GVC model incorporates a new loss to train the generator, which combines a base loss, a discriminator-dependent feature loss, and a perceptual loss. Experiments on HEVC test sequences demonstrate that the proposed GVC model provides superior performance at extremely low bit rates, compared to existing learned and traditional video coding schemes.

Compressed sensing is an effective solution for signal acquisition and signal reconstruction at a much lower rate than the Nyquist rate. Traditional methods, such as orthogonal matching pursuit and basis pursuit, for image compressed sensing reconstruction have unsatisfying reconstruction quality and long reconstruction time. Researchers now focus on neural network and deep learning methods for the better reconstruction of compressed-sensed signals at a very low sampling rate and a fast speed. However, current neural network approaches for image compressed sensing do not consider the similarities between images or within images, or the types of image blocks; thus, performing poorly in images with complex contents. In this paper, we develop a novel neural network framework that utilizes the similarities between image blocks through Gaussian-mixture models without recording the similarity information to achieve better reconstruction quality than the state-of-the-art neural network methods for block-level image compressed sensing.

Deep learning methods have been actively applied in moving object detection and achieved great performance. However, many existing models render superior detection accuracy at the cost of high computational complexity and slow inference speed, which hindered the application on mobile and embedded devices with limited computing resources. In this paper, we propose a two-branch 3D separable convolutional neural network named “F3DsCNN” for moving object detection. The network extracts both high-level global features and low-level detailed features. It achieves a fast inference speed of 120 frames per second, suitable for tasks that need to be carried out in a timely manner on a computationally limited platform with high accuracy.

Line-of-sight wireless communications can benefit from the simultaneous transmission of multiple independent data streams through the same medium in order to increase system capacity. A common approach is to use conventional spatial multiplexing with spatially separated transmitter/receiver antennae, for which inter-channel crosstalk is reduced by employing multiple-input-multiple-output (MIMO) signal processing at the receivers. Another fairly recent approach to transmitting multiple data streams is to use orbital-angular-momentum (OAM) multiplexing, which employs the orthogonality among OAM beams to minimize inter-channel crosstalk and enable efficient (de)multiplexing. In this paper, we explore the potential of utilizing both of these multiplexing techniques to provide system design flexibility and performance enhancement. We demonstrate a 16 Gbit/s millimeter-wave link using OAM multiplexing combined with conventional spatial multiplexing over a short link distance of 1.8 meters (shorter than Rayleigh distance). Specifically, we implement a spatial multiplexing system with a 2 × 2 antenna aperture architecture, in which each transmitter aperture contains two multiplexed 4 Gbit/s data-carrying OAM beams. A MIMO-based signal processing is used at the receiver to mitigate channel interference. Our experimental results show performance improvements for all channels after MIMO processing, with bit-error rates of each channel below the forward error correction limit of 3.8 × 10-3. We also simulate the capacity for both the 4 × 4 MIMO system and the 2 × 2 MIMO with OAM multiplexing. Our work indicates that OAM multiplexing and conventional spatial multiplexing can be simultaneously utilized to provide design flexibility. The combination of these two approaches can potentially enhance system capacity given a fixed aperture area of the transmitter/receiver (when the link distance is within a few Rayleigh distances).

This paper reports an experimental investigation of using orthogonal frequency division multiplexing (OFDM) on orbital-angular momentum (OAM) channels when there is intersymbol interference (ISI) caused by multipath effects. The impulse response measurement indicates that due to the divergence of OAM beams, channels of larger OAM number are more affected by the ISI caused the reflections from surrounding objects. Channel performance of three different OAM numbers l = 0, +1 and +3 are studied. When a single QPSK channel is used, the more channel performance degradation in term of BER and EVM is observed for higher OAM channels. OFDM with 4 sub-carriers is used on the OAM channels to help reduce the ISI and improve the channel performance to some extent.

In this paper, the generation and steering of beams carrying orbital angular momentum utilizing a custom-designed circular antenna array has been demonstrated at 28 GHz. A steering angle as large as 30 degrees for an orbital angular momentum (OAM) beam has been achieved. The effect of number of antennas and the distance from antennas to the array center to the quality of beam generation and beam steering is investigated through both experiments and simulations. Our results indicate that: (1) As the steering angle increases, the mode purity of the generated OAM beams decreases; (2) Increasing the number of antennas improves the OAM mode purity; (3) For a fixed number of antennas, high mode purity is observed for lower order OAM modes; (4) Placing the antennas farther away from the array center allows for reduced divergence of the generate OAM beams.

This paper reports an experimental demonstration of a 32-Gbit/s wireless link using orbital angular momentum (OAM) and polarization multiplexing in a millimeter-wave regime at 60 GHz. Results of the analysis show that a higher carrier frequency reduces the propagation loss as well as the size of the transmitter and receiver, particularly for OAM channels with higher OAM values. Further, two different OAM channels (with l = +1 and l = +3) on each of the two polarizations are spatially multiplexed, and each channel carries 2-Gbaud signals with 16-QAM modulation. Spiral phase plates are used to generate 60-GHz OAM beams. The simulation results show that a higher carrier frequency plays a more significant role in reducing both the aperture size and the transmission loss for channels with higher OAM values. The bit error rates (BERs) of 4 channels are measured, and the raw BERs are found to be less than 3.8 × 10−3

We present a short-range experimental orbital-angular-momentum (OAM) multiplexing millimeter-wave communication system using patch antenna arrays. The dependence of the evolution of OAM beams on the number of array elements and array radius are analyzed. We also find phase delay deviation of 30 degrees leads to mode purity degrading to ∼1% and power deviation of 6 dB reduces the purity by <40%. An obstruction of size <10mm has not much influence on the evolution of OAM beams generated from antenna arrays with r = 8mm, which enables the stacking of multiple antenna arrays. We also use these patch antenna arrays to demonstrate a 60 GHz wireless communication link using two multiplexed OAM modes, each carrying a 500-Mbaud 16-QAM or 2-Gbaud QPSK signal. A channel crosstalk of less than −20 dB and bit-error-rates (BER) of less than 3.8 × 10−3 are achieved.

This paper reports on an experimental measurement and analysis of multipath-induced intra- and interchannel crosstalk effects in a mm-wave communications link using orbital angular momentum multiplexing at 28 GHz. The reflection is from an ideal reflector parallel to the propagation path. The intra-channel crosstalk effect is measured when a single OAM beam is transmitted, and inter-channel crosstalk effect is measured when 2 multiplexed OAM beams are transmitted. Both simulation and experimental results show that OAM channels with larger OAM number ℓ tend to have stronger intra-channel crosstalk because less power is received from the direct path and more power is received from the reflected path. This effect is caused by OAM beam divergence, as OAM beams with larger ℓ spread into a larger beam size and have less power in the beam center. For the same reason, OAM beams of larger ℓ lead to stronger inter-channel crosstalk with the other OAM channels.

We present the design and performance characterization of a thin metamaterial plate for generation of orbital angular momentum (OAM) modes of a millimeter-wave beam, which can carry independent data streams over the same physical medium. The plate has a thickness of 1.56 mm, and consists of 3.06 × 0.68 mm rectangular apertures with spatial variant orientations. It generates OAM beams l = +1 and l = +3 with mode purity larger than 77.5% over a bandwidth of 6 GHz (25-31 GHz). We then use these streams to experimentally demonstrate a 16-Gbit/s millimeter-wave wireless communications link using two multiplexed OAM modes, each carrying a 2-Gbaud 16-QAM signal. A channel crosstalk less than -20 dB over a bandwidth of 4 GHz (26-30 GHz) and biterror-rates (BER) less than 3.8 × 10-3are achieved.

This paper reports an experimental demonstration of a 16 Gbit/s millimeter-wave communication link using MIMO processing of 2 OAM modes on each of two transmitter/receiver antenna apertures. Two groups of multiplexed OAM beams, each containing OAM beams of ℓ =1 and +3 are generated and transmitted through two transmitter apertures respectively. The two transmitter apertures are separated with a certain distance such that the two groups of OAM beams are spatially overlapping at the receiver aperture plane. Each channel carries 1-GBaud 16-QAM signals at the same carrier frequency of 28 GHz. Our experimental results show that MIMO equalization processing can help mitigate the interferences from the other OAM channels and the BER performance of each channel improves significantly after MIMO processing. Our results indicate that OAM multiplexing and traditional spatial multiplexing combined with MIMO processing can be compatible and complementary with each other.

This paper reports an experimental demonstration of modal multiplexing using orbital angular momenta (OAM) in the millimeter-wave regime (28 GHz). Using 1 GBaud/s signals with 16-QAM modulation, a data rate of 32 Gbit/s is obtained by employing 4 different OAM modes (l= -3,-1,+1 and +3) on two orthogonal linear polarization states. Without any digital post-processing, the receiver can separate the different data streams since crosstalk among the OAM channels is low: measured to be better than -23dB with a 28 GHz narrowband carrier on a single polarization, and better than -16dB on both polarization states. The bit-error rate (BER) of 8 channels is measured, and found the raw BER is below 3.8×10-3, thus allowing efficient use of forward error correction (FEC) codes.

The upper bound of error rate performance of coded free space optical (FSO) communication system is studied in this paper. A new tightened upper bound over fully interleaved turbulence channel is proposed based on Gallager's second-bounding techniques. Two turbulence models are considered, log-normal model for weak fluctuation scenarios and negative exponential model for strong fluctuation scenarios. To simplify the computation complexity, the derived bound, the second version of the Duman-Salehi (DS2) exponential bound, is derived into closed-form expression by selecting an optimal tilting measure. Meanwhile, for comparison, the union bound is also discussed starting with the approximation of pairwise error probability. Furthermore, we present the analytical approach for average weight enumerator of turbo product codes. Then the DS2-exponential bound along with union bound are applied for these codes, of which performance bounds are seldom discussed in the literature. Simulation results are also presented to shed light on the tightness of our derived bounds.

The performance of free-space optical (FSO) communication suffers from atmospheric turbulence, which mainly results in scintillation and phase fluctuations to optical signal. In this paper, we study the bit error rate (BER) performance of coherent FSO system employing phase compensation and spatial diversity. Based on weak fluctuation theory, the amplitude fading and phase fluctuation induced by atmospheric turbulence are assumed to follow log-normal model and Gaussian distribution, respectively. Distinct from previous works, a new closed-form expression for signal fading at the receiver is derived. Then the impact of various compensating parameters is examined on the BER performance of the system, including the normalized receiver aperture diameter and the number of phase compensation modes. The optimal compensation parameters are found showing that the FSO coherent system can obtain the minimum BER. Furthermore, we explore the potential of spatial diversity, particularly receiver diversity to further enhance the system performance. Numerical results show that the addition of phase compensation and diversity techniques can largely reduce the impact of atmospheric turbulence.