The click-through rate (CTR) prediction task is to predict whether a user will click on the recommended item. As mind-boggling amounts of data are produced online daily, accelerating CTR prediction model training is critical to ensuring an up-to-date model and reducing the training cost. One approach to increase the training speed is to apply large batch training. However, as shown in computer vision and natural language processing tasks, training with a large batch easily suffers from the loss of accuracy. Our experiments show that previous scaling rules fail in the training of CTR prediction neural networks. To tackle this problem, we first theoretically show that different frequencies of ids make it challenging to scale hyperparameters when scaling the batch size. To stabilize the training process in a large batch size setting, we develop the adaptive Column-wise Clipping (CowClip). It enables an easy and effective scaling rule for the embeddings, which keeps the learning rate unchanged and scales the L2 loss. We conduct extensive experiments with four CTR prediction networks on two real-world datasets and successfully scaled 128 times the original batch size without accuracy loss. In particular, for CTR prediction model DeepFM training on the Criteo dataset, our optimization framework enlarges the batch size from 1K to 128K with over 0.1% AUC improvement and reduces training time from 12 hours to 10 minutes on a single V100 GPU. Our code locates at github.com/bytedance/LargeBatchCTR.

Gait training is widely used to treat gait abnormalities. Traditional gait measurement systems are limited to instrumented laboratories. Even though gait measurements can be made in these settings, it is challenging to estimate gait parameters robustly in real-time for gait rehabilitation, especially when walking over-ground. In this paper, we present a novel approach to track the continuous gait cycle during overground walking outside the laboratory. In this approach, we instrument standard footwear with a sensorized insole and an inertial measurement unit. Artificial neural networks are used on the raw data obtained from the insoles and IMUs to compute the continuous percentage of the gait cycle for the entire walking session. We show in this paper that when tested with novel subjects, we can predict the gait cycle with a Root Mean Square Error (RMSE) of 7.2%. The onset of each cycle can be detected within an RMSE time of 41.5 ms with a 99% detection rate. The algorithm was tested with 18840 strides collected from 24 adults. In this paper, we tested a combination of fully-connected layers, an Encoder-Decoder using convolutional layers, and recurrent layers to identify an architecture that provided the best performance.