Modern sensors generate rich, high-fidelity data, yet applications operating on wearable or remote sensing devices remain constrained by bandwidth and power budgets. Standardized JPEG and MPEG codecs achieve efficient trade-offs between bit-rate and quality for audio, images, and video, but have limited applicability for machine-perception tasks and non-traditional modalities. Recent generative neural codecs, or tokenizers, require millions of samples to train and are impractical for resource-constrained environments due to their large DNN-based analysis transforms. We introduce a Lightweight, Versatile, and Asymmetric neural codec design (LiVe-Action), focusing on two key ideas. (1) To reduce the complexity of the encoder, we impose an FFT-like structure and reduce the overall size and depth of the neural-network-based analysis transform. (2) To reduce the training burden and increase versatility for other types of signals, we remove adversarial and perceptual losses and focus on MSE loss with a simplified rate penalty. We release our code, experiments, and python library at https://github.com/ut-sysml/liveaction.

Modern sensors produce increasingly rich streams of high-resolution data. Due to resource constraints, machine learning systems discard the vast majority of this information via resolution reduction. Instead, compressed-domain learning allows models to operate on compact latent representations. However, existing compression strategies are ill-suited for compressed learning. Linear transform coding and end-to-end learned compression systems reduce bitrate, but do not uniformly reduce dimensionality. Thus, they still need high compute. Generative autoencoders reduce dimensionality, but their adversarial or perceptual objectives result in loss of detail, leading to decreased model quality. To address these limitations, we introduce WaLLoC (Wavelet Learned Lossy Compression), a neural codec architecture that combines linear transform coding with nonlinear dimensionality-reducing autoencoders. WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottle-neck between an invertible wavelet packet transform. Across several key metrics, WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion models. WaLLoC does not require perceptual or adversarial losses to represent high-frequency detail, providing compatibility with modalities beyond RGB images and stereo audio. WaLLoC's encoder consists almost entirely of linear operations, making it exceptionally efficient and suitable for mobile computing and remote sensing. It also enables learning directly from compressed data. We demonstrate WaLLoC's capability for compressed-domain learning across several tasks, including image classification, colorization, document understanding, and music source separation. Our code, experiments, and pre-trained audio and image codecs are available at https://ut-sysml.org/walloc/.

We evaluate various perception models—–including image classification, segmentation, speech recognition, and music source separation—–under severe lossy compression. Figure 1 summarizes the results underlining our insights. The datasets in the top row originally use near-lossless quality levels (ratios of about 5:1), while those in the bottom row are lossless. We apply additional compression to these six datasets using conventional, neural, and generative codecs, resulting in ratios between 20:1 and 1000:1. Our results indicate three key findings: (1) across nearly all tasks, generative compression methods like HiFiC and EnCodec provide the best performance despite having the lowest bitrates; (2) downstream performance correlates strongly with deep similarity metrics like LPIPS; and (3) Using lossy compressed datasets like ImageNet for pre-training can lead to counter-intuitive scenarios where severe lossy compression improves performance rather than degrading it. Our results provide a basis for integrating more potent compression into perception systems. Our code and experiments are available at: https://github.com/danjacobellis/MPQ .