Neural codecs and compression algorithms
Neural bandwidth reduction
April 23, 2020 — December 20, 2023
compsci
computers are awful
information
metrics
music
photon choreography
standards
Not compressing neural networks themselves, but rather using them to compress other things, in the sense of using neural nets to reconstruct signals (images, audio, video) with low error from a small bit-rate summary, especially in concert with existing noisy data transmission pipelines. Of course, we might try to do both at once and then think about minimum description length. that might be interesting.
- Descript Audio Codec / descriptinc/descript-audio-codec “ State-of-the-art audio codec with 90x compression factor. Supports 44.1kHz, 24kHz, and 16kHz mono/stereo audio.” (Kumar et al. 2023)
- Image Compression with Neural Networks – Google AI Blog
- mlomnitz/DiffJPEG
- rshin/differentiable-jpeg: Code for “JPEG-resistant Adversarial Images”
1 References
Ananthabhotla, Ewert, and Paradiso. 2019. “Towards a Perceptual Loss: Using a Neural Network Codec Approximation as a Loss for Generative Audio Models.” In Proceedings of the 27th ACM International Conference on Multimedia.
Collobert, Hannun, and Synnaeve. 2019. “A Fully Differentiable Beam Search Decoder.” In Proceedings of the 36th International Conference on Machine Learning.
Eusebio, Ascenso, and Pereira. 2021. “Optimizing an Image Coding Framework with Deep Learning-Based Pre- and Post-Processing.” In 2020 28th European Signal Processing Conference (EUSIPCO).
Guleryuz, Chou, Hoppe, et al. 2021. “Sandwiched Image Compression: Wrapping Neural Networks Around A Standard Codec.” In 2021 IEEE International Conference on Image Processing (ICIP).
Klopp, Liu, Chen, et al. 2021. “How to Exploit the Transferability of Learned Image Compression to Conventional Codecs.” In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Kumar, Seetharaman, Luebs, et al. 2023. “High-Fidelity Audio Compression with Improved RVQGAN.”
Qiu, Yu, and Li. 2021. “Codec-Simulation Network for Joint Optimization of Video Coding With Pre- and Post-Processing.” IEEE Open Journal of Circuits and Systems.
Shin, and Song. 2017. “JPEG-Resistant Adversarial Images.” In NIPS 2017 Workshop on Machine Learning and Computer Security.
Shwartz-Ziv, and LeCun. 2023. “To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review.”
Toderici, O’Malley, Hwang, et al. 2016. “Variable Rate Image Compression with Recurrent Neural Networks.”
Xu, and Raginsky. 2017. “Information-Theoretic Analysis of Generalization Capability of Learning Algorithms.” In Advances In Neural Information Processing Systems.
Yang, Mandt, and Theis. 2023. “An Introduction to Neural Data Compression.”