Using Random Codebooks for Audio Neural AutoEncoders

Welcome to this demo website, in which we display examples of reconstructions we obtained thanks to the model introduced in the paper “Using Random Codebooks for Audio Neural AutoEncoders”, that we will present at EUSIPCO, in Lyon, in August 2024. These excerpts are displayed in Audio Examples. The code of the paper is available on GitHub.

Abstract

Latent representation learning has been an active field of study for decades in numerous applications. Inspired among others by the tokenization from Natural Language Processing and motivated by the research of a simple data representation, recent works have introduced a quantization step into the feature extraction. In this work, we propose a novel strategy to build the neural discrete representation by means of random codebooks. These codebooks are obtained by randomly sampling a large, predefined fixed codebook. We experimentally show the merits and potential of our approach in a task of audio compression and reconstruction.

Index Terms - feature extraction, quantization, random codebooks, audio reconstruction

Code

The implementation of our model is available at github.com/RandRVQ/RandRVQ. It is based on the code of the Descript Audio Codec (DAC), and contains the model definition, the training code, the configuration files of all the models of the paper, and the evaluation scripts needed to reproduce our results.