Authors
- Carlos Hernandez-Olivan*
- Koichi Saito
- Naoki Murata
- Chieh-Hsin Lai
- Marco A. Martínez-Ramírez
- Wei-Hsiang Liao
- Yuki Mitsufuji*
* External authors
Venue
- ICASSP-2024
Date
- 2024
Share
VRDMG: Vocal Restoration via Diffusion Posterior Sampling with Multiple Guidance
Carlos Hernandez-Olivan*
Koichi Saito
Marco A. Martínez-Ramírez
Wei-Hsiang Liao
* External authors
ICASSP-2024
2024
Abstract
Restoring degraded music signals is essential to enhance audio quality for downstream music manipulation. Recent diffusion-based music restoration methods have demonstrated impressive performance, and among them, diffusion posterior sampling (DPS) stands out given its intrinsic properties, making it versatile across various restoration tasks. In this paper, we identify that there are potential issues which will degrade current DPS-based methods' performance and introduce the way to mitigate the issues inspired by diverse diffusion guidance techniques including the RePaint (RP) strategy and the Pseudoinverse-Guided Diffusion Models (ΠGDM). We demonstrate our methods for the vocal declipping and bandwidth extension tasks under various levels of distortion and cutoff frequency, respectively. In both tasks, our methods outperform the current DPS-based music restoration benchmarks. We refer to \url{this http URL} for examples of the restored audio samples.
Related Publications
Generative adversarial network (GAN)-based vocoders have been intensively studied because they can synthesize high-fidelity audio waveforms faster than real-time. However, it has been reported that most GANs fail to obtain the optimal projection for discriminating between re…
Vector quantization (VQ) is a technique to deterministically learn features with discrete codebook representations. It is commonly performed with a variational autoencoding model, VQ-VAE, which can be further extended to hierarchical structures for making high-fidelity recon…
Generative adversarial network (GAN)-based vocoders have been intensively studied because they can synthesize high-fidelity audio waveforms faster than real-time. However, it has been reported that most GANs fail to obtain the optimal projection for discriminating between re…
JOIN US
Shape the Future of AI with Sony AI
We want to hear from those of you who have a strong desire
to shape the future of AI.
