Have you ever been fascinated by the seemingly magical ability of artificial intelligence to generate creative, dynamic music? Have you found yourself curious about the mechanisms behind this intriguing technology? In this comprehensive session, we delve deep into the world of AI-powered music creation, unraveling the mystery of how machines can emulate the creativity usually attributed to human musicians.

In this hands-on tutorial, after a brief introduction to the theory of generative AI for audio, we will introduce you to several cutting-edge, open-source tools and pre-trained models for audio generation. Then, we will demonstrate how to harness these tools' power to generate your unique compositions from scratch.

The code shown is in Python, and we will start with a simple example and build on it each time, adding a little bit of complexity from text-conditional generation to melody-conditional generation to audio-continuation and audio-inpainting. Join us as we demystify the process of AI music creation and turn this cutting-edge technology into an accessible reality!

Lesson Plan

- Lesson 1: With Python, we will learn how to use text-conditional generation to generate some music based on a description (also known as a prompt).
- Lesson 2: What if we have a melody we'd like to use? Then, we can provide an audio clip with a melody and use melody-conditional generation.
- Lesson 3: How about we know how the music starts but want ideas of how to continue? That's when audio-continuation would be helpful to take an existing clip and fill in what comes afterward.
- Bonus lesson: Tying it together, take some whistling and a random prompt, and leverage melody-conditional generation and audio-continuation to make a song.

Learning Objectives

- How to create audio generation systems with AI models (and for music generation in particular), and the theory and use cases behind them
- How to use Huggingface Hub and Pytorch checkpoints to download and load a pre-trained model.
- How to leverage AudioCraft open-source models for music generation tasks.