Composable Conditioning Blocks

This post explains how we steer our model, it's capabilities, and what the future of control at Suno looks like. It's written for a nontechnical audience, with the goal of equipping designers with the mental models needed to make magical interfaces.

Building Blocks

Our model is a Block Causal Transformer, or BCT. Simply put, our model processes sequences of Blocks from left to right. These blocks represent various types of conditioning—lyrics, styles, or audio samples—that guide the final audio output. Conditioning blocks precede the final audio output block, shaping the audio the model generates. These are some types of blocks we support:

• Text blocks: Lyrics and style tags.
• Cover blocks: Audio of a cover song used to predict the original version.
• Artist blocks: Audio from another song by the same artist, shaping the musical persona. This doesn't always preserve vocal identity since artists often collaborate with other vocalists.
• Playlist blocks: Audio from songs within the same playlist, offering loosely themed conditioning. We can have multiple cover, artist, and playlist blocks in a sequence to condition on multiple songs.
• Overpaint blocks: Instrumentals of a song, combined with new vocals.
• Underpaint blocks: Vocal-only tracks, combined with new instrumentals.
• Stem blocks: Similar to overpaint and underpaint but applicable to various instrument tracks. Stem blocks are a superset of over and underpaint blocks.
• Sample blocks: Short sections from stem blocks, used to build around specific audio samples. Sample blocks are basically stem blocks but with an extra random crop.
• Ditto blocks: Compressed, interpolatable embeddings representing songs.
• Past and Future blocks: Audio context surrounding the target section, enabling infilling or section replacement.
• Audio output block: Always last, generating the final audio based on previous conditioning blocks.

Each block is already a feature in its own right, but the fun really begins when you start composing them together.

Composability

We can add as many conditioning blocks as we like to our sequences. This allows some very powerful capabilities.

Want to add a new vocalist to a section? Use past and future blocks to infill, and an artist block to get a new singer. Don't want the instrumentals to change? Stem separate them out and add an overpaint block. Not the right vibe? Drag around a ditto embedding and insert a ditto block.

By thoughtfully combining blocks, many user-requested functionalities become possible without additional features. This composability is unique to our architecture, providing an exponentially growing set of capabilities. Every new block we add stacks with existing blocks.

Our challenge today is exposing the power of these composable building blocks in an intuitive way.

Whats Coming?

We're continually building more types of blocks to condition with. Some simpler ones are painter type blocks for precise control of song attribute over time. Midi and voice persona blocks are in the pipeline.

More blocks will become time aligned so you can control at which times they should apply their conditioning. Later they may be lyrics aligned, so you can specify the chorus to be extra hype.

For us, model training has become as intuitive and playful as building with Lego blocks. Lets pass this on to our users.