What is MusicFX? We put Google's AI text-to-music generator to the test

Have you ever wished you could create music without any knowledge of musical theory or instruments whatsoever? That’s now possible, thanks to a new Google AI experiment. Dubbed MusicFX (formerly MusicLM), it’s a new kind of generative AI that can create brand-new music. The AI music generator tool is a lot like text-to-image converters where you simply type in a few lines of text to get beautiful art in return. So how well does Google’s new MusicFX work and how can you try it for yourself? Here’s everything you need to know.

What is MusicFX?

MusicFX is a language model that’s capable of generating music based on a text-based description. For example, the prompt “Calming, soft music I can study to” will generate a lo-fi track. Likewise, a prompt along the lines of “Epic orchestral track that builds tension in an action-adventure movie scene” yields music with a distinct sense of urgency that would feel right at home in Terminator or Mission Impossible.

While its current release is limited, MusicFX can do much more than just turn text into music. In an academic paper, Google’s researchers demonstrated that it can generate audio tailored to images and paintings. And in the future, you might even be able to convert a recorded clip of your humming into a full-blown music track.

Google trained MusicFX on over 200,000 hours of music spanning over 5,500 clips similar to how AI image generators like Midjourney are trained on millions of sample photos. These were manually categorized by human experts, helping the machine learning model distinguish jazz from techno and even specific eras (90s pop) and styles (Afro-Cuban dance music). The company has released the labeled audio dataset with captions on Kaggle, allowing other AI developers to develop their own AI tools.

How well does MusicFX work?

I’ll walk you through how to sign up for MusicFX in the next section, but let me first explain what using it is like. In MusicFX’s current state, you can type in a description and hit the generate button. Think of it like Midjourney, but instead of four AI-generated images, you get two 20-second music tracks instead.

MusicLM used to surfaces suggestions below the input box, including some bizarre ones in my case like “Bubbly, optimistic, cyber pizza party music at the underwater arcade”. I hit generate on that one out of curiosity and the result was an upbeat electronic mix. Basically what I’d expect from a “cyber pizza party”, but I’m not sure about the underwater arcade part.

Moving on, I entered the prompt “Spooky, slow music that you’d hear in a haunted forest” and the resulting track fit the description quite well. However, it didn’t take long to see why Google considers MusicFX an experimental project.

Simply entering “rap music” produced a track with shrill, piercing sounds and a synthetic-sounding background vocal track. The latter seemed like MusicFX was reproducing something straight from its training data.

So while MusicFX doesn’t require any musical skill, it’s also not quite a musician. The results may or may not impress you, similar to chatbots before ChatGPT disrupted that landscape. Still, MusicFX represents the world’s first generative AI for music and that’s impressive in its own right. Here are some more facts about it:

• You cannot ask for music in the style of a specific artist. MusicFX also cannot generate vocals reliably at this time.
• In many instances, MusicFX generated music with unintelligible vocals. Even when I specifically asked for instrumental tracks, the AI ignored my request.
• The current iteration of MusicFX only generates 20-second tracks, even though the underlying model is capable of much longer generations.
• You get two audio clips per generation, which I found were always mostly similar to each other. Still, Google lets you vote for your favorite via a trophy button.
• According to Google, MusicFX can reproduce electronic and classical instruments better than other kinds of music. However, it did decently well when I tried to generate Indian folk music and 8-bit soundtracks in the style of old video games.

How to sign up for MusicFX

Here’s a quick guide on how to use MusicFX:

1. Navigate to Google’s AI Test Kitchen homepage, scroll down to the MusicFX section, and click on the Launch button.
2. Click on Sign in with Google.
3. In the next step, sign in using your Google account.
4. Fill out the form that asks for your country of residence, profession, the reason you’d like to use AI Test Kitchen, and whether you’ll participate in voluntary surveys.
5. Once logged in, you should get full access to the MusicFX platform. Simply enter a prompt, hit Generate, and wait for up to a minute to hear the result.

Keep in mind that many of Google’s AI experiments like MusicFX only work in certain regions like the United States for now. At this stage, it’s unclear if Google will commit to developing MusicFX into a final product. One of the biggest hurdles remains copyright infringement. The company’s researchers found that the AI would replicate existing work in around one percent of cases, which could invite lawsuits from music labels and artists.

Moreover, with virtually no major competition in the generative music space, it remains to be seen whether it remains a priority for the search giant. It’s entirely possible that Google prioritizes all of its AI resources on its Bard chatbot and the Gemini language model instead.

FAQs