Researchers develop real-time lyric generation technology to inspire songwriting

Newswise — Music artists can find inspiration and new creative directions for their songwriting thanks to technology developed by Waterloo researchers.

lyric jam, a real-time system that uses artificial intelligence (AI) to generate lyric lines for live instrumental music, was created by members of the The university’s natural language processing laboratory.

The lab, led by Olga Vechtomova, a co-appointed Waterloo engineering professor in computer science, has been researching creative applications of AI for several years.

Initial work in the lab led to the creation of a system that learns artists’ musical expressions and generates lyrics in their style.

Recently, Vechtomova, along with Waterloo graduate students Gaurav Sahu and Dhruv Kumar, developed technology that draws on various aspects of music such as chord progressions, tempo and instrumentation to synthesize lyrics that reflect the ambiance and emotions expressed by live music.

When a musician or band plays instrumental music, the system continuously receives the raw audio clips, which the neural network processes to generate new lines of lyrics. Artists can then use the lines to compose their own song lyrics.

“The purpose of the system is not to write a song for the artist,” says Vechtomova. “Instead, we want to help artists realize their own creativity. The system generates poetic lines with new metaphors and expressions, potentially leading artists in creative directions they haven’t explored before.

The neural network the researchers designed learns which lyrical themes, words and stylistic devices are associated with the different aspects of the music captured in each audio clip.

For example, researchers have observed that the lyrics generated for ambient music are very different from those for upbeat music.

The research team conducted a user study, inviting musicians to play live instruments while using the system.

“An unexpected finding was that participants felt encouraged by the generated lines to improvise,” Vechtomova said. “For example, the lines inspired artists to structure chords a little differently and steer their improvisation in a new direction than originally intended. Some musicians also used the lines to test whether their improvisation had the desired emotional effect.

Another finding of the study highlighted the co-creative aspect of the experience. Participants said they saw the system as a nonjudgmental jamming partner and felt encouraged to play their musical instruments even if they weren’t actively trying to write lyrics.

Since LyricJam went live in June this year, more than 1,500 users worldwide have tried it.

The team’s research, which will be presented at the International Conference on Computations Creativity in September, was pre-published on arXiv. Musicians interested in trying LyricJam can access it at

Comments are closed.