Musical artists can find inspiration and new creative directions for their songwriting using technology developed by Waterloo researchers.
LyricJam, a real-time system that uses artificial intelligence (AI) to generate lyric lines for live instrumental music, was created by members of the University’s Natural Language Processing Lab.
The lab, headed by Olga Vechtomova, an engineering professor at Waterloo jointly appointed in computer science, has been researching creative applications of AI for several years.
The lab’s initial work led to the creation of a system that learns artists’ musical expressions and generates lyrics in their style.
Recently, Vechtomova, along with Waterloo graduate students Gaurav Sahu and Dhruv Kumar, developed technology that relies on various aspects of music such as chord progressions, tempo, and instrumentation to synthesize lyrics reflecting l mood and emotions expressed by live music.
When a musician or group plays instrumental music, the system continuously receives the raw audio clips, which the neural network processes to generate new lines of lyrics. Artists can then use the lines to compose their own song lyrics.
“The purpose of the system is not to write a song for the artist,” explains Vechtomova. “Instead, we want to help artists realize their own creativity. The system generates poetic lines with new metaphors and expressions, potentially leading artists in creative directions they have never explored before.”
The neural network designed by the researchers learns which lyrical themes, words and stylistic devices are associated with the different aspects of the music captured in each audio clip.
For example, the researchers observed that the lyrics generated for ambient music are very different from those for upbeat music.
The research team conducted a user survey, inviting musicians to play instruments live while using the system.
“An unexpected finding is that the participants felt encouraged by the lines generated to improvise,” Vechtomova said. “For example, the verses inspired artists to structure the chords a little differently and take their improvisation in a new direction than originally intended. Some musicians also used the verses to check if their improvisation was having the desired emotional effect. “
Another result of the study highlighted the co-creative aspect of the experience. Participants said they viewed the system as a non-critical jamming partner and felt encouraged to play their musical instruments even if they were not actively trying to write lyrics.
Since LyricJam went live in June of this year, more than 1,500 users worldwide have tried it.
The team’s research, which will be presented at the International Conference on Creativity in Computing in September, has been pre-published on arXiv. Musicians interested in trying LyricJam can access it at https://lyricjam.ai.
Source of the story:
Material provided by University of Waterloo. Note: Content can be changed for style and length.