Rhyme with AI

The season's upon us, it's that time of year;
Creating Christmas rhymes instills in us great fear!
Luckily, pre-trained neural networks are easy to apply.
With great pride we introduce our new assistant: Rhyme with AI.

NLP's ImageNet moment may have arrived in 2018, but the ecosystem around NLP models really has matured in 2019. Many of these models (BERT, GPT-2, Transformer-XL, DistilBERT, etc.) are easy to use for your use cases. Our service uses BERT to help us with our (Christmas) rhymes.

Language modeling helps state-of-the-art models understand languages before solving tasks like sentiment analysis or translation. Masking, where the model tries to predict a word that is hidden from a sentence, is one of BERT's innovations. We can use it to help us rhyme by rephrasing rhyming as a task to predict missing words.

From the "The Illustrated BERT, ELMo, and co.". Predicting masked tokens is one of BERT's language modelling techniques.

Our problem involves multiple masks: we know the first sentence and the last word of the second sentence. For instance:

Santa delivers gifts by sleigh

  • ... [MASK] [MASK] [MASK] [MASK] [MASK] [MASK] [MASK] [MASK] day

Inspired by BERT has a Mouth, and It Must Speak, we first let BERT fill in the [MASK]'s and then randomly sample new tokens. Some example rhymes from our model:

Santa delivers gifts by sleigh

  • ... and drinks and celebrates his wedding day
  • ... dressed as preacher and nurse they say
  • ... or bicycle if he has to pay

This already looks pretty good, but we need a solution that people can use!

Luckily, creating an end-to-end machine learning solution is fairly simple. The Datamuse API gives back rhyme words, BERT is available via huggingface and creating an app is no sweat with streamlit. Put it all together in a Docker container and a hosted solution is one command away with Google App Engine.

A few days well spent for us will hopefully save you a lot of pain. Check out or code on GitHub or try our solution at Rhyme with AI!

Follow us for more of this
Recent posts
Recent tweets
Stay up to date on the latest insights and best-practices by registering for the GoDataDriven newsletter.
Follow us for more of this