Deep Learning in Natural Language Processing has been dominated in the past years by recurrent models. But a new type of network architecture has emerged: the Transformer (Google, 2017). Unlike traditional translator models, this one does not rely on heuristics and solely focuses on attention instead of recurrence. This both reduces training time and translation quality.
As Dataiku is a growing startup with over 20 nationalities, we have decided to use this to develop our own translation bot on the company chat.
In this talk, we will explain the theory behind this model, show how to deploy it in production and give a live demo of our chatbot.