Transformers are relatively new architectures that transformed the Deep Learning Industry. While originally intended for Natural language Processing, it soon became widely used in Computer Vision, Generative Art and Reinforcement Learning. Moreover, Transformers achieved this in the span of few years, vastly outperforming the predecessors (RNNs, LSTMS, GRUs) who were in the spotlight for decades.

The pace of the session will depend on the general familiarity level of the audience with the subject.

we will briefly go through the history of NLP and methods from heuristics to simple Machine Learning. We will explore RNNs, LSTMs & GRUs and compare them to the basic transformer architecture that changed the world. We will then discover what Transformers-based models have managed to achieve theoretically and practically, sharing links with you in the process.

Here are the links for the presentations: NLP DL - Transformers