From RNN to Transformers

Unversity of Sherbrooke, 2500 Boulevard Université, Sherbrooke, J1K 2R1

From RNN to Transformers: Explaining how we started with simple RNNs and got to the current Transformers models.

Oct 6, 2022, 10:00 – 11:30 PM


Key Themes

AICloudCloud Study JamDataMachine Learning

About this event

Transformers are relatively new architectures that transformed the Deep Learning Industry. While originally intended for Natural language Processing, it soon became widely used in Computer Vision, Generative Art and Reinforcement Learning. Moreover, Transformers achieved this in the span of few years, vastly outperforming the predecessors (RNNs, LSTMS, GRUs) who were in the spotlight for decades.

The pace of the session will depend on the general familiarity level of the audience with the subject.

we will briefly go through the history of NLP and methods from heuristics to simple Machine Learning. We will explore RNNs, LSTMs & GRUs and compare them to the basic transformer architecture that changed the world. We will then discover what Transformers-based models have managed to achieve theoretically and practically, sharing links with you in the process.

Here are the links for the presentations: NLP DL - Transformers 



Thursday, October 6, 2022
10:00 PM – 11:30 PM UTC


  • Elyes Manai

    Université Laval

    Google Developer Expert - Machine Learning


  • Nadia Tahiri, PhD

    University of Sherbrooke


Contact Us