Join us for Gemma Day: Decoding Gemma: How to Build Open Source LLMs from Scratch, an engaging and interactive event hosted by GDG on Campus at COMSATS University Islamabad. This session is designed to demystify the architecture and innovations behind Large Language Models (LLMs) like Gemma, tracing their evolution from the groundbreaking "Attention is All You Need" paper to modern optimizations.

What to Expect?

• In-Depth Exploration: Dive into the core components of LLMs, starting with the Transformer architecture, self-attention, and multi-head attention, and progressing to advanced techniques like Grouped Query Attention (GQA), KV-Caching, and Local-Global Attention.

• Historical Context: Understand the evolution from early word embeddings (e.g., Word2Vec) and sequence models (RNNs, LSTMs) to the revolutionary Transformer model, with a focus on positional encodings like Rotary Position Embedding (RoPE).

• Hands-On Learning: Engage with mathematical concepts (e.g., Scaled Dot-Product Attention), real-world analogies, and optional code snippets to make complex ideas accessible.

• Interactive Discussions: Participate actively, ask questions, and address misconceptions as we work backward from state-of-the-art implementations to foundational principles.

Prerequisites

• Basic understanding of college-level calculus (derivatives, gradients), matrices, vectors, and trigonometry.

• No need for full mastery—just familiarize yourself with key concepts to enrich your experience.

This event is perfect for anyone curious about the inner workings of LLMs, from students to enthusiasts. Come ready to collaborate, learn, and decode the magic of models like Gemma