Training GPT from scratch with nanoGPT

What to expect?

In this talk, we will explore the implementation of nanoGPT, a lightweight version of the highly popular language model, GPT. We will go over the code base developed by Andrej Karpathy and show how it can be run for various text-generation tasks. Additionally, we will highlight how Weights & Biases can be used for experiment tracking to monitor training performance and make informed decisions. Throughout this talk, we will understand the mechanics behind GPT and gain a deeper insight into how it can be used and modified for our specific text generation needs. Join us if you would like to have: - a clear understanding of GPT - knowledge of how to run nanoGPT for your own purposes - the ability to track your model training using Weights and Biases This presentation is suitable for individuals who have experience with natural language processing and are interested in exploring more advanced text generation models.