SDN TIP: ‘Learn about the training pipeline of GPT assistants by Andrej Karpathy’

Learn about the training pipeline of GPT assistants, from tokenization to pretraining, supervised finetuning, and Reinforcement Learning from Human Feedback. Dive deeper into practical techniques and effective mental models including prompting strategies and finetuning.

 

Andrej Karpathy

AI Researcher & Founding Member, OpenAI – OpenAI

Andrej Karpathy is a researcher and a founding team member at OpenAI. From 2017 to 2022, he was the Director of AI at Tesla, leading the computer vision team for Tesla Autopilot. He received his PhD in Computer Science from Stanford, where he developed early multimodal deep neural networks that processed both images and text, for example, for image captioning applications.