LLM

Deep Dive into LLMs like ChatGPT

A 3.5-hour end-to-end walkthrough of how large language models like ChatGPT are actually built — from tokenization and pretraining through supervised fine-tuning and RLHF. Karpathy explains everything in plain engineer-to-engineer language with no hype.

Andrej Karpathy's most comprehensive public lecture on LLMs. In three and a half hours he traces the complete training pipeline: how raw internet text becomes tokens, how transformers learn to predict the next token at scale, and how base models are then fine-tuned with supervised learning and reinforcement learning from human feedback (RLHF) to become assistants like ChatGPT. Essential viewing for anyone who wants a real mental model of what these systems are doing under the hood.

LLMChatGPTRLHFAndrej KarpathyTransformersPretrainingDeep Learning

Visit Deep Dive into LLMs like ChatGPT