Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training.
The only prerequisite is being comfortable with learning through code & excel examples.
1 comment
[ 3.6 ms ] story [ 13.7 ms ] threadThe only prerequisite is being comfortable with learning through code & excel examples.
1. Sampling Large Language Models https://go.justinangel.ai/video-1
2. Reverse Engineering Large Language Model https://go.justinangel.ai/video-2
3. Perceptrons: wx+b https://go.justinangel.ai/video-3
4. Activation Functions: ReLU, GELU, SwiGLU https://go.justinangel.ai/video-4
5. GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton https://go.justinangel.ai/video-5
6. MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks https://go.justinangel.ai/video-6
7. Loss Functions: Residual errors, RMSE, Cross Entropy, Loss Landscapes https://go.justinangel.ai/video-7
8. Backpropagation: Training loops, Optimizers, Learning Rate, Batch Size https://go.justinangel.ai/video-8
9. Saving & Loading Models https://go.justinangel.ai/video-9
10. Initialization: Kaiming, Glorot https://go.justinangel.ai/video-10
11. Residuals: Addition, Scaling, Gated, Concatenation https://go.justinangel.ai/video-11
12. Normalization: Pre-norm vs. Post-norm, RMSNorm, BatchNorm, LayerNorm https://go.justinangel.ai/video-12
13. Regularization: Dropout, Gradient Clipping, Weight Decay https://go.justinangel.ai/video-13
14. SoftMax https://go.justinangel.ai/video-14
15. Tokenizers: By Character, By Word, BPE, SentencePiece https://go.justinangel.ai/video-15
16. Embeddings: Absolute vs. Learned, Sinusoidal vs. RoPE https://go.justinangel.ai/video-16
17. Attention: MHA, GQA, MQA, MLA https://go.justinangel.ai/video-17
18. Transformers https://go.justinangel.ai/video-18
19. Pre-training: Data Sources, Datasets, HTML Cleaning, Quality Filtering, Sharding https://go.justinangel.ai/video-19
20. Evaluation: Leaderboards, Benchmarks, Verifiers vs LLM-as-Judge https://go.justinangel.ai/video-20
21. Instruction Tuning: Alpaca & Other Formats, ...