LLM Lab
Start by serving an open LLM behind an API, then learn how to make it faster and smaller, what actually happens inside the model, how to train it for the job you need, and finally how to build a small model from scratch. Every unit is written from results measured on one DGX Spark.
- Status
- In progress. The syllabus draft is done and units are being written
- Size
- 8 parts, 57 units, each about one hour of lecture. Part 7 (13 units) is advanced and optional
- Hardware
- NVIDIA DGX Spark (GB10, 128 GB unified memory)
- Models
- One open LLM family in several sizes. Serving, compression and LoRA specialization use a 4B Instruct model; full fine-tuning runs on a ladder of 0.6B, 1.7B and 4B Base models. A newer family with a different architecture is used for comparison, and part 7 builds a 100–150M model we design ourselves
- Language
- The course material is written in Korean
- Access
- llm-lab.danny22.com is open to signed-in readers. Get in touch to request access
What you learn
- Putting a model behind an API: run a model on an inference engine and cover an industry-standard API, keys and rate limits, and baseline measurements of speed and quality.
- Working out memory in advance: calculate the memory taken by weights, the KV cache and training state before loading anything, and find the upper bound on generation speed from memory bandwidth.
- Serving fast and small: try batching, KV cache management, speculative decoding and quantization, and judge how much quality you lose for the speed and memory you gain.
- Opening up the model: implement the tokenizer, attention, transformer blocks and generation yourself and check they match a public model, and see where the earlier speed and memory numbers come from in the code.
- Training it for the job: specialize with LoRA and serve the adapters, reproduce a post-training recipe from a Base model with SFT and several reinforcement learning stages, and judge every stage with metrics.
- Building a small model from scratch: go all the way through data cleaning, tokenizer training, model design, pre-training and post-training with a larger model as teacher, then compare fairly with a public model of the same size.
What a unit contains
- Theory: concepts, diagrams and numbers measured on this machine.
- Notebook: the core idea written out step by step at small scale, ending in a check cell that judges the result.
- Pipeline code: the same work done at real scale with production tools, tested to give the same values as the notebook for the same input.
Parts
The course works down from applications to foundations: serve an open model, make it faster and smaller, open it up, and train it yourself. Each concept comes right before it is used, and implementation comes later. The baseline measured in part 1 is measured again at the end with the model you trained.
| Part | What you learn | Units |
|---|---|---|
| 0 Environment | Measuring what the machine can do, the workflow used throughout, and the essentials of a deep learning framework | 3 |
| 1 API serving | Putting a model behind an API with an inference engine, keys and rate limits, and the speed and quality baselines every later part compares against | 6 |
| 2 Serving optimization | How inference runs (prefill and decode), batching, KV cache management, speculative decoding, load tests and response-time targets | 6 |
| 3 Model compression | How quantization without training works and how to apply it, and the trade between quality lost and memory and speed gained for each format | 5 |
| 4 LLM foundations | Implementing the tokenizer, attention, transformer blocks, generation and evaluation scoring, and checking them against a public model | 5 |
| 5 PEFT | Chat-format training data, an evaluation system, specializing with LoRA, and serving several adapters from one server | 6 |
| 6 Full FT | Reproducing a post-training recipe with SFT and several reinforcement learning stages, preference learning, distillation, and compressing and serving your model to compare it with the official one | 13 |
| 7 Training from scratch | (Advanced, optional) Building a small model end to end: data cleaning, tokenizer training, model design, pre-training, post-training with a teacher, and evaluation suited to small models | 13 |
Parts 5 and 6 are based on research that scales down a post-training recipe published in September 2026 to a single machine.