Legacy mirror · noindex · upstream phase 11

LLM Engineering

Put LLMs to work in production.

Provenance: this phase outline and its lesson readings are preserved from ai-engineering-from-scratch by Rohit Ghumare under the MIT License. InterviewsVector does not claim authorship. These archive pages remain available for old links and progress, but are excluded from indexing.

Attributed readings

01Prompt Engineering: Techniques & PatternsMost people write prompts like they are texting a friend. Then they wonder why a 200-billion parameter model gives mediocre answers. Prompt engineering is not about tricks. It i…
02Few-Shot, CoT, Tree-of-ThoughtTelling a model what to do is prompting. Showing it how to think is engineering. The gap between 78% and 91% accuracy on the same model, same task, same data is not a better mod…
03Structured OutputsYour LLM returns a string. Your application needs JSON. That gap has crashed more production systems than any model hallucination. Structured output is the bridge between natura…
04Embeddings & Vector RepresentationsText is discrete. Math is continuous. Every time you ask an LLM to find "similar" documents, compare meanings, or search beyond keywords, you're relying on a bridge between thes…
05Context EngineeringPrompt engineering is a subset. Context engineering is the whole game. A prompt is a string you type. Context is everything that goes into the model's window: system instruction…
06RAG: Retrieval-Augmented GenerationYour LLM knows everything up to its training cutoff. It knows nothing about your company's docs, your codebase, or last week's meeting notes. RAG solves this by retrieving relev…
07Advanced RAG: Chunking, RerankingBasic RAG retrieves the top-k most similar chunks. That works for simple questions. It falls apart for multi-hop reasoning, ambiguous queries, and large corpora. Advanced RAG is…
08Fine-Tuning with LoRA & QLoRAFull fine-tuning a 7B model requires 56GB of VRAM. You don't have that. Neither do most companies. LoRA lets you fine-tune the same model in 6GB by training less than 1% of the …
09Function Calling & Tool UseLLMs cannot do anything. They generate text. That is the entire capability. They cannot check the weather, query a database, send an email, run code, or read a file. Every "AI a…
10Evaluation & TestingYou would never deploy a web app without tests. You would never ship a database migration without a rollback plan. But right now, most teams ship LLM applications by reading 10 …
11Caching, Rate Limiting & CostMost AI startups do not die from bad models. They die from bad unit economics. A single GPT-4o call costs fractions of a cent. Ten thousand users making ten calls per day costs …
12Guardrails & SafetyYour LLM application will be attacked. Not might. Will. The first prompt injection attempt against your production system will come within 48 hours of launch. The question is no…
13Building a Production LLM AppYou have built prompts, embeddings, RAG pipelines, function calling, caching layers, and guardrails. Separately. In isolation. Like practicing guitar scales without ever playing…
14Model Context Protocol (MCP)Every LLM app built before 2025 invented its own tool schema. Then Anthropic shipped MCP, Claude adopted it, OpenAI adopted it, and by 2026 it is the default wire format for con…
15Prompt Caching & Context CachingYour system prompt is 4,000 tokens. Your RAG context is 20,000 tokens. You send both with every request. You also pay for both — every time. Prompt caching lets the provider kee…
16LangGraph: State Machines for AgentsA ReAct loop written by hand is a `while True`. A ReAct loop written in LangGraph is a graph you can checkpoint, interrupt, branch, and time-travel through. The agent hasn't cha…
17Agent Framework TradeoffsEvery framework sells the same demo (research agent builds a report) and hides the same bug (state schema fights with the orchestration layer). Pick the framework whose abstract…