Q: What are embeddings and what are they used for?

**TL;DR.** An embedding is a **dense vector** representing text (or images/code) such that **similar meaning → nearby points**. Similarity becomes geometry, powering **semantic search, RAG, dedup, clustering, classification and recommendations**.

Q: What belongs in the system prompt versus the user message?

**TL;DR.** **System** = durable, app-controlled behaviour: role, rules, output format, tool policy. **User** = the per-turn task and data. A clean split improves instruction-following, enables **prompt caching**, and is the first layer of **injection defense** — untrusted content never goes in system. <rect class='d-box-accent' x='

Q: What is a vector database and when do you actually need one?

**TL;DR.** A vector DB stores embeddings with **ANN indexes plus filtering, CRUD and scaling**. But under ~100k vectors a flat search is fine, and **pgvector** keeps vectors next to your relational data. Reach for a dedicated engine at **millions of vectors, high QPS, or heavy filtered search**. <rect class='d-box' x='16' y='

Q: What is RAG and why use it instead of fine-tuning for knowledge?

**TL;DR.** **Retrieval-Augmented Generation**: retrieve relevant documents for the query and put them in the prompt so the model answers **from evidence**. For knowledge it beats fine-tuning on every axis that matters: **instantly updateable, auditable via citations, per-tenant scopable, cheaper**. Fine-tune for behaviour, RAG for facts. <svg class='iq-diagram' viewBox='0 0 460 150' role='img' aria-label='RAG pipeline: query is embedded, nearest chunks retrieved from the index and passed with t

Q: What are tokens and why do they matter for cost and limits?

**TL;DR.** Tokens are the **sub-word units** a model reads and writes (~4 English characters each). They are the currency of everything: **pricing** (per input/output token), **context limits**, and **latency** (one forward pass per output token). <text class='d-sub' x='40'

Q: What are evals and why does every LLM feature need them?

**TL;DR.** Evals are **automated tests for model behaviour**: realistic inputs + grading criteria + graders (code or LLM judges) producing scores. They exist because LLMs are **nondeterministic** and changes have **non-local effects** — without evals every prompt tweak and model upgrade is vibes-tested. They are the highest-leverage asset in an AI product. <svg class='iq-diagram' viewBox='0 0 460 150' role='img' aria-label='Eval loop: dataset runs through the system, graders score outputs, scor

Q: What are hallucinations and what causes them?

**TL;DR.** Hallucinations are **fluent but false** outputs — invented citations, nonexistent APIs, confident wrong facts. Root cause: the model optimizes **plausibility, not truth**; when the answer is missing from weights and context, the most likely-sounding continuation may be fabricated. <rect class='d-box' x='16' y='24' wi

Q: What is the difference between zero-shot, one-shot and few-shot prompting?

**TL;DR.** **Zero-shot** = instruction only; **one-/few-shot** = include worked examples so the model infers the pattern (**in-context learning**, no training). Modern chat models are strong zero-shot — add examples when **format or edge-case behaviour** must be nailed, not by default. <rect class='d-box-muted' x='20' y='24' width=

Q: What is a context window and what happens when you exceed it?

**TL;DR.** The context window is the **maximum tokens (input + output)** a model can attend to per request. Exceed it and you get an **API error or truncation**; apps cope by **trimming, summarizing, or retrieving** only what matters. Long contexts also degrade recall (**lost in the middle**) and cost more.

Question 1

What is a Large Language Model and how does it generate text?

Accepted Answer

**TL;DR.** An LLM is a **transformer** trained on massive text to **predict the next token**. Generation is **autoregressive**: it predicts one token, appends it to the input, and repeats. Instruction-following and reasoning emerge from this objective plus **post-training** (instruction tuning, preference training).

Question 2

What are embeddings and what are they used for?

Accepted Answer

**TL;DR.** An embedding is a **dense vector** representing text (or images/code) such that **similar meaning → nearby points**. Similarity becomes geometry, powering **semantic search, RAG, dedup, clustering, classification and recommendations**.

Question 3

What belongs in the system prompt versus the user message?

Accepted Answer

**TL;DR.** **System** = durable, app-controlled behaviour: role, rules, output format, tool policy. **User** = the per-turn task and data. A clean split improves instruction-following, enables **prompt caching**, and is the first layer of **injection defense** — untrusted content never goes in system.

Question 4

What is a vector database and when do you actually need one?

Accepted Answer

**TL;DR.** A vector DB stores embeddings with **ANN indexes plus filtering, CRUD and scaling**. But under ~100k vectors a flat search is fine, and **pgvector** keeps vectors next to your relational data. Reach for a dedicated engine at **millions of vectors, high QPS, or heavy filtered search**.

Question 5

What is RAG and why use it instead of fine-tuning for knowledge?

Accepted Answer

**TL;DR.** **Retrieval-Augmented Generation**: retrieve relevant documents for the query and put them in the prompt so the model answers **from evidence**. For knowledge it beats fine-tuning on every axis that matters: **instantly updateable, auditable via citations, per-tenant scopable, cheaper**. Fine-tune for behaviour, RAG for facts.

Question 6

What are tokens and why do they matter for cost and limits?

Accepted Answer

**TL;DR.** Tokens are the **sub-word units** a model reads and writes (~4 English characters each). They are the currency of everything: **pricing** (per input/output token), **context limits**, and **latency** (one forward pass per output token).

Question 7

What are evals and why does every LLM feature need them?

Accepted Answer

**TL;DR.** Evals are **automated tests for model behaviour**: realistic inputs + grading criteria + graders (code or LLM judges) producing scores. They exist because LLMs are **nondeterministic** and changes have **non-local effects** — without evals every prompt tweak and model upgrade is vibes-tested. They are the highest-leverage asset in an AI product.

Question 8

What are hallucinations and what causes them?

Accepted Answer

**TL;DR.** Hallucinations are **fluent but false** outputs — invented citations, nonexistent APIs, confident wrong facts. Root cause: the model optimizes **plausibility, not truth**; when the answer is missing from weights and context, the most likely-sounding continuation may be fabricated.

Question 9

What is the difference between zero-shot, one-shot and few-shot prompting?

Accepted Answer

**TL;DR.** **Zero-shot** = instruction only; **one-/few-shot** = include worked examples so the model infers the pattern (**in-context learning**, no training). Modern chat models are strong zero-shot — add examples when **format or edge-case behaviour** must be nailed, not by default.

Question 10

What is a context window and what happens when you exceed it?

Accepted Answer

**TL;DR.** The context window is the **maximum tokens (input + output)** a model can attend to per request. Exceed it and you get an **API error or truncation**; apps cope by **trimming, summarizing, or retrieving** only what matters. Long contexts also degrade recall (**lost in the middle**) and cost more.

AI Engineering Interview Library

Question Catalog

What is a Large Language Model and how does it generate text?

What are embeddings and what are they used for?

What belongs in the system prompt versus the user message?

What is a vector database and when do you actually need one?

What is RAG and why use it instead of fine-tuning for knowledge?

What are tokens and why do they matter for cost and limits?

What are evals and why does every LLM feature need them?

What are hallucinations and what causes them?

What is the difference between zero-shot, one-shot and few-shot prompting?

What is a context window and what happens when you exceed it?

How do you choose the right model for a task?

When should you use batch APIs instead of real-time LLM calls?

How do you build resilient LLM calls with rate limits, retries and fallbacks?

How does streaming work in LLM APIs and why does it matter for UX?

What is the difference between a base model, an instruction-tuned model and a chat model?

How do you get reliable structured JSON output from an LLM?

How do you build a golden dataset for evaluating an LLM feature?

What is the Model Context Protocol (MCP) and what problem does it solve?

When do you use code-based graders versus model-based graders?

How do multimodal models handle images, audio and other non-text inputs?

What is the difference between fine-tuning, LoRA and prompt engineering?

How do you estimate and control LLM costs in production?

How does similarity search work — cosine, dot product and Euclidean?

How do you manage conversations that outgrow the context window?

How do temperature, top-p and other sampling parameters affect output?

How do you design human-in-the-loop approval for agent actions?

What chunking strategies exist and how does chunk size affect RAG quality?

What is prompt caching and how does it reduce cost and latency?

What is context engineering and how does it differ from prompt engineering?

What is an AI agent and how does the agent loop work?

What is chain-of-thought prompting and when does it actually help?

How do you choose between open-weight models and proprietary API models?

How do you reduce hallucinations in a production LLM application?

What is reranking and why add it after retrieval?

How do you design good tool definitions for an LLM?

What is LLM-as-judge and what are its known biases?

How do you scope retrieval with metadata filtering and multi-tenancy?

How do you handle PII and sensitive data when calling LLM APIs?

What is hybrid search and why combine BM25 with vector search?

What are guardrails and how do you implement them around a model?

How do you catch regressions when changing a prompt or upgrading a model?

How does function calling (tool use) work under the hood?

What is the ReAct pattern and why did it become the default for agents?

What should you log and trace in an LLM application?

How should an agent handle tool errors, timeouts and retries?

How do you sandbox an agent and enforce least-privilege tool access?

How do you evaluate a RAG pipeline end to end?

What is semantic caching and when is it safe to use?

When should you split work across multiple agents instead of one?

How do you evaluate and debug agent trajectories?

Why are BLEU and ROUGE poor metrics for LLM outputs, and what replaced them?

How do you version, test and safely roll out prompt changes?

What are common RAG failure modes and how do you debug them?

What is prompt injection and how do you defend against it?

How do you validate and repair structured LLM output at runtime?

How do you monitor LLM quality drift in production?

What techniques reduce LLM latency at each stage of a request?

What is quantization and when would you use a quantized model?

What is red-teaming an LLM application and how is it done?

How do agents maintain memory across steps and sessions?

Difficulty Distribution

Tested Topics

Study Guide