easyAI Engineering

How do you choose the right model for a task?

Q: How do you choose the right model for a task?

**TL;DR.** Score the task on **capability needed, latency budget, cost at volume, context length, and features** (vision, tools, structured output) — then pick the **cheapest model that passes your evals**, and **route**: small models for classify/extract/simple chat, frontier for deep reasoning and agentic work. task router rules / classifier <line class='d-edg

110 views

Understand the problem

Capability, cost, latency, context and modality — a routing mindset instead of one-model-fits-all.

model-selectionroutingcostcapability

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Stuck? AI Nudge Available

Get a conceptual hint to guide your logic without spoiling the final implementation.

Study the solution

The solution is waiting

Give it an honest attempt first — then compare your thinking with the full walkthrough.

Read the code

Tiered routing with escalation

async function classify(ticket: string) {
  const fast = await small.complete({
    prompt: classifyPrompt(ticket), maxTokens: 10, temperature: 0,
  });
  const parsed = Label.safeParse(fast.text.trim());
  if (parsed.success && fast.confidenceHint !== "low") return parsed.data;

  // escalation path: the frontier model sees only the hard residue (~5-10%)
  const careful = await frontier.complete({
    prompt: classifyPromptDetailed(ticket), maxTokens: 10, temperature: 0,
  });
  return Label.parse(careful.text.trim());
}

Join the discussion

Discussion (0)

No responses yet. Be the first to share what you think.

What is a context window and what happens when you exceed it?

When should you use batch APIs instead of real-time LLM calls?