What AI Does—and Does Not Do
Build an accurate first mental model of generative AI as pattern-driven software with useful capabilities, observable limitations, and evidence requirements.
By the end
You will be able to
- Distinguish a model, an AI-enabled product, a search or retrieval system, and an agentic workflow.
- Match common AI capabilities to tasks while recognizing where context, tools, or current evidence are required.
- Explain why fluent output can be wrong, stale, biased, incomplete, or inconsistent.
- Scale verification and human review to consequence, volatility, uncertainty, and reversibility.
Separate the model from the product and workflow
A model is a learned computational component that maps inputs to outputs. A language model generates likely token sequences from learned parameters and supplied context. Products can add search, files, memory, tools, policies, user interfaces, and deterministic code around a model. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
A search or retrieval system locates stored material; a model can summarize or reason over that material but does not become the source. An agentic workflow adds state, tools, feedback, and model-directed continuation. Describe the observed system behavior instead of assuming capability from a product or model name. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Use AI for bounded language, pattern, and multimodal work
Generative systems can explain, summarize, transform, classify, extract, compare, brainstorm, translate, draft, write code, analyze supplied data, and work across supported text, image, audio, or video inputs. Performance depends on the model, modality, task, context, tools, language, and evaluation criteria. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
A useful result begins with a concrete outcome, relevant permitted context, constraints, an output contract, and a way to check success. Provider claims and model catalogs change, so evaluate the exact available system on representative work. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
{
"contractVersion": "1.0",
"executionBoundary": "bounded-task",
"inputValidation": {
"maxTokens": 4096,
"allowedModalities": ["text", "json"]
},
"outputSchema": {
"type": "object",
"required": ["status", "evidence", "confidence"],
"properties": {
"status": { "type": "string", "enum": ["verified", "rejected"] },
"evidence": { "type": "array", "items": { "type": "string" } },
"confidence": { "type": "number", "minimum": 0.0, "maximum": 1.0 }
}
}
}Fluency, confidence, and reasoning are not evidence
Generated statements may be fabricated, stale, overly broad, biased, internally inconsistent, or unsupported by supplied material. A model can produce citation-shaped text, code that looks valid, and confident explanations without a verified source or working result. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Models do not automatically know private systems, current events, local policy, or the user's unstated intent. Context windows are finite, tool results can be wrong or untrusted, and repeated runs can differ. Treat output as a candidate that earns trust through evidence. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Choose appropriate tasks and preserve human responsibility
Start with bounded, reversible work where a person can inspect the result. Brainstorming, drafting, restructuring, orientation, and low-risk classification often tolerate uncertainty when labeled appropriately. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Do not delegate accountability merely because a system is capable of producing an answer. Medical, legal, financial, employment, access, security, safety, production, and other consequential decisions require qualified oversight, current evidence, authorization, and recovery controls. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Scale verification to consequence and uncertainty
Ask how harmful an error would be, how quickly the facts change, how uncertain the evidence is, and how reversible the action would be. Increase independent sources, tests, reviewers, and approval as those factors rise. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Verify facts against authoritative current sources, calculations with independent checks, code with executable tests, and actions against real target-system postconditions. Report unresolved uncertainty and stop when the workflow lacks the evidence or authority required to continue. In production environments, engineers must account for token utilization patterns, context window pressure, and deterministic execution boundaries. When system constraints or rate limits are approached, explicit retry strategies with exponential backoff and jitter prevent cascading failures across downstream dependencies.
Practice activity
Map one AI task to capability, limitation, and evidence
- Choose one AI-enabled task and identify the model behavior, product features, retrieval, tools, memory, and agentic steps actually involved.
- Write a bounded outcome, allowed context, output contract, and one explicit out-of-scope behavior.
- List three plausible failure modes and rate consequence, volatility, uncertainty, and reversibility.
- Create a verification and human-review plan, then decide whether to use AI, use a deterministic workflow, narrow the task, or stop.
What to produce
- A system-boundary map distinguishing model, retrieval, product, tools, and workflow behavior.
- A task decision containing failure modes, risk factors, evidence checks, human responsibility, and a use, narrow, redesign, or stop disposition.
Reflect before continuing
Which capability did you initially attribute to the model but later locate in the surrounding product, data, or tool system?
Evidence
Sources and verification
- Artificial Intelligence Risk Management FrameworkNIST · verified 2026-07-27
- Large language models: What's a large language model?Google · verified 2026-07-27
- ModelsOpenAI · verified 2026-07-27
Knowledge check
Make it stick.
Choose the strongest answer for each question. Your attempts become part of your account transcript.