Choosing an AI Approach for a Customer

After this lesson, you will be able to describe a customer's task, compare a fixed workflow with a model-based option, and explain why a particular approach fits the evidence and constraints. You will also have a decision record for comparing model providers. NorthPort Freight is fictional. Its dispatch examples and all records in them are invented.

A request for an "AI agent" does not tell you whether an agent is needed. It may be a label for a fixed sequence of steps, a text classification task, a search feature, or a system that chooses tools and takes actions. Start with the task the user needs to finish, then choose an approach whose behavior can be tested and controlled.

Describe the task before naming the technology

Write down the user's input, the result they need, the decisions involved, and what happens when the system is uncertain. Include what must stay under human control. A task statement for NorthPort might read: "When an order has two conflicting references, show the dispatcher the approved source values and relevant guidance, then let the dispatcher decide whether to hold or release it." This describes a task and a boundary. It does not imply that any AI system should make the release decision.

Look at how much the steps vary. If the same inputs lead to the same known steps, ordinary application logic may be enough. If a model must interpret language or classify an ambiguous note, a model call may help with that part. If it needs current information from approved documents, retrieval can provide relevant passages as context. If the system must choose among tools over several steps, an agent may be a candidate. These options can be combined, but every added component creates more behavior to evaluate and operate.

A workflow is a set of steps selected in advance by the team. A workflow can call a model at one step and still retain fixed control flow. An agent is a system that uses a model to choose actions or tools while working toward a goal. That added flexibility can help with tasks whose steps depend on what the system finds, but it also makes outcomes harder to predict. For a short introduction and examples, see workflows and agents in practice.

Do not use "agent" as a synonym for any feature that calls a model. A fixed sequence such as "read an order, classify the mismatch, show the result" is a workflow even if one step uses a language model. If a model decides whether to query an order system, ask a coordinator tool for more information, or return a result, then the system has agent-like control. Name the actual behavior so reviewers can evaluate its permissions and failure modes.

Choose the lightest approach that meets the need

A language model is software trained to predict or produce language and other supported outputs from an input. A model call can be useful for summarizing an operator note, extracting fields from unstructured text, or drafting a response that a person reviews. For a stable rule such as "hold an order when its approved status is blocked," ordinary code is easier to test and explain. A model should not replace a clear rule merely because the project includes AI.

Retrieval-augmented generation, often shortened to RAG, finds relevant information from a source and supplies it to a model when it answers. Consider it when answers depend on customer documents that change and need source references. Retrieval does not guarantee that the model uses the right passage or interprets it correctly. Test retrieval quality, answer behavior, and what the system does when it finds no useful source. The RAG and vector database guide covers the implementation concepts.

Fine-tuning means training a model further on examples so its behavior changes for a particular task. It may be worth evaluating when repeated examples reveal a stable pattern that prompting alone does not handle well. Fine-tuning does not automatically give a model access to changing customer facts. It also adds dataset preparation, training, evaluation, versioning, and privacy questions. For many projects, compare a carefully written prompt and retrieval system first. See prompt success criteria and test cases and comparing prompts with graders before proposing a training job.

Use an agent when the task genuinely requires selecting among actions based on what happens along the way. Keep its tools narrow, require confirmation for consequential actions, and define where it must stop. If an agent can update a dispatch record, the authorization boundary, duplicate-action behavior, audit trail, and recovery path all need to be designed. If read-only retrieval solves the task, the write-capable agent adds risk without clear user value.

A team may begin with a manual workflow or a fixed sequence, learn which cases need model judgment, and add that step later. A complex agent can sometimes be replaced with a smaller flow once its common paths are understood. Keep a path to revise the approach as customer evidence changes.

Treat customer constraints as design inputs

A technically capable model may still be a poor fit if its service terms, deployment region, integration, latency, cost, or lifecycle do not match the customer's needs. Turn each constraint into a question someone can answer. "Must be private" is too vague to evaluate; "prompts and outputs may not be retained after a response, and processing must remain in the approved region" can be checked against a specific contract and service configuration.

Check capability with task-specific evaluation examples. A provider's model card or capability table helps you shortlist options, but a vendor description does not prove performance on the customer's task. Use examples the customer is allowed to share. Remove or synthesize sensitive details when appropriate, and record the dataset source and approval. Measure the behavior that matters, such as whether extracted fields are correct, whether answers cite approved sources, or whether the system refuses when evidence is missing. See choosing and configuring a model for model selection mechanics.

Check the exact provider service and model

Provider means the company or platform serving the model. A provider may expose several products, endpoints, account types, regions, and model versions, each with different terms or features. Compare the exact path the customer would use, not a brand name in isolation.

On 5 November 2026, the official OpenAI API data controls say that default abuse-monitoring logs may include prompts and responses for up to 30 days, with exceptions, and describe approval requirements for Zero Data Retention or Modified Abuse Monitoring. Anthropic's commercial API retention guidance describes deletion of API inputs and outputs within 30 days, with exceptions such as features that retain data longer and legal or usage-policy needs. Its API retention documentation also describes feature and model-specific differences. These descriptions are not interchangeable privacy guarantees. Confirm the customer's account, chosen features, contract, and current configuration before sending data.

Region has a specific meaning in a provider's documentation. It may describe where an inference request is processed, where stored content is kept, or both. Read the scope and exclusions. OpenAI's data controls document limitations on data residency, including system data and some cases involving customer infrastructure or third-party services. Anthropic's pricing and regional endpoint documentation distinguishes global and regional routing, with pricing that can vary by route and model. A region label alone does not tell you where every related log, backup, integration, or support record is handled.

Check model lifecycle as well as current capability. Model IDs can be aliases or fixed snapshots. Providers can deprecate a model or change availability. Anthropic publishes model deprecation notices, and OpenAI's model catalog lists model capabilities and specifications. Record the exact model identifier, provider route, check date, and a migration plan. Re-run the task evaluation before switching versions.

Estimate the whole cost and the cost of change

Token prices are one part of cost for a hosted language model. Depending on the design, usage may also incur fees for tools, search, retrieval, storage, fine-tuning, or high-throughput and regional service options. A longer prompt, a long answer, retries, and repeated agent steps can increase usage. Estimate a representative request and the expected request volume, then include the application's infrastructure and human review. Do not compare providers using a single input-token rate.

Provider pricing changes. The official OpenAI API pricing page and Anthropic pricing page show that prices and charging rules depend on model and feature. Checked on 5 November 2026, those pages are the places to verify a quote; the lesson does not freeze a price table. Recheck before presenting a customer estimate.

Lock-in is the effort or cost of moving away from a provider or product. It can come from proprietary tool APIs, model-specific prompt behavior, hosted conversation state, file storage, embeddings, evaluation formats, or operational knowledge tied to one service. Portability does not require every component to be identical across vendors. It means the customer understands what would need to change and has access to the data and records needed to make that change.

Ask whether the provider offers an export path for stored files and state, whether the customer controls prompts and evaluation data, and how application code selects a model. Keep provider-specific details behind a small interface where that makes sense. Retain test cases and measured results in a format the customer can use independently. A second provider is not automatically a useful fallback: test that it supports the required behavior and terms before promising a switch.

Write the decision down

A decision record gives the customer and delivery team one place to review the task, constraints, evidence, selected approach, and unresolved risks. Record alternatives you actually considered. If a constraint rules one out, note the source of that constraint. Avoid saying "best model" without naming the evaluation set and criteria behind the choice.

This record is a working decision, not a permanent commitment. Revisit it when the workflow changes, evaluation evidence exposes a weakness, the provider changes its terms, or actual usage differs from the estimate. If a change affects data access, action permissions, or user review, get the responsible owner to approve it before rollout.

Mistakes beginners make

1. Starting with an agent because the request used that word. Describe the task and the actions required before selecting the architecture.
2. Using a model for a rule that can be expressed directly in code. Keep deterministic decisions explicit and testable.
3. Assuming retrieval guarantees a grounded answer. Test whether the right source is found and whether the response uses it correctly.
4. Treating fine-tuning as a way to keep changing documents current. Retrieval is usually the more direct way to supply changing reference material.
5. Comparing vendors by model name or a single token price. Evaluate the exact API route, data terms, tools, full workload cost, and task performance.
6. Reading a privacy summary without checking exceptions or feature behavior. Confirm the product, model, account settings, contract, and geography the customer will actually use.
7. Promising that a customer can switch providers without disruption. Identify the provider-specific state and behavior, then test a realistic migration path.

Recap and try it yourself

Start from the user's task and constraints. A fixed workflow is often enough when its steps are known. Use a model where language or ambiguity calls for it, retrieval when answers need approved changing information, fine-tuning when examples justify changing stable behavior, and an agent only when choosing actions across steps is necessary. Compare providers on evaluated capability, data handling, region, total cost, lifecycle, and portability. Record the reason, evidence, owner, and date so the choice can be reviewed.

Try it: use NorthPort's fictional mismatch review. Write down the dispatcher input, desired result, and actions the system must never take alone. Compare a fixed workflow, a model-assisted classification step, retrieval over approved operating guidance, and a tool-using agent. State which evidence would make you choose each option. Then complete the decision record with unknowns clearly marked. Do not invent customer terms, evaluation results, or savings.

Sources

Checked on 5 November 2026. Model capabilities, model identifiers, provider terms, prices, and regional options change. The links below are official documentation for the services named; verify the current account configuration and agreement before using customer data.

• OpenAI model catalog, for model capabilities and specifications.
• OpenAI API data controls, for abuse-monitoring retention, approved controls, and data-residency limitations.
• OpenAI API pricing, for model and feature-specific billing.
• Anthropic model overview and model deprecations, for capability comparisons, model identifiers, and lifecycle changes.
• Anthropic API retention documentation and commercial data-retention policy, for product, feature, and account-specific data handling.
• Anthropic pricing and regional endpoint documentation, for model and feature pricing, routing, and data-residency price differences.