Designing an AI Agent for a Real Use Case: A First-Principles Architecture Walkthrough

Designing an AI Agent for a Real Use Case: A First-Principles Architecture Walkthrough

Here is the kind of failure that shows up a month after an agent ships: two customers in identical situations get different support tiers on consecutive days, and a salesperson cannot explain why to either one. The agent was built the obvious way, pick a model, pick a framework, prompt it to decide, and the inconsistency was baked in before anyone wrote a scoring rule. Teams ask "which model should I use" or "should I use LangGraph" before they can answer the questions that actually determine the design: what does the task need from the world, what are the boundaries of the agent's authority, and how will we know it worked. Those questions have nothing to do with which LLM you pick, and everything to do with whether the agent survives its first month in front of real users.

This is a first-principles walkthrough: we design one concrete agent, a customer-tiering agent for a mid-size SaaS that decides which support tier an account gets, and we deliberately resist picking tools before the design demands them. Every design decision here follows from something about the task, because the task is the only thing that should be allowed to drive an agent's architecture.

Starting from the task, not the model#

The first mistake is to ask about tooling. So start by writing down what the task actually requires, as facts, not as a pitch. The tiering agent's job: given a customer account, determine one of three support tiers, basic, priority, premium, based on signals like contract value, usage, active seats, support history, and current churn risk. It must be consistent, a customer in the same state should get the same tier on Tuesday and Wednesday. It must be explainable, a salesperson has to be able to say why a customer did not get a higher tier. It must be safe, no customer whose behavior suggests imminent churn can be silently downgraded.

Read those requirements again, because they are doing real work. Consistency is a quality that a raw language model is terrible at, it will give different tiers to identical inputs on different days just from sampling noise. Explainability means the agent has to reveal its reasoning in a structured form. Safety means there are hard constraints that the agent cannot override. Each of these is a constraint on the design, and together they point away from "let the model free-style a tier label" and toward something more governed. The model is not the primary decision-maker here, and that conclusion is reached entirely from the task, with zero knowledge of which model exists.

What the agent can and cannot do#

Before any architecture, define the boundaries of authority, because an agent without clear boundaries is just a model with a suggestion box, and a suggestion box that can do damage is a liability. For the tiering agent, the scope is deliberately narrow. It can compute and propose a tier. It cannot change a customer's contract. It cannot lock in a tier without an approver for anything that moves a customer up or down by more than one level. It cannot act on a customer whose churn signal is critical without escalating to a human account manager.

Boundaries turn into two concrete things in the design. The first is the tool allowlist, the narrow set of things the agent may actually do. The second is the escalation policy, the conditions under which the agent must stop and hand off to a person. Both are deterministic and structural, they are decided by the design, not by the model. This is the point where a naive design would let the agent be too powerful, and a first-principles design recognizes that power is the risk, not the feature.

The data the task needs#

The next step is to enumerate the data sources the task actually needs, and this is another place where teams jump ahead to retrieval or vector databases when the task, if analyzed, may not need them at all. The tiering task needs structured, factual signals: contract value, seat count, monthly usage, open support tickets, recent downgrade attempts, prior tier, churn-risk score from a separate model. None of this is prose that you would embed and search. It is mostly structured, queryable data that lives in a database and an internal API.

So the honest design conclusion is that a vector store and semantic retrieval are largely irrelevant here, and reaching for them would be architecture theater. What the agent needs is tool access to the account service and the metrics service, which return JSON, not an embedding pipeline over documents. This is the moment where first principles saves a team from a lot of unnecessary machinery. When the task needs numbers and facts, you plug into the systems that hold numbers and facts; you do not build a retrieval system over text that does not describe the domain.

import httpx

ACCOUNT_SVC = "https://internal/account"
METRICS_SVC = "https://internal/metrics"

def fetch_account(account_id: str) -> dict:
    return httpx.get(f"{ACCOUNT_SVC}/{account_id}", timeout=5).json()

def fetch_usage(account_id: str) -> dict:
    return httpx.get(f"{METRICS_SVC}/{account_id}/monthly", timeout=5).json()

Three to five tools is the realistic budget for this agent, and that is a feature, not a limitation. Each tool is a real boundary with a real implementation, and a narrow toolset is both easier to secure and easier for the model to use correctly. If your agent needs thirty tools, that is not ambition, it is evidence that you have not scoped the task.

From requirements to a deterministic core#

Now the most important design decision, and it is one that separates senior work from the naive attempt. The task demands consistency and explainability, and a pure "ask the model for a tier and a reason" design fails both, because the model is not consistent and its explanation is unreliable. So the first-principles move is to push the actual scoring out of the model and into deterministic code, and let the model do what it is genuinely good at: reading the factors, summarizing them for a human, and producing the explanation.

The scoring becomes a scored rule set, deterministic and auditable. Model scores of each factor get weighted and thresholded in code, which means identical inputs produce identical tiers, which is the consistency requirement, satisfied by construction rather than by hope.

def breakdown(customer: dict) -> dict:
    return {
        "contract_value": "high" if customer["contract_value"] >= 50000 else "low",
        "usage": "high" if customer["monthly_usage_hours"] >= 200 else "low",
        "seats": "high" if customer["active_seats"] >= 50 else "low",
        "churn_risk": float(customer["churn_risk"]),
        "recent_downgrade": bool(customer["recent_downgrade_requests"]),
    }

def tier_score(customer: dict) -> float:
    score = 0.0
    score += 40.0 if customer["contract_value"] >= 50000 else 10.0
    score += 25.0 if customer["monthly_usage_hours"] >= 200 else 5.0
    score += 20.0 if customer["active_seats"] >= 50 else 5.0
    score -= 15.0 * customer["churn_risk"]      # churn_risk in [0,1]
    score -= 10.0 if customer["recent_downgrade_requests"] else 0.0
    return max(0.0, min(100.0, score))

def propose_tier(customer: dict) -> tuple[str, dict]:
    s = tier_score(customer)
    if s >= 70:
        tier = "premium"
    elif s >= 40:
        tier = "priority"
    else:
        tier = "basic"
    signals = {
        "weighted_score": round(s, 1),
        "factors": breakdown(customer),
        "critical_churn": customer["churn_risk"] >= 0.8,
    }
    return tier, signals

Notice what the model is no longer responsible for. It does not decide the tier, so it cannot be inconsistent about it, and it cannot be talked into the wrong tier by a prompt. The deterministic core enforces both consistency and the basic safety floor, because no amount of prompt engineering against the model can change a number computed in code. This is the healthy inversion: the model becomes the explainer and the communicator, and the deterministic core becomes the decider. For a task with a definable scoring function, that is dramatically more reliable than letting the model be both judge and courtroom.

What the model is actually for#

With the core determined, the model's real job becomes clear and it is a good job. Given the account facts and the computed score, the model drafts the tier decision summary for the salesperson, which is explainability, and it formats the recommendation consistently with a template. It does not reason about arithmetic, it reasons about how to present the arithmetic. This is the right use of an LLM: taking structured, verified input and turning it into clear human-facing language, while all the risky inference has been removed to deterministic code.

import json

from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", (
        "You summarize an account tiering decision for a salesperson. "
        "Use ONLY the facts provided. Explain which factors drove the score. "
        "Do not invent numbers. State plainly if the score is near a threshold."
    )),
    ("human", "Account facts:\n{facts}\n\nScore: {score}\nTier: {tier}\n\nSummary:"),
])

chain = prompt | ChatOpenAI(model="gpt-4o") | StrOutputParser()

tier, signals = propose_tier(customer)          # customer is the account dict
summary = chain.invoke({
    "facts": json.dumps(signals["factors"], indent=2),
    "score": signals["weighted_score"],
    "tier": tier,
})

There is an important fail-closed design here that the naive model-as-decider misses. If the model is asked to produce the tier and the reasoning, and it also has to be prevented from inventing facts, you need grounding and citation checks. If instead the tier and score are already computed deterministically and handed to the model as facts, the model has nothing to invent; it can only paraphrase what it was given. The architecture has removed the hallucination vector for the most sensitive part of the output by construction. That is worth more than any prompt or verify layer bolted on later, because you cannot hallucinate a number that was never in your control to produce.

Guarding the boundaries#

Even with the decision in deterministic code, the agent parts of the system, which tools run, which approvals are needed, still need guardrails, and this is where the earlier boundary work gets enforced. The agent may call the lookup tools freely, but if proposed_tier differs from current_tier by more than one level, the system must not finalize anything without human approval. If the churn risk is critical, it must escalate rather than propose at all. These are deterministic gates in the flow, not wishes in the prompt.

from dataclasses import dataclass

LEVELS = ["basic", "priority", "premium"]

@dataclass
class Decision:
    tier: str
    needs_review: bool
    auto_apply: bool

def gate(account, proposed_tier: str, signals: dict) -> Decision:
    jump = abs(LEVELS.index(account["tier"]) - LEVELS.index(proposed_tier))
    if signals["critical_churn"]:
        return Decision(proposed_tier, needs_review=True, auto_apply=False)
    if jump > 1:
        return Decision(proposed_tier, needs_review=True, auto_apply=False)
    if jump == 1:
        return Decision(proposed_tier, needs_review=False, auto_apply=True)
    return Decision(proposed_tier, needs_review=False, auto_apply=True)

This gate is the safety layer for this specific tiering decision. Because tier changes are consequential, the gate defines which moves are routine enough to finalize automatically: a one-level move on a healthy account. Anything more disruptive, an upgrade or downgrade of two or more levels, or a critical-churn signal, is routed to a person before it can touch the account. The guardrail is concrete and specific to this domain, a threshold on tier distance applied to the account record, which is exactly the kind of hard rule that should live in deterministic code rather than in a prompt. The agent proposes; the gate decides whether the proposal is safe to execute; only well-defined safe moves happen autonomously.

How we will know it is working#

The last thing the first-principles process forces, before any code ships, is a definition of success that the agent can be measured against, because you cannot know if the design is good without knowing what good means. For the tiering agent, success is not "the model generated text." It is measurable behavior: is the tier for a given account state consistent across repeated runs, is the recommended tier accurate against a labeled ground-truth set of accounts where a human already decided the correct tier, is the explanation faithful to the numbers, and is the human-approval rate reasonable, too many auto-applies means the gate is too loose, and too many escalations means the gate is so strict the agent is useless.

A labeled eval set is the anchor. You gather accounts where the correct tier is known, run the agent, and measure how often it proposes the right tier. Because the scoring is deterministic, that accuracy is really a measure of whether your scoring weights are right, not whether the model is right, which is exactly the right thing to be measuring at the design stage. It tells you immediately whether your first-principles weight choices match how your company actually evaluates accounts, and it lets you tune the red line parameters with evidence instead of vibes.

The position to leave with#

Designing an agent from first principles means refusing to let the tooling or the model dictate the shape, and instead letting the task dictate it. That discipline produced a specific and somewhat counterintuitive architecture: an agent that is mostly deterministic code, with a narrow toolset, a hard authority boundary, and a model relegated to explaining already-computed facts. It is boring compared to the demos, and that is exactly the point. It is reliable, explainable, safe, and measurable in ways that a free-wheeling model-as-decider simply cannot match.

The ugliest failures in agent engineering come from reversing the order, picking the model and the framework first and then contorting the task to fit. The strongest agents come from the opposite direction: understand the requirements, define the boundaries, query the data the task needs, push the decidable parts into deterministic code, guard the boundaries, and measure it. None of those steps mention a specific model, because the model is the last and smallest decision. When you have done the design right, picking the model is nearly trivial, and the agent works because the architecture, not the language model, is carrying most of the weight.