The four interfaces: chat, structured, embeddings, tools

Chat is the right UI when the user is exploring, the task is open, and they can judge the answer (draft an email, poke a knowledge base). Chat is the wrong UI when the output must land in SAP, when there is one button ('classify this'), or when the user is a night-shift operator who needs three fields, not a conversation. If you ship chat into those jobs, you will spend the next quarter teaching people to prompt, and they will hate you. They did not ask for a hobby. They asked to file an incident.

If you must ship chat, still structure the back: retrieve, then generate with citations, then optional tools. Never 'just a prompt box on the intranet.' A prompt box on the intranet is a consumer product with your logo and your liability. Permissions will be wrong. Logs will be wrong. People will paste payroll. The UI can still be a thread. The architecture cannot be a thread. Architecture is retrieve / extract / tool / human. UI is how you hold it.

Chat has a hidden cost you met yesterday: history. Every turn resends the novel. Champions become the most expensive users. Chat also has a hidden quality problem: the user can pull the model off the spec (instruction neglect, sycophancy). A form cannot say 'paint us green.' A chat box can. If the task is a form, ship a form. You can still use a model behind the form. That is structured output, not a chatbot. Meeting language: 'We can put a thread on it later if people need to explore. v1 is a form because the output is fields. A thread on a form is a training program we do not have time to run.' Then draw the form. Stop talking.

When a sponsor says 'like ChatGPT but on our data,' they are asking for a UI they recognize. Translate: search plus cited answer, permissions from the source system, a refuse-when-unknown, an eval on groundedness. Maybe the UI is chat. Maybe it is a search box with an answer panel. The second one is often better on a floor. Pick the UI after you pick the interface. Picking the UI first is how you get a chatbot of everything. Write the translation on the intake ticket before you schedule the workshop. If you accept their noun, you have accepted their architecture. You are the person who is not supposed to do that.

Multimodal does not change this. Image and speech are extra doors into the same four interfaces. A photo of a damaged shipment can become structured fields (extract). A voice note can become a ticket (speech to text, then extract or a tool). A chat that accepts images is still a UI. Do not start a 'multimodal program.' Start with the input you actually have and the system of record it has to land in. Extra doors, not a different religion.

Meeting language: 'Chat is one UI. Tell me whether the output is a paragraph a human will edit, fields in a system of record, a ranked list of existing documents, or an action in a system. Those are four interfaces. We will pick one. We will not pick 'chatbot' as the architecture.' Then draw the matrix. Then stop. If they still want a chatbot of everything, you are in a branding meeting. End it or you will own forty UIs.

JSON / schema / function-call output turns language into rows: owner, due date, risk type, amount. This is how you feed Jira, ServiceNow, a data warehouse. Delivery implication: the schema is a contract with the downstream system. Version it. Eval it field-by-field (did we get the date, not 'was the paragraph nice'). A pretty paragraph that cannot load is a failed interface, even if the steering committee liked the demo. Liked is not loaded. Loaded is the only score that matters once you claimed you would write to the system of record.

Extraction jobs (invoices, complaints, contracts) almost always want structured output plus a confidence / 'needs human' flag — not a chat transcript. The human queue is part of the interface. If you skip it, you will invent owners, dates, and amounts, and you will write them to the system of record. That is faster than a junior analyst and worse. HITL on low confidence is not a lack of faith in the model. It is how you get to ship.

Field-level eval is the adult version of 'it worked.' Precision and recall per field, plus a 'did we write a null when we should have' score. A model that gets the amount right and the currency wrong is a fail on a payments field and a pass on a comments field. Do not average them into a single 'accuracy' that hides the blast radius. You already know not to average severity-1 and spelling. Same here. Put the high-blast fields at the top of the score table: amount, currency, customer ID, due date. Comments can be messy. Money cannot. A 90 percent 'accuracy' that hides a 30 percent miss on amount is a lie with a decimal.

The schema will change. v1 had four fields. v2 adds a code from a controlled list. If the prompt says 'pick a reasonable category' and the system of record says 'one of these twelve enums,' you will spend a month mapping free text. Put the enum in the schema. Reject invalid values in code. Retry once. Then human. Do not let the model invent enum 13. Enum 13 is how reporting dies. Own the contract the way you own an interface spec with a vendor. Unsigned enums are unsigned interfaces. You already refuse those.

Structured output is also how you kill a chatbot request. 'Chatbot that fills the incident form from an email' is extraction plus a form preview plus a human submit. The user never needs a thread. They need a pre-filled form they can correct. That product is better, cheaper, and safer. If you call it a chatbot to get the funding, you will be asked why it does not also chit-chat. Name it extraction. Fund extraction. If the sponsor needs the word chatbot on the slide to get budget, put it in the subtitle and keep the architecture box honest. Subtitles do not ship. Architecture boxes do. Do not let the subtitle write the schema.

Failure modes: valid JSON with invented facts (schema is not grounding). Valid facts in invalid JSON (you skipped structured-output mode). Schema drift vs the downstream (no owner of the contract). No needs_human flag, so garbage writes at 2 a.m. Controls: grounding from the source, vendor schema mode, a versioned contract, a flag, a queue. That list is the delivery design. The model is one line in it. If your design doc is mostly the model name, you do not have a design. You have a shopping list with extra adjectives.

An embedding model turns a chunk of text into a list of numbers such that similar meanings sit nearby. That is semantic search: 'password reset' matches 'cannot log in' without sharing words. This is the retrieve in RAG. It is also duplicate-ticket detection, 'find similar incidents,' and clustering. It is not a conversationalist. Do not chat with an embedding model. Do not search with a chat model. Two products, two evals (retrieval recall vs answer quality). If a vendor demo 'searches' by asking a chat model to list documents, they are completing a plausible bibliography, not retrieving. Ask to see the ranked chunks and the ACL filter. No chunks, no search.

Keyword search is not dead. SKUs, case IDs, error codes, and exact policy numbers want keyword or a filter, not a vibe-neighbor. Hybrid (keyword + embedding) is the grown-up default on a corpus that has both prose and identifiers. If you only embed, you will lose on 'show me INC-10412.' If you only keyword, you will lose on 'cannot log in.' Delivery question: what do users actually type? Look at six months of search logs before you pick a religion.

Eval retrieval separately from generation. Recall@k: of the pages a human says matter, how often are they in the top k chunks. If recall is bad, do not tune the generator. Fix chunking, metadata, permissions, hybrid search. A fluent answer from the wrong page is a well-spoken miss. Sponsors will call it hallucination. Sometimes it is. Sometimes it is a search miss wearing a fluent coat. Your job is to know which, because the fix is different. A generator tweak will not find a page the index never returned. A retrieve tweak will not stop a model from fusing two real quotes. Name the failure before you name the lever.

Permissions are a retrieval problem. If the index can see a doc the user cannot, you will leak. If the index cannot see a doc the user can, you will miss. The retrieve step must enforce the same ACL as the source, or you have a new data store with worse rules. This is why 'just upload the drive' is not a plan. Week 3 will press governance. Today: embeddings without ACL is a leak with a cosine score. Meeting language: 'We will not index a corpus we cannot filter. If the source ACL cannot be applied at retrieve, that corpus is out of v1.' Then narrow the corpus. Do not 'add permissions later.' Later is the leak.

Chunking is a delivery decision dressed as a technical one. Too big: the needle is diluted and you pay for tokens. Too small: you lose the section that made the sentence true. Start with heading-aware chunks of a few hundred tokens, keep the title and URL as metadata, and eval recall. Do not start with a 40-page chunk because the window can hold it. You already know stuffing fails. Chunking is how you refuse stuffing. Put chunk size and overlap on the design the way you put batch size on a conversion. If nobody can say the chunk size, you do not have retrieval. You have a paste with extra steps and a vector database invoice.

'Chatbot that finds similar past outages' is embedding search, maybe with a small summary on top. The value is the list of incidents, not a paragraph that mashes them together. Show the list, scores, links. If you generate a mash-up, you will fuse two outages into a third that never happened (day 2, fusion). Find first. Generate only if a human asks for a draft. That order is the interface. Operators will trust a ranked list they can open. They will not trust a fluent story that names an outage nobody remembers. Do not write that story.

Tool / function calling: the model emits a structured request ('create_ticket', 'lookup_customer'), your code runs it, the result returns to the model. This is the seed of agents. It is also how you keep the model away from writing SQL by hand if you wrap a safe query tool. The model does not 'have access' in the mystical sense. Your code does, with the credentials you gave it. Those credentials are the blast radius. Treat a write-tool like a service account, because it is one.

Permissions, idempotency, and human approval on write-tools are delivery problems. A read tool is a search problem. A write tool is a change-management and audit problem. Do not let a demo skip that distinction. Demos love 'and then it files the ticket.' Production asks: under whose identity, with what duplicate check, with what audit row, with what approval over $X, with what way to undo. If those answers are 'the bot user' and 'we'll see,' you do not have a tool. You have an incident in staging.

Read vs write, drawn hard. lookup_customer, search_policy, get_ticket: read. create_ticket, send_mail, post_journal, update_CRM: write. Reads can still leak (ACL). Writes can move money, customers, and legal position. HITL on writes is the default. You can relax it later on a narrow, reversible, low-blast action after an eval. You cannot start relaxed because the demo was smooth. Smooth writes are the scary ones. A smooth write with no audit row is an incident that looks like a feature until the first wrong customer is emailed. Keep the gate.

Idempotency: if the model retries, does it create two tickets. If the user double-clicks, does it send two mails. You already fight this in every integration. The tool needs an idempotency key. The model is not a reliable once-only caller. It will retry. It will also call the tool with half-filled arguments. Validate in your code. Do not 'trust the model to call it right.' Your code is the adult. The model is the intern with a structured mouth. Put the key on the RAID row for create_ticket before the first demo wires it. A demo with one happy click will not show the duplicate. The first bad afternoon will. Write the key while it is still boring.

Agent loops are tool use plus a graph. Each hop is cost (day 5) and a chance to call the wrong tool. Cap hops. Restrict the tool list per task. A policy copilot does not get send_mail. A refund assistant does not get a generic 'run SQL' tool. Least privilege is not only for humans. If a builder wants one mega-agent with every tool, they are asking for a service account with every permission. You would not grant that to a contractor. Do not grant it to a loop.

Intake translation: 'We want ChatGPT on our data' usually means search + cited answer (RAG). 'We want it to file tickets' means structured output or a write-tool with approval. 'We want it to know our customers' means a retrieval + CRM tool, not a bigger model. 'We want it to email the customer' is a write-tool you should probably refuse in v1. Translate, then pick. If you accept the sponsor's noun ('chatbot') you have accepted their architecture. You are the person who is not supposed to do that.

Hold the 2×2 in your head. Find vs act on one axis. Human-judged prose vs machine-consumed fields on the other. Chat sits in find + prose. Embeddings sit in find + (usually) a list, not prose. Structured output sits in act-preparing + fields. Tools sit in act + side effects. Real products mix them: retrieve (embeddings), generate a cited paragraph (chat-like), extract a row (structured), maybe a write (tool). Mix on purpose. Do not mix by saying chatbot four times.

Pick in one sentence. 'This is extraction into ServiceNow with a human confirm, not a chat.' 'This is hybrid search with a cited answer panel.' 'This is a write-tool on create_ticket with approval over medium severity.' If you cannot say the sentence, you are not ready to estimate, eval, or staff. The sentence is the architecture. Everything else is a vendor. Practice the sentence on the five intake requests in today's lab until it is boring. Boring sentences ship. Chatbot-of-everything sentences recruit a crowd and then stall.

What each one evals. Chat / RAG answers: groundedness, citation match, task success. Structured: field-level precision/recall, schema validity, needs_human rate. Embeddings: recall@k, ACL miss rate. Tools: argument validity, idempotency, unauthorized-call rate, hop count. If you use one eval for all four, you will optimize a demo. Match the eval to the interface the way you match acceptance criteria to a workstream. A single 'accuracy' number across these four is how a retrieve miss gets called a model problem. It is not. Fix the matching eval.

What each one costs. Chat / RAG: generator tokens, history. Structured: generator tokens, usually shorter output, retries on schema. Embeddings: cheap per token, plus the index job. Tools: generator tokens × hops, plus the downstream system's load, plus the incident you will have if you skip HITL. Cost follows the graph, not the noun. A 'search chatbot' that is really 12 hops is an agent bill. Rename it so the envelope is honest. Honest names are how finance and security join the meeting on the right slide. Cute names are how they join six months late.

What each one does to change management. Chat: you must teach people to check answers. Structured: you must teach people to correct a pre-filled form — they already know forms. Search: you must teach people to use a better search; adoption is the risk. Write-tools: you must teach people (and auditors) who did the thing. The write-tool change is the heaviest. That is why v1 should often stop before the write, even if the demo included it. A demo that writes is a sales move. A v1 that writes is a change program. Do not let the sales move set the v1. Name the human on the write, or do not ship the write.

Kill three chatbot requests this week at work, on paper if not in the room. HR policy: search + cited answer. Incident form from email: extraction + form. Similar outages: embedding search + list. You just cut two UIs, one unbounded history bill, and a write you were not ready to own. That is delivery. The fourth request ('email the customer') waits for a write-tool with approval, which is a later slice, which is a kill criteria you can defend.

A system of record is the authoritative business system: CRM, ERP, ITSM, claims, core banking. If the fact is not there, the business does not officially know it. Chat that cannot write here cleanly stays a side conversation. Side conversations can still be valuable (draft, search, coach). They are not 'AI in the process.' They are AI next to the process. Be honest about which you are shipping. Honesty changes the eval, the owner, and the change plan. A side chat is owned by the team that wanted a draft. A SoR write is owned by the system owner. If the system owner is not in the room, you are scoping a side chat whether you say so or not.

To enter the system of record you need structured output or a write-tool, a schema that matches the system, authz, audit, and usually a human on the first slice. You also need to decide who the actor is: the user (the tool runs as them) or a service account (the tool runs as a bot). Running as the user respects their permissions and makes audit easier. Running as a bot is how you get a god-account. Prefer the user. If you cannot, the bot is a RAID item with a name.

Failure mode: two truths. The chat said the refund was approved. The CRM still says pending. The customer was told. You will spend a week in a war room. The fix is not a better paragraph. The fix is: the only approval is the one that landed in the CRM, and the chat may only say what the tool returned. If the tool did not run, the chat may not claim. That is a spec line (day 4) and an interface choice (today).

A useful v1 is often read-from-SoR plus draft, with the write still human. The copilot looks up the customer (read tool), drafts the case notes (chat/structured), and the human pastes or clicks submit. You get value without owning a write. v2 adds create_ticket as the user, with idempotency. v3 maybe auto-submits below a threshold. That sequence is a program. 'Chatbot that does the process' is not a program. It is a wish that skips v1. Write v1, v2, v3 on the charter with a kill between them. If v1 does not save time on the draft, you do not buy a write. The kill is how you stay a delivery lead instead of a demo owner.

When to refuse the SoR write entirely. The system has no API worth using. The audit team has no pattern for non-human actors. The blast radius is money, safety, or a regulated communication. The team cannot staff a queue. Any one of those is enough to keep v1 in draft-and-copy. You are not a coward. You are the person who has watched a bad integration. AI does not make a bad integration better. It makes it fluent. Fluency is how a bad write reaches a customer before anyone notices. Meeting language: 'We can draft. We cannot write here until there is an API, an actor, and a queue. That is a later slice, not a no forever.'

Draw it for the sponsor: two boxes. Left: chat, ephemeral, user-judged, not authoritative. Right: SoR, fields, audit, authoritative. Arrows from left to right are structured output or tools, with a human gate labeled. If the arrow is missing, you are shipping a side chat. If the arrow is unlabeled, you are shipping an incident. Label the arrow. That is the whole meeting. If they want the arrow unlabeled so the demo looks magical, they are asking you to own the incident. Decline. Magic is not a cutover plan.

A write-tool is a production action with a fluent intern holding the trigger. Name it that way on the RAID log or you will treat it like a feature. Risk: unauthorized write. Trigger: the tool ran as a bot, or the model called it without a human, or the argument validator was skipped. Mitigation: run as the user, HITL above a threshold, validate arguments in your code, audit row per call. Owner: the system-of-record owner, not 'the AI team.' If the SoR owner will not own it, you do not have a write. You have a demo that will become their incident.

Worked RAID row for send_mail. Description: model can send customer email. Probability: high in a demo culture, because someone will wire it to look clever. Impact: commitment we cannot keep, or a confidential paste. Mitigation: no send_mail in v1; v2 sends only to a staging inbox; v3 sends as the user with a confirm click and a template that cannot add new promises. Kill: if we cannot staff the confirm queue, it stays off. That row is a program. 'We'll be careful' is not a row. Careful is not a control. Confirm clicks are.

Worked RAID row for create_ticket. Description: duplicate incidents from retries. Probability: certain, because the model will retry and the user will double-click. Impact: noisy queue, wrong metrics, angry operators. Mitigation: idempotency key on the tool, dedupe window, the model is told 'if you already called create_ticket this turn, stop.' Eval: unauthorized-call rate and duplicate rate on a 50-row set that includes retries. If you skip idempotency because the demo was a single happy click, you will discover duplicates on the first bad afternoon. You already know this from every other integration. Write the key.

Read-tool risk is different and still real. lookup_customer can leak a record the user should not see if the tool ignores ACL. search_policy can retrieve a draft the user is not cleared for. Put ACL miss rate on the eval for retrieve, not only on write. A leak is not a write, but it is still an incident. The RAID item is 'index or tool bypasses source ACL.' Mitigation: the tool runs as the user and the index is filtered the same way. If you cannot do that this quarter, you do not retrieve that corpus this quarter. Narrow the corpus. Do not 'we'll add permissions later.' Later is the leak.

Agent graphs turn several tools into one RAID item with a worse blast radius. A 12-hop agent with send_mail, create_ticket, and run_sql is a service account with three powers and no memory of last week. Cap the tool list per task. Policy copilot gets search_policy only. Refund assistant gets lookup_customer and, later, a write with HITL. Nobody gets run_sql. Least privilege is a delivery design, not an InfoSec lecture. If a builder wants one mega-agent, ask them to draw the service-account equivalent. They will usually recant.

Meeting language: 'Writes are v2 or v3. v1 is read and draft. The RAID log will name the actor, the HITL, the idempotency key, and the kill. If we cannot name those, we are not scoping a write. We are scoping a paragraph that pretends it already wrote.' Then sit down. Sponsors who wanted the chatbot to 'just file it' will either accept v1 or they will find another lead who will file it. Let them. You have watched un-owned writes. You are not auditioning to own the next one without controls.