AI Integration

The model call is one step in a longer pipeline. The application decides when AI is appropriate and what the user is allowed to do; the backend builds the context, calls the provider with server-side credentials, validates the response, and only then hands a result to the product. The browser never talks to the model provider directly.

A production prompt is closer to an API contract than a chat message: a system instruction that defines the task and its constraints, clearly separated user input, only the context needed, and an explicit output format. User content is delimited and treated as data, never as instructions. Prompts are versioned and tested against real examples like any other code.

Applications need data they can act on — a category to route a ticket, fields to fill a form, a list to display — not a paragraph to parse with regular expressions. Asking for a defined JSON structure, using the provider's structured-output features where they exist, turns a model response into something the rest of the system can validate and use.

Every response is validated against a schema before use: required fields present, types correct, values within an allowed set, confidence within range. An invalid response is retried once with the validation error included, then falls back — to a default, a rule-based path or a person — and the failure is logged so it can be measured.

Models can be confidently wrong. Reliability comes from the system around them: narrowly defined tasks, grounding in provided context, deterministic business rules the AI output can't override, confidence thresholds that send uncertain cases to human review, and timeouts and retries for provider errors.

For anything consequential — money, permissions, legal content — AI suggests and a rule or a person decides.

Output quality depends more on context than on prompt wording. Send what the task needs — the relevant document section, the user's settings, recent history — and nothing else. Excess context costs money, slows responses and can dilute the answer. User-specific context must respect the same permissions as the rest of the application.

RAG grounds answers in your own content. Documents are split into chunks, the chunks are turned into embeddings and stored for similarity search, the most relevant chunks are retrieved for each question, and the model answers using that context.

Most of the engineering is in the retrieval: chunk boundaries that keep meaning intact, metadata for filtering by tenant and permission, and checking that the right passages are found before judging the answer.

Model providers change quickly in quality, price and limits. I keep the provider behind a small internal interface — task in, validated result out — so switching models, or using different providers for different tasks, is a configuration change rather than a rewrite. That doesn't mean abstracting every provider feature; it means the product depends on our contract, not theirs.

Managed services such as AWS Bedrock help by putting several model families behind one AWS API with IAM-based access. The product should still depend on its own interface rather than on any single provider's request format.

AI cost scales with tokens — input context, output length — and with the number of calls. The levers are mostly architectural.

API keys stay on the server. Sensitive data is sent only when the task needs it and the provider's data handling is acceptable. Prompt injection — instructions hidden in user content or documents — is a real risk, so model output never directly triggers privileged actions; it goes through the same authorization and validation as user input. Retrieval and tools only reach what the current user is allowed to see.

The most useful SaaS AI features are usually unglamorous: classifying incoming items, extracting fields from documents, summarising long threads, drafting content for a person to edit, recommending next actions, automating repetitive workflow steps. Each one needs an owner, a success measure, a cost budget and a fallback for when the model is unavailable.

If the logic can be written as rules — validation, calculations, routing on known fields, permissions — deterministic code is cheaper, faster, testable and explainable. AI is the wrong tool when an error is unacceptable and can't be reviewed, when the input space is small and known, or when nobody can say which problem it solves.