AI Solutions Architect

I start from the decision the client wants to change rather than the technology. What happens today, who does it, how long it takes, what it costs per unit, and what an error costs, because if nobody can tell me the price of a wrong answer we are not ready to design anything. Then I look for the cheaper explanation. Is this really a search problem, a reporting problem, a process problem, or a data quality problem wearing an AI costume? A surprising share of these requests turn out to be an unmaintained knowledge base or four teams holding four definitions of a customer. If the task is high volume, tolerant of a bounded error rate, and has a human positioned to catch the residual mistakes, it is a good candidate. If it is low volume, high stakes, and expressible as rules, a deterministic system beats a model on cost and on defensibility. I will say that even when the client was hoping for a larger programme, because dropping an AI system onto a broken process just buys an expensive way to be wrong faster.

Three questions, in this order. First, is the capability a differentiator or a commodity for this business? If a competitor could buy the same thing tomorrow and it would not matter, building it is usually vanity. Second, what does building really mean across five years? Not the prototype, but the evaluation harness, the on-call rotation, the model refresh cycle, and the two engineers who have to exist permanently. Clients price the build carefully and forget the run entirely. Third, what is the cost of being wrong? A vendor behind a clean interface is a reversible decision; a vendor whose data model has spread through the enterprise is not. My default recommendation is to buy the platform and build the parts that encode the client's own domain knowledge and data. I push back on buying when the vendor's roadmap owns something the business depends on, or when residency and auditability rules mean the vendor cannot be used without exceptions that hollow out the value. The client should end up with a decision they can defend to a board, with run cost stated as plainly as build cost.

I write it in layers so different readers can stop at different depths. Page one is the decision summary: what we are building, what it costs to run at expected volume, what we are deliberately not doing, and the three risks that could sink it. An executive should be able to stop there and still argue the position in a steering meeting. Then context, meaning the business process, the systems involved, the data and where it is permitted to live. Then the design: component boundaries, integration points with named existing systems, and sequence diagrams for the two or three flows that matter, because a box diagram without a flow hides every interesting problem. Then the section I care most about, a decision log recording each significant choice, the options considered, why we picked one, and the assumption that would invalidate it. That is what makes the document useful a year later when somebody asks why this vendor. I keep non-functional requirements concrete, with a latency budget, throughput, retention, and an availability target, since vague ones get negotiated away quietly during delivery.

I stop asking what they want and go watch the work. Week one is observation and interviews with the people who actually perform the task, not the sponsors who described it. I want to see the spreadsheet, the copy-paste, the second system somebody keeps open in another window, and the exceptions they handle that appear in no documentation. Then I write down the top ten cases with real examples, current handling time, and error cost. Week two I bring back three options at different levels of ambition, each with a cost envelope and a rough timeline, and let them react. People who cannot specify requirements can almost always criticise a concrete proposal, and their criticism is the requirement. I also insist that one thing gets decided early: what good enough looks like, expressed as a number over examples everyone has seen. Without it, sign-off becomes a matter of taste and the project never ends. The risk in this approach is that observation surfaces a process problem the sponsor did not want raised, and I would rather have that argument in week two than at delivery.

For most mid-size enterprises I start hosted and treat self-hosting as an option we keep open rather than a starting position, because a hosted frontier model removes serving, scaling, and upgrade work at the stage where the real risk is that nobody uses the product. Self-hosting wins on three specific conditions and I check them explicitly. One, data or regulation genuinely prohibits traffic leaving a boundary and no deployment option from the vendor satisfies it. Two, volume is high and stable enough that amortised accelerator cost beats per-token pricing; that crossover is real but arrives later than clients expect, and it moves against you every time API prices fall. Three, the workload is narrow enough that a small tuned model matches a frontier model on that task, which is common for classification and extraction and rare for open-ended work. Either way I insist on an abstraction at the boundary, a request contract we own with evaluation that runs against any provider, so the decision stays revisitable. Sizing the accelerator fleet and its idle cost is the part clients consistently underestimate.

Bottom up from a usage model, with the client owning the volume assumptions since those are the inputs I cannot verify. Requests per user per day, active users at each rollout phase, tokens in and out per request including retrieval context, which is the term everyone forgets and which usually dominates. Multiply by unit price across a couple of model tiers. Then add the parts that are not the model: vector storage and its rebuild cost, embedding generation for the initial corpus and the incremental churn, logging and evaluation traffic, and the non-production environments, which quietly run at roughly a third of production. Then apply the multipliers experience insists on, meaning retries, agentic loops that call the model several times per user action, and caching that performs better in the demo than in production. I present a range with the assumptions visible and state the sensitivities, since halving context length or moving a classification step to a small model usually changes the answer more than any vendor negotiation. Finally I write down kill criteria: the cost per successful outcome above which the business case fails.

Regional stacks with a thin global control plane, rather than one index with access filters. Each region holds its own document store, its own embeddings, and its own index, and the embedding job runs inside that region. People forget that an embedding derived from regulated text is often treated as the text for residency purposes, and I want a written answer from the client's legal team on that before anything is built. Queries route by the user's jurisdiction, and a request never fans out across a boundary unless a document class is explicitly marked shareable. Global by design: the schema, the prompts, the evaluation suite, the deployment configuration, and aggregate telemetry with content stripped out. The cost is duplication, several indexes to maintain and several endpoints to hold at the same version, so I invest early in one deployment pipeline that treats regions as targets. The honest limitation is cross-border questions. A user in one region genuinely cannot see the other's corpus, so I surface that in the product as a stated scope rather than letting it look like the system simply failed to find an answer.

For sharing: one evaluation harness, one set of guardrails to maintain, a single vendor negotiation with real bargaining power, and a common integration pattern so the seventh unit takes weeks rather than a quarter. Cost per unit drops materially because most of the spend is fixed platform work rather than inference. Against it: business units differ in data sensitivity, latency tolerance, and language mix, and averaging across them produces something mediocre for everybody. Shared infrastructure also creates shared incidents and shared change windows, so one unit's regression blocks the other five, and a unit with a regulator watching wants control over upgrade timing that a shared platform struggles to offer. My usual answer is to share the platform but not the behaviour: common serving, retrieval, evaluation, and observability, with per-unit prompts, per-unit retrieval scope, and the ability to pin a unit to a specific model version. That preserves the cost argument while letting the regulated unit move at its own pace. Governance matters more than the architecture here. Somebody must own the shared layer and be funded for it, or it becomes whichever unit shouted loudest last quarter.

I assume the core cannot change and design around it, which usually means reading from and writing to whatever it already exposes. In order of preference: an existing API, change data capture off its database, a batch export, and last, screen-level automation, which I treat as a temporary bridge with an expiry date because it breaks on every vendor upgrade. The AI capability lives outside as a separate service with its own storage, and the integration surface stays deliberately narrow, a handful of well-defined operations rather than deep coupling, since the entire point is that the legacy release cycle must not become ours. Where the legacy system holds the record of truth it keeps that role, and our data is derived, so any disagreement resolves in its favour automatically. Two constraints usually decide the shape. Staleness tolerance, because a nightly extract makes some products impossible. And whether writes are permitted at all, since many of these systems have no safe write path, which means the honest design puts a human between the model's suggestion and the record it would change.

I sequence for evidence and then negotiate the politics. The first unit should be the one where the process is well understood, the data is usable today, the sponsor is engaged, and the cost of a mistake is recoverable. Deliberately not the largest opportunity, because the first delivery's job is to produce a credible number and a working platform. The second should stress a different dimension on purpose, another language, a stricter regulatory posture, a different source system, so the platform gets generalised while the team is still funded to do it. Only then the big unit. Between phases I insist on a real gate with criteria agreed in advance, otherwise phases become a queue and nobody ever stops to repay the platform debt from phase one. I also plan the people side per phase: training, a support path, and somebody inside the unit who owns the thing after we leave. The failure I have seen most is a successful pilot in a friendly unit followed by an eighteen-month stall, because the second unit was never staffed and the first team got reassigned the week after go-live.

I translate the design into money and risk and drop the components entirely. A finance leader cares about three numbers: what it costs to run at expected volume, what it costs if adoption is triple that, and what we save or earn per unit of work. So I present cost per transaction rather than monthly infrastructure spend, because that makes design decisions legible, as in this choice adds a fraction of a cent per request and removes a compliance exposure. I bring the do-nothing baseline and the cheapest credible alternative, including the option where we do not build it, since a CFO who suspects they are being shown only the option we prefer will discount everything else I say. Then the risk side in plain terms: what commits us for how long, what is cancellable, and where the spend ramps. I name my uncertainties with ranges attached, because a confident single number that turns out wrong costs credibility on every future proposal. What I have learned not to do is defend elegance. If I cannot explain why a component exists in terms of a cost or risk it removes, it probably should not exist.

Some lock-in is worth paying for, and pretending otherwise produces a lowest-common-denominator design that wastes the capability you are buying. I sort dependencies by how expensive they would be to unwind. Inference is cheap to switch provided the application talks to an internal contract rather than a vendor SDK, so I abstract there and keep the evaluation suite provider-neutral, which is the thing that actually makes switching possible; without it you cannot tell whether the replacement is worse. Embeddings cost more, because changing them means re-embedding the corpus and re-tuning retrieval, so I treat that as a two-year commitment and budget the eventual migration. Data gravity is the worst of them. Once the vendor holds the documents, the labels, and the feedback, leaving becomes a programme rather than a decision, so my rule is that the client's data and its derived artifacts stay in the client's own storage and stay exportable, whatever else we buy. Conversely I will happily use a proprietary feature that gives a genuine advantage, as long as we have written down what we would do without it and roughly what that would cost.

I plan for a band instead of a number and make the system's behaviour at both ends explicit. Low case: what does it cost when almost nobody uses it? That number decides whether the client can afford to keep it running through a slow first quarter, and it argues for scale-to-zero and shared infrastructure rather than dedicated capacity. High case: what breaks first, and it is rarely the model. Usually it is a provider rate limit, a connection pool into the system of record, or an embedding job that cannot keep up with document churn. I identify that first bottleneck deliberately and either raise it or put a queue in front of it. Then I protect the business from the tail with hard controls, meaning per-tenant quotas, a global spend ceiling with alerting well before it, and graceful degradation to a cheaper model or a queued response rather than an outage. Contractually I avoid committing to a large reserved block before real usage data exists, and I tell the client that their first month of production traffic is worth more than any forecast we could write today.

I refuse to sign a statement of work whose success criterion is that the system was delivered, because that guarantees a fight at the end. What I want instead is a small number of measurable acceptance criteria agreed before build starts, defined over a data set both sides have seen. Typically that means task-level quality on a held-out evaluation set the client's own experts labelled, a latency budget at a stated percentile and load, and an operational criterion such as runbook handover and a named support path. Business outcomes like hours saved or deflection rate belong in the document too, but as a jointly measured target with a baseline taken before we start, not as a pass-fail gate, since they depend on adoption the client controls and I do not. I also write down what is explicitly out of scope and how change is priced, because scope creep on these projects always arrives as a question about handling some additional document type. The uncomfortable part is agreeing a quality number early, so I handle it with a short paid discovery phase that establishes the baseline first.

The pilot's code is usually not the asset. The evaluation set, the prompts, and the domain knowledge encoded in them are, and I say that early so nobody defends a notebook out of pride. I start by writing down what the pilot actually does, including the manual steps its authors perform without noticing, because those are the requirements that never reach a handover document. Then I rebuild on the target platform incrementally while the pilot keeps running in parallel, comparing outputs on identical inputs until they agree within an agreed tolerance. That parallel period is what earns the original team's trust; a big-bang cutover where quality shifts and nobody can explain why sets a programme back months. Non-negotiables in the target are an owner, a deployment path, versioned prompts and models, logging, and visible cost. The tension to manage is speed, since the platform version changes more slowly than the notebook did and the original team feels that as a downgrade, so the paved path has to give them something back, usually faster evaluation and an end to manual Friday runs.

First I find out precisely what the constraint is, because that sentence is usually a summary of something more specific: a regulation, a contract clause, or an internal policy somebody wrote. The options differ enormously depending on which. If it is policy, the answer may be a legal review of the vendor's in-region deployment plus a data processing agreement. If it is a regulation covering a particular data class, the design answer is often to keep that class local and send only what is permitted, so redact or tokenise identifiers, retrieve locally and pass derived text, or run a small in-region model for the sensitive step and the hosted model for the rest. If it is absolute, I stop negotiating and design for the best in-region option, usually self-hosted open weights, and I tell the client plainly what capability that costs them so the tradeoff is made by the people accountable for it. What I will not do is build the architecture on an exception somebody has promised to obtain, since those get withdrawn at the security review and the design collapses at the worst possible moment.

I make the boundary explicit per layer rather than picking one answer for the whole system. Compute and the serving layer are shared, because that is where the economics come from. Data is isolated per subsidiary, with separate stores, separate indexes, and separate keys, since a shared index with a filter is one bug away from a cross-entity disclosure, and between separate legal entities that is a reportable incident rather than an embarrassment. Policy and configuration are per tenant: model version, retention, region, guardrail settings, permitted use cases. Between those sits what makes it a platform rather than a hosting arrangement, meaning an identity model, per-tenant quotas so one subsidiary's batch job cannot starve another, and metered usage producing a bill each finance team accepts, which usually needs to be defensible more than it needs to be precise. Governance is the harder half. Who decides when the shared model version upgrades, and what happens when one subsidiary refuses? I would write that into an agreement upfront, allowing a bounded window on an older version with a stated end, or the platform fragments within a year.

It has come up a few times and it is usually one of four situations. The data does not exist, meaning not that it is messy but that the ground truth they want to predict was never recorded, and no architecture repairs that. The value is too thin, so the process being automated costs less annually than the run cost of the system, which happens more often than people expect once evaluation and support are priced honestly. The organisation cannot absorb it, because nobody will own it, the affected team was not consulted and will route around it, or the sponsor leaves next month. Or the risk is misallocated, where the client wants automation on a decision whose wrong answer is a regulatory event and they have not accepted the residual error rate. How I say it matters as much as saying it. I bring the analysis rather than an opinion, offer the smaller version that would work, often decision support instead of automation or fixing the data foundation first, and put the number on the table. A client told no once tends to believe the yes afterwards.

I design as though the notice is coming, because eventually it always is. The structural defences are a provider-neutral request contract inside the application, prompts stored as versioned artifacts rather than embedded in code, and the one that decides everything, an evaluation suite good enough to tell me whether a replacement is genuinely worse on our tasks with our data. Without that, migration is guesswork and the team spends ninety days arguing about impressions. Given the notice, my sequence is to run the suite across two or three candidates in week one, choose on quality and cost together, then work through the differences evaluation will not show: tokenizer and context limits, tool-calling format, latency profile, and prompt behaviours that were tuned to the old model's quirks. Expect to re-tune prompts, since that is the actual work. Then shadow the replacement against production traffic, compare on identical requests, and ramp by cohort with the old model available until the window closes. I flag early to the business that quality changes and not just cost, and some tuned behaviour may not be recoverable in time.

I aim for something light enough that teams use it willingly, because a heavyweight board just gets bypassed by anybody with a deadline. Concretely: a short written decision record for choices that are expensive to reverse, meaning vendor selection, data boundaries, a new persistent store, or anything committing spend past a threshold, reviewed by two or three named people inside a fixed turnaround, with approval as the default if nobody objects in time. Speed is what buys compliance. I publish those decisions in one searchable place, because most of the value is not the review but the record, so the next team discovers we already evaluated this vendor and why we rejected it. Alongside that, a small set of defaults forming the paved path, meaning approved patterns for retrieval, evaluation, secrets, and tenancy, so most projects need no review at all. To make it survive me, the reviewers cannot all be architects, so I rotate senior engineers through it and the judgement spreads. And I measure it honestly: if decisions are being made outside the process, the process is wrong, not the teams.