Questions

The questions worth asking first.

Including the ones that argue against buying anything. If a question here has an uncomfortable answer, that's deliberate. You'll find out eventually, and it's cheaper for both of us if it's now.

Do we even need this?

How do we know if we actually need a private system?

Three questions, in order. Does your work carry a genuine confidentiality obligation: privileged legal matter, client financial records, health information? Do you have sites or situations with no reliable internet? Do you want work happening around the clock that would be uneconomic when charged per task?

If the answer to all three is no, you probably don't need this, and an off-the-shelf business subscription will serve you better and cost less. I'd rather tell you that than sell you a machine you'll resent.

Does New Zealand law require AI to run on-premise?

No. There is no rule mandating it, and anyone who tells you otherwise is overselling. The AI strategy from MBIE is deliberately light-touch and technology-neutral.

What is true: your obligations under the Privacy Act follow the data wherever it goes. Keeping it in the building is the most straightforward way to discharge them. It is a risk-management choice you make, not a box you're forced to tick.

Isn't this cheaper than paying for cloud AI?

Usually not, and I won't pitch it that way. Frontier API prices have fallen steadily. Claude's flagship dropped from $15 to $5 per million input tokens in about a year. A ten-person firm doing the maths against per-seat subscriptions will often find the cloud cheaper.

The honest cost argument is different: a fixed monthly cost that doesn't move, immune to the per-task blowout that happens when reasoning models start emitting ten to a hundred times more tokens per job. You're buying predictability and confidentiality, not savings.

Privacy and trust

Does anything actually leave the building?

On a sealed installation, no. The machine has no outbound path to any AI provider, and that's a firewall rule you can inspect rather than a promise you have to take on faith.

If we deliberately enable web search, the search terms themselves do go out, like any browser. I'll tell you that up front rather than let you discover it. "Web-aware" and "fully sealed" are different settings and you should know which one you have.

The same honesty applies to using a frontier model. That is a mode you can be put in, not one you are in by default.

Can it use something like Claude or GPT when a question is genuinely hard?

It can, but never automatically, and this is the part most people get backwards. The obvious design is to send the hard questions to a frontier model and keep the easy ones local. The problem is that difficulty and sensitivity are the same axis: your hardest questions are the ones with the most client detail in them. Routing on difficulty would send exactly the wrong things out and keep the small talk at home.

So the choice sits with the person who wrote the question, one message at a time. They can see which model they are about to use and pick. That is the only version of this I'll build, because it is the only one where the person who understands the sensitivity is the one making the call.

Where it makes sense: engineering and survey work, where the constraint is expertise rather than confidentiality. Where it doesn't: anything privileged, which stays sealed regardless. If a route out is enabled, the cloud account runs on terms that forbid training on your content, and every call that leaves is logged where you can read it.

You'd have remote access to a machine holding our client files. Isn't that the same problem?

It's the sharpest question anyone asks, and it deserves a real answer rather than a reassurance.

For confidentiality-bound work the arrangement I recommend is that maintenance access is off by default and you switch it on for a scheduled window. Every session is logged, and the log is visible to you. It's slower for me, which is rather the point.

Could a model trained on our documents leak them to another client?

No. An adapter trained on your work is built for you, used only for you, and never reused. You own it outright. It's a few megabytes and it's yours to take.

That's written into the data-handling agreement before a single file moves, along with your warranty that you have the right to share the material in the first place.

What happens to our system if you get hit by a bus?

A fair thing to ask of a one-person business. The stack is deliberately built from standard open-source components with version-controlled configuration and written handover documentation, specifically so another competent engineer can pick it up.

You are not buying something only I can operate. That's a design constraint, not an afterthought.

What it can and can't do

Is a local model as good as ChatGPT?

On the work your staff actually do, yes: reading documents, drafting, pulling facts out of reports, answering questions about your own files. Near enough that you wouldn't reliably pick the difference.

On the hardest reasoning, very long documents, and long chains of autonomous steps, the frontier cloud models are still clearly ahead. The best model that fits comfortably on an office machine benchmarks roughly a generation behind the frontier. I'd rather you hear that from me than discover it in month two.

Where does it fall over?

Long-context reasoning is the honest weak spot. Ask a local model to reason across a hundred pages at once and quality degrades considerably more than the benchmark headlines suggest.

Long autonomous chains are the other. A three-step task (read this, extract that, format it) is reliable. A thirty-step agent left to its own devices is not, because small per-step error rates compound.

Can it learn our house style?

Yes, and that's the most valuable thing it does. Train an adapter on a few hundred of your past documents and it picks up your structure, phrasing, standard clauses and sign-off.

The rule that decides every one of these engagements: retrieval for what it needs to know, fine-tuning for how it should behave. Most firms ask for the second when they need the first, and fine-tuning a model to memorise facts makes it hallucinate more, not less.

How do we know the custom model actually worked?

A blind test, written into the contract. Ten unlabelled documents go to your senior reviewer, some drafted by the model, some by your own people. If they can't reliably pick which is which, it's delivered. If they can, it isn't.

Practicalities

How do staff use it day to day?

In the office they open a browser to an address on your own network. It looks and behaves like the chat tools they already know, with their own accounts and history. Nothing to install.

Working from home, they connect over a private encrypted network with no public entry point. People who prefer a desktop or phone app can use one pointed at the same machine.

How many people can use it at once?

Office use is bursty: ten people with a chat window open rarely produce more than two or three simultaneous requests. That's comfortable.

What you'd notice under genuinely heavy simultaneous load is queueing, not errors: responses start later rather than failing. If your firm is bigger or your usage is unusual, that's something to measure before buying rather than assume.

What happens when better models come out?

Swapping them in is part of what the monthly fee covers, and it's the main reason the fee exists. New open models land every few months.

The honest limit: hardware has a useful life of roughly two to three years before the models worth running outgrow it. That refresh conversation is one I'll start early rather than spring on you.

Who owns what at the end?

You own the hardware, your data, and any custom model trained on your work. The stack is open-source components you could keep running without me.

There's no lock-in mechanism and no hostage-taking. Retention comes from being useful, which is the only kind worth having.