Cloud & Deployment

A model that only runs on the laptop it was trained on is a result, not a system. The work that turns one into the other is unglamorous, and it is most of the job. The model needs somewhere to run that survives a traffic spike, a path for data to reach it that does not involve anyone emailing a spreadsheet, a way for the people who use it to say when it is wrong, and a route for that correction to reach the next version.

Running models on Kubernetes

Inference is spiky. A quality-control model sees nothing overnight and everything at shift change; a screening run is idle for a week and then wants forty GPUs for an afternoon. Kubernetes handles that pattern well, which is why we use it by default, either on GKE or on whichever cluster you already run.

Security is part of the architecture from the start

Scientific and industrial data, such as process parameters, unpublished results and patient records, is usually the most sensitive thing a company owns. The infrastructure has to assume that.

• Isolation by default. Clusters are private, nothing gets a public endpoint unless it needs one, and network policy separates workloads instead of leaving them in a flat namespace.
• Least privilege. We use workload identity instead of long-lived keys and scoped service accounts, and secrets live in a managed store, never in an image or a repository.
• Auditability. Who called the model, with what input, and what it answered is retained wherever that matters for review or regulation.
• Data residency. Storage and processing stay in the EU region where that is a requirement, and you get a clear answer about where every copy lives.
• Supply chain. Dependencies are pinned, images are scanned, and the base image gets rebuilt instead of staying pinned to 2021.

Integration with your data

The most common reason a good model falls out of use is that nobody rebuilt the path that fed it. Deployment and data plumbing are the same project.

We build the collection and validation side too. That means ingesting from instruments, databases, object storage or an existing warehouse, and validating on arrival so that a schema change surfaces as an alert instead of as an unexplained accuracy drop months later. It also means making sure the transformations applied at inference are the same ones applied during training, because when they diverge the failure is subtle and expensive.

Where a database is the right home for it, the model's outputs (predictions, confidence and model version) go back in alongside the inputs, so the record of what was decided can be queried instead of sitting in a log file.

Automated processing pipelines

Almost none of this is a single job. Between an instrument writing a file and someone reading a number there is a chain of steps, and at each of them things can stop happening without anyone noticing. That chain is worth building as one orchestrated pipeline, not as a folder of scripts and a cron entry nobody remembers writing.

1. 1

   Ingest

   From instruments, databases, object storage or an existing warehouse, either on a schedule or when something arrives.

2. 2

   Validate

   Schema, ranges and completeness are checked. A row that fails is rejected and raises an alert instead of passing through unnoticed.

3. 3

   Transform

   This is the same code path that will run at inference time, so training and serving cannot drift apart.

4. 4

   Store

   Data is versioned, and the raw input is kept alongside the derived form, because the next question is always one the current transform throws away.

5. 5

   Run

   Scoring, aggregation, retraining, or whatever else this pipeline exists to do.

6. 6

   Publish

   Results go to a database, a dashboard, an API or a report, with provenance attached to every figure.

What separates a pipeline from a scheduled script is mostly what happens when something goes wrong:

The same machinery runs the training side. A retrain is a pipeline that pulls the current labelled set, trains, evaluates against the incumbent on a held-out set, and registers the candidate. Promotion to production remains a deliberate step, never an automatic one.

Active learning: the deployment improves the model

A deployed model generates the most valuable training data you will ever get, and most deployments throw it away.

Because our models carry uncertainty alongside each prediction, we can rank incoming cases by how unsure the model was about them. Those are the cases most worth a human's attention, and their labels improve the next version most per hour of expert time. Once the loop exists, it is straightforward:

1. The model flags the cases it found hardest, instead of an operator sampling at random.
2. Someone who knows the domain resolves them, in the same tool they were already using.
3. Those resolutions land in the training store with provenance attached.
4. Retraining runs on a schedule or on a trigger, and the candidate is evaluated against the current model before anything is promoted.

Alongside this, drift monitoring watches whether today's inputs still look like the ones the model was trained on. That is what tells you a retrain is needed before performance degrades far enough for someone to complain.

Applications people use

An API is only useful to a system. When the user is a person, such as an operator on a line, a researcher at a bench or a technician in a field, the model needs a front end, and often one that works where the network does not.

We build these ourselves because of the return path. An app that shows a prediction is a demo; an app where the user can say "no, that one is a crack, not a scratch" and have it reach the training set is part of the system. Every correction, every new observation and every image the model has not seen before flows back into the same store the next training run reads from.

Where to go next