Skip to content
Marc Allington
Marc Allington
SubscribeSign in
AI

Smart AI teams take a local first approach

Marc AllingtonMarc Allington · 17 min read
Smart AI teams take a local first approach

Local first means every service your application depends on runs on the developer's machine before it runs anywhere else. Database, auth, cache, queue, email, file storage, and increasingly the AI model. Not a shared staging environment and not the free tier of someone's SaaS. The laptop, or for the largest teams a cloud machine that behaves like one.

(It's a different idea from "local-first software", where user data lives on the device and syncs later. That's a product architecture. This is about how a team works.)

Every external dependency in the inner loop is something that can be down, rate limited, slow, or configured correctly for one developer and not another. It can also be quietly costing money while someone runs the test suite for the fifth time. A developer who can't work because a third party's dashboard is having a bad morning is a developer I'm paying to wait. Multiply that across a team and it's a real number.

There's a quality argument too. If the environment is reproducible, bugs reproduce. Most "works on my machine" problems come from machines that differ in ways nobody wrote down.

The model runs on the laptop, and that's the decision that matters most

Ollama runs open-weight models on a laptop and exposes an OpenAI-compatible endpoint at http://localhost:11434/v1. The standard OpenAI SDK works against it with a changed baseURL and a dummy key. So the application talks to one interface and the environment decides which model answers.

That changes how a team works. The local model is not where prompts get written. Prompt behaviour is specific to the model, and a prompt tuned against an 8B model tells you little about what the production model will do with it. What the local model is for is everything around the prompt: the tool call firing, the context being assembled, the output being parsed, the fallback running when the model times out, the retrieval query returning the right rows. That code is most of an AI feature and it changes far more often than the prompt does. It runs with the wifi off. Embeddings follow the same route, because Ollama serves embedding models and pgvector is already in the local Supabase Postgres. The one condition is the same embedding model locally and in production. Vectors from two different models aren't comparable, so a retrieval test against the wrong index proves only that the pipeline runs.

Prompt work goes against the model that will run in production, or the cheapest tier in the same family, which shares the tool-schema and formatting habits the 8B model doesn't. Prompt caching is on and there's a spend limit on the workspace the key belongs to. Fifty runs of a flow in an afternoon against that tier is pennies, so one developer's prompt loop was never the bill. The bill is what the whole team and CI run: an agent-loop integration test that makes thirty calls per task, run on every push across a team, is thousands of calls a day that check nothing about quality. A local model keeps that off the meter, and it's the budget argument I'd make to the business. Evals stay on the production model by design, under the same spend limit.

Security and legal care about a different number. During development nothing customer-shaped should be going to a third party at all. A zero-retention agreement with the vendor is a contract. Seed data that never left the building is a control. Local first makes the second one the default instead of a policy people have to remember.

Tests sit in three layers. Unit tests run against a fake behind the model interface, replaying recorded responses, so they cost nothing and give the same answer every run. Integration tests run against the local model. That's what proves the client code, the streaming, the tool-call round trip and the parsing all work against something that behaves like a model. Evals run against the production model. The middle layer needs a word on CI: Ollama installs on a hosted runner, but the standard runners have no GPU and the model is gigabytes, so cache the model between runs or put the integration suite on a GPU runner or a self-hosted one, and keep that suite small.

Two caveats. Ollama supports a subset of the OpenAI API, so logprobs, tool_choice and image URLs aren't there, and you'll find the gaps when you hit them. And a small local model doesn't behave like a frontier one. It follows a tool schema less reliably and formats JSON its own way. It refuses different things, too. An integration test that passes locally says the plumbing works, and nothing more.

So the production model's job during development is prompt work and evals. Keep a suite with fixed inputs and scored outputs. Run it against the production model on a schedule and on every prompt change that reaches a pull request, and keep the results in the repo. Local gets you fast iteration on the code around the model. The eval suite tells you whether a prompt change improved anything. If you point the integration suite at the production model and a test that passes locally fails there, or the other way round, the prompt is leaning on behaviour one model has and the other doesn't. That's a prompt bug. Keep the test.

The split I set for a team: local model for the plumbing, production model for quality, on a schedule, with a budget. Both behind one abstraction, so switching is configuration and never a code change, and so the day a better model ships you can trial it by editing one environment variable.

One interface, the model behind it chosen by an environment variable: a local model for the plumbing in dev and CI, the frontier model for a scheduled, budgeted eval suite

Next.js, React and the whole Supabase stack start from two commands

The front end was never the problem. Next.js and React have always run locally. Run npm run dev and you're working. What changed over the last few years is that the backend caught up.

With the Supabase CLI, supabase init then supabase start pulls the whole stack into Docker: Postgres, Auth, Storage, the REST API, Realtime, Edge Functions, and Studio on localhost:54323 so you can browse the database with a UI. It also starts Mailpit on localhost:54324, a local inbox that catches every email the system sends. Sign-up confirmations and password resets land there instead of in anyone's real mailbox. The CLI prints the API URL, the database connection string, and the anon and service role keys, and that's your .env.local.

Migrations live in the repo. supabase db reset rebuilds a clean database from them and runs your seed script, so every developer and every CI run starts from the same state. Supabase's own docs make the case plainly: changes show up instantly, you can work offline, it costs nothing against your project quota, and sensitive data stays on the machine.

Can a new starter clone the repo on Monday morning and run the full product by lunch without creating an account anywhere? That's the test I apply to any team's setup. If the answer is no, there's a dependency to deal with.

Caches, queues, email and object storage all run in a container

Redis is one line in docker-compose.yml. Pull redis:7-alpine (or Valkey if you'd rather stay on an open licence), map port 6379, done. Postgres you already have from Supabase. Email is handled by the Mailpit instance Supabase starts, and if something outside Supabase sends mail, point it at the same SMTP port.

Object storage is covered by Supabase Storage for most apps. If you're on AWS, LocalStack emulates S3, SQS, SNS, DynamoDB and Lambda in a single container on port 4566. Point boto3 or the AWS SDK at http://localhost:4566 with dummy credentials and your code doesn't know the difference. MinIO is the lighter choice if all you need is S3.

Background jobs are where teams drift back to hosted services without noticing. My rule: choose a queue that runs where your other containers run. BullMQ on Redis or pg-boss on Postgres both qualify. A job runner that only exists as a hosted product is a dependency you can't reproduce, and it will be the thing that breaks the day before a demo.

A service goes in docker-compose.yml, or someone writes down why it can't.

A local equivalent exists for almost everything

This is the list I keep. Most of it is a Docker image; the rest is a CLI the vendor ships.

What you need

What teams usually reach for

Runs locally with

Relational database

Supabase, RDS, Azure SQL, PlanetScale

Postgres, MySQL or SQL Server images; the Supabase CLI

Document database

MongoDB Atlas, Cosmos DB

mongo image; Cosmos DB emulator

Cache

Upstash, ElastiCache, Azure Cache

redis or valkey image

Queue and messaging

SQS, SNS, Service Bus, Confluent, CloudAMQP

LocalStack; Service Bus emulator; rabbitmq image; Redpanda or Kafka in KRaft mode

Background jobs

Hosted job runners

BullMQ on Redis, pg-boss on Postgres, Inngest dev server (npx inngest-cli dev)

Email

Resend, SendGrid, Postmark, SES

Mailpit (SMTP in, web inbox out); Supabase starts one for you

Object storage

S3, Azure Blob, Supabase Storage

MinIO, LocalStack S3, Azurite, Supabase Storage

Auth and identity

Auth0, Clerk, Entra ID

Supabase Auth; Keycloak image for OIDC and SAML

Payments

Stripe

Stripe CLI, stripe-mock, localstripe

Search

Algolia, Elastic Cloud

Meilisearch, Typesense, OpenSearch or Elasticsearch images

Vector store

Pinecone, hosted Qdrant

pgvector in Postgres; Qdrant or Chroma images

Language models

OpenAI, Anthropic, Azure OpenAI

Ollama, LM Studio, vLLM

Serverless functions

Lambda, Azure Functions, Vercel, Cloudflare Workers

SAM or LocalStack; Azure Functions Core Tools; vercel dev; wrangler dev

Realtime and websockets

Pusher, Ably

Supabase Realtime; Soketi (Pusher-compatible)

Feature flags

LaunchDarkly

Unleash or Flagsmith self-hosted; OpenFeature with an in-memory provider

Secrets

Cloud key vaults

.env files from .env.example; Vault in dev mode when you need the API

Observability

Datadog, New Relic

OpenTelemetry collector with Grafana's otel-lgtm image or Jaeger; the Aspire dashboard

Local HTTPS

Real certificates

mkcert

Containers in tests

Shared test environments

Testcontainers, available for Node, .NET, Java, Go and Python

The editor itself

Everyone's own machine setup

Dev Containers (devcontainer.json) so the toolchain is in the repo too

Two things are missing on purpose. SMS providers have no local mode worth using, so they go behind an interface with a fake that logs to the console. And inbound webhooks from a provider with no CLI need a tunnel such as cloudflared or ngrok, which breaks the offline rule and is why I count it as a last resort rather than a tool.

Stripe is the exception, and the Stripe CLI gets you most of the way

Card networks exist and you don't own one. But you can get close, and there are three layers to use.

The first is the Stripe CLI. stripe listen, pointed at localhost:3000/api/webhooks, opens a connection from your machine to Stripe and forwards every sandbox event to your local endpoint. No ngrok and no public URL. It prints a signing secret for local use and stripe trigger payment_intent.succeeded fires events on demand. Your webhook handler is the same code that runs in production.

Stripe's own mock server, stripe-mock, is the second layer. It runs in Docker on port 12111, accepts the same parameters the real API does and returns responses of the right shape. Unknown parameters get rejected. Stripe uses it in its own SDK test suites and says openly that it doesn't reproduce real behaviour. So it's for unit tests, where you want to know you called the right endpoint with the right fields, and nothing more.

The third is localstripe, a community fake that's stateful: create a customer and it's still there on the next request. It can even stand in for Stripe Elements in the browser. It tracks the latest API on a best-effort basis, so treat it as a tool for end-to-end flows that must run offline, with a sandbox smoke test to back it up.

The pattern is unit tests against stripe-mock, the daily inner loop against the sandbox with CLI forwarding, and a small labelled suite against the sandbox in CI. The same pattern applies to any provider you can't run locally, SMS and identity included. Find the local mode. If there isn't one, put the provider behind an interface and fake it.

.NET and Java teams have the same tools, and in places better ones

Both frameworks start the containers for you, so there's no compose file to maintain by hand.

On .NET, Aspire is the answer. You describe your dependencies in an AppHost project (Postgres, SQL Server, Redis, RabbitMQ, Kafka, Keycloak) and dotnet run starts the containers and wires the connection strings into each service. A dashboard shows logs and traces. For Azure resources it has an emulator pattern: RunAsEmulator on a storage resource swaps the real account for Azurite in a container, with no subscription and no credentials. Cosmos DB and Service Bus have emulators of their own. Azure Functions run locally through the Core Tools. SQL Server runs from the mcr.microsoft.com/mssql/server image. For the AI layer, Semantic Kernel has an Ollama connector and the Aspire Community Toolkit adds Ollama as an AppHost resource, so the same local-model split applies.

On Java, Spring Boot has had this built in since 3.1. Add spring-boot-docker-compose and the app runs docker compose up at startup and reads the connection details off the running containers. When the app stops, so do they. Or use Testcontainers at development time: declare your containers in Java with @ServiceConnection and launch with mvn spring-boot:test-run, and the same declarations drive your integration tests. Quarkus goes a step further with Dev Services, which start Postgres, Redis or Kafka automatically in dev mode with no configuration at all. Spring AI and LangChain4j both have Ollama support, and LocalStack covers the AWS side for either language.

The frameworks have already made local first the default. The only decision left on a .NET or Java project is which containers go in the AppHost or the compose file.

The principle survives when a team outgrows the laptop

What has to survive the move is one environment per developer, built from the repo and thrown away when it's done. Where it runs is the only thing that changes. There are two patterns, and I'd run both.

The first is remote development environments. Stripe is the well-documented case: a Ruby monolith of millions of lines, plus a growing number of services outside it, which no laptop can run. Each developer gets a "devbox", a cloud VM running the whole system, with code synced from the laptop on save, and later the editor itself running on the box through VS Code's server. Their stated reason was accuracy: the dev environment has to behave like production, and a MacBook doesn't. A former Stripe engineer's write-up of the design makes the point that keeping the devbox ephemeral is what stopped environments drifting over time, which is the same argument as supabase db reset at a different scale. The pattern is off the shelf now: GitHub Codespaces, Coder and Gitpod each give a developer a cloud machine built from the devcontainer.json in the repo. Each one is isolated and disposable, and local first in everything but location. When I see thirty-plus services, or a full build that runs past a few minutes, this is where I push a team.

The second is an ephemeral environment per pull request. Every PR gets its own full deployment with its own database, alive for the life of the PR and deleted on merge. Vercel preview deployments handle the front end. Supabase Branching creates a separate Supabase instance per PR and builds it from the migrations and seed.sql in the repo. Neon does the same for Postgres and can fork from production-sized data. On Kubernetes, Argo CD or Flux with a namespace per branch does it for anything else. Review changes with it. The reviewer clicks a link and uses the feature rather than reading a diff and trusting the author, and QA happens before merge instead of after. Nobody waits for staging to be free.

Laptop, cloud devbox and one environment per pull request, all standing on the same rule: one environment per developer, built from the repo, thrown away after

For AI work this is where the reviewer judges quality. The local model proves the plumbing. The ephemeral environment runs the eval suite against the production model, on data the size production sees, for every PR that touches a prompt or a retrieval path. That's inference spend worth making, because it's attached to a specific change and a specific reviewer, and it stops when the PR closes.

One rule holds at every size. An emulator is never the last line. stripe-mock, LocalStack and Azurite buy you a fast inner loop. A labelled smoke suite against the real service, run in CI, is what tells you the emulator didn't lie.

What a lead gets out of it

Onboarding becomes a morning. Clone, docker compose up, supabase start, npm run dev. No tickets raised for access to a staging account, no service role key pasted into Slack.

CI matches the laptop. The same containers run in the pipeline, so a green build means something. LocalStack runs as a GitHub Actions service container for exactly this reason.

Then there are the tests a shared staging database stops you writing, the ones that matter most: they create an order, fail a payment, retry it, cancel it, and check every table afterwards. On shared data those tests collide with whatever a colleague is doing, so people make them read-only, or run them once a week and hope. With a database per developer, every test starts from a known state and resets to it afterwards, whatever it changed in between. Two developers can run the same destructive suite at the same time. CI can run it in parallel across jobs, each with its own containers. The AI flows benefit most, because an agent test might touch a dozen tables and a vector index in one run, and you want that run to be yours alone.

The bill goes down. Test suites stop burning quotas on dev projects and sandbox accounts, and nobody has to explain an unexpected invoice.

And customer data stays off laptops. There's no reason for it to be on eight of them when seed data does the job, and there are far fewer shared credentials to rotate when someone leaves.

These are the rules I set for a team:

  • Every dependency has a local mode, or sits behind an interface with a fake.

  • .env.example describes a complete working local environment. Nothing lives only in someone's head.

  • Seed data is in the repo and supabase db reset gets you to it.

  • The test suite passes with the network off. External smoke tests are a separate, labelled suite.

And one position I'll hold whenever the team evaluates a new service. "Can we run it locally?" is a first-order question, right next to price. If the answer is no, the service goes behind an interface with a fake before anyone writes a line against it.

Sources

#local-first#Developer experience#ollama#supabase#leading-delivery

Comments

Sign in or create an account to join the conversation.

Be the first to comment.

Enjoyed this? Get new articles by email.

Subscribe