# Building local-first software

> Notes from building quiet-index, a retrieval tool where the data never leaves the machine. What local-first buys you, and what it costs.

Published: 2026-07-29
Category: Systems
Author: Relayweave Labs
Tags: local-first, retrieval, sqlite
Canonical: https://relayweavelabs.com/notes/building-local-first-software

We built [quiet-index](/#lab) because we wanted retrieval over our own working notes without shipping those notes to anyone. It started as an experiment in how far you can get with a laptop, SQLite, and a small embedding model. The answer is further than we expected, with a few sharp edges worth writing down.

## What local-first actually promises

Local-first is often described as offline support, but that undersells it. The real promise is that the software works fully on the user's machine, with their data, and that any network involvement is an optimisation rather than a dependency. Sync, if it exists, is a feature layered on top. Nothing stops working when the server is down, because there is no server in the critical path.

For a retrieval tool the benefits are immediate. Latency is a function of your disk, not a round trip. Privacy is a property of the architecture, not a policy. And the failure modes are the failure modes of a program, which are easier to reason about than the failure modes of a program plus a network plus somebody else's infrastructure.

## SQLite carries more than people expect

Everything in quiet-index lives in one SQLite file. Documents, chunks, embeddings, the full-text index, and the metadata that ties them together. This felt like a compromise at first and turned out to be the best decision in the project.

Full-text search comes from the built-in FTS5 extension, which handles tokenisation, ranking, and prefix queries without another dependency. Vector search is a small extension that stores embeddings as blobs and does a brute-force scan with a distance function. On a working set of a few hundred thousand chunks, a brute-force scan finishes in well under a second, and that is before any indexing. Most personal or team-sized corpora never get large enough to need more.

The operational story is the real win. Backup is copying a file. Migration is a transaction. Corruption is something SQLite has spent two decades making rare.

## Hybrid retrieval beats either alone

Pure vector search is fuzzy in useful ways and useless in others. It finds the paragraph about deployment pipelines when you ask about CI, and completely misses the one document that contains the exact error string you pasted. Pure keyword search has the opposite profile.

We run both and merge with reciprocal rank fusion. The implementation is a dozen lines. The improvement in the top five results was large enough that we stopped tuning anything else.

## The costs

Local-first is not free, and the costs are mostly on the edges.

Embedding models have to run on the machine, which means picking one small enough to be tolerable on a laptop CPU and accepting that it is not the best model available. We settled on a model in the hundred-megabyte range that embeds a chunk in a few milliseconds. It is noticeably worse than the large hosted models on ambiguous queries and perfectly good on everything else.

Sync is genuinely hard if you want it. We do not have it yet, and the honest reason is that conflict resolution for a mutable index is a research problem when you approach it generally. The pragmatic path is to treat the index as derived data, sync the source documents with whatever tool already does that, and rebuild locally. That is what we do.

Distribution is fiddlier than a web app. Each platform has its own way of being unhappy about a native extension. We spend more time on packaging than on retrieval, and that ratio is unlikely to change.

## Where this is going

quiet-index is still an experiment and will stay one for a while. But the shape has convinced us of something more general. For a large class of tools, the ones that help one person or one team think about their own material, the local-first architecture is simpler to build, simpler to run, and easier to trust than the hosted alternative. The reason it is not the default is habit, not engineering.