Long-form, irregular, and mostly about the parts of a system that only reveal themselves in production.
A TypeScript template, two Rust frameworks and one in Go. What each got wrong in order, and the two ideas that survived all of them.
Read →Oct 2025Checkpointed workflows, leader-elected cron and a dead letter queue, imported into the process you already deploy. What the model gives you and where it stops.
Read →Sept 2025Rotation that requires a coordinated deploy happens once, at audit time. A grace window where both keys validate is what turns it into a background process.
Read →Aug 2025A hash chain makes tampering detectable, which is the point. It also makes deletion impossible, which is a legal problem. Per-subject keys resolve it, with conditions.
Read →Jul 2025Roles, attributes and relationships answer different questions and most systems need all three. Fixing the precedence rule up front is most of the design.
Read →Jun 2025A manifest published by the service, consumed by the gateway. Why the configuration belongs with the service, and what putting discovery on the critical path costs.
Read →May 2025Generating each database its own dialect instead of a portable subset. What that buys, what it costs, and the benchmark numbers with their caveats.
Read →Apr 2025Every cross-tenant leak I have looked at came from one query missing one clause. Making the boundary structural means the mistake stops being available.
Read →Mar 2025Four stores, one handler, no transaction. How I ended up with a reconciliation job nobody owned, and what a single transactional write path actually costs.
Read →Feb 2025Job queues, webhook delivery, authorisation and billing as libraries you compile in rather than services you operate. The reasoning, and the three places it fails.
Read →Jan 2025Minimalism does not remove decisions, it relocates them. Twenty-three things I decided before serving a single request, and the two the framework got wrong.
Read →Nov 2024Training code is a fraction of a production machine learning system. The rest is data validation, serving, monitoring and the debt that accumulates when those are afterthoughts.
Read →Oct 2024Routing, caching, guardrails and budgets are worth having. A third party in the path of every prompt is a data governance conversation before it is an architecture one.
Read →Sept 2024The model is a client you cannot brief. Every ambiguity in a tool description becomes a wrong call, and the fixes are the ones you would apply to any public API.
Read →Jul 2024Ordering detectors by cost and precision, with the right to terminate, produces decisions with a single named cause. That is worth more than the latency it saves.
Read →Jun 2024A single accuracy number over a small hand-picked set tells you almost nothing. Baselines, scenario coverage and regression detection are what make a change reviewable.
Read →May 2024Approximate nearest neighbour search has recall you chose, a build cost you pay on every update, and no transactional relationship to the data it indexes.
Read →Apr 2024Returning the twenty things most related to an asset is easy. Explaining why each one is in the list is what determines whether an operator uses the result.
Read →Mar 2024Deriving node identity from a normalised natural key instead of allocating it makes multi-source ingestion converge. Here is exactly where that stops working.
Read →Feb 2024Most retrieval failures I have debugged had the right passage in the result set and lost it to a naive concatenation with no token budget.
Read →Jan 2024What actually degrades in a long-running agent around turn two hundred, and why bounded working memory addressed it where more tokens did not.
Read →Dec 2023The modelling is the easy part. Well-maintained equipment fails rarely, prevented failures are not recorded, and the label you need usually does not exist.
Read →Nov 2023Three questions decide where a computation runs, and none of them are about architecture preference. What breaks if the link drops, what the bandwidth costs, and how fast the answer is needed.
Read →Oct 2023The engine matters less than what you decide to keep and at what resolution, because that decision is irreversible and gets made implicitly by default settings.
Read →