Publishing the private half

#What is changing

Nineteen repositories go public over the next few months: Bastion, Dispatch, Ctrlplane, Relay, Confy, ForgeUI, Grove, Trove, Ledger, Authsome, Warden, Vault, Keysmith, Chronicle, Cortex, Weave, Nexus, Sentinel and Shield. Every one of them has lived in a private organisation since it was written, showing up on this site as a name, a paragraph and a chip that says Private.

Most began as an extension inside Forge and were pulled out when a second service needed them.

#Why they were private

Weave took about eleven months of evenings. Keeping it closed was a bet that those eleven months of head start were worth more than anything I would learn from other people running it, and until this year the bet was clearing.

It stopped in March. I spent an afternoon with a model and came out with a durable execution library that ran: checkpointed steps, a lease-based scheduler, a dead letter queue you could replay from. It was worse than Dispatch in ways I could name inside ten minutes. It also took an afternoon against the year Dispatch took, and most of that year was me finding out which of my early decisions were wrong.

What got cheap is the accidental half of the work [1]. Typing, layout, wiring, choosing between four libraries that all do the job: a model does all of that now, faster than I do. The essential half sat still. Knowing which of two designs that both look correct will produce a support call in six months is still eight years of having picked the other one.

Eleven months of evenings bought a head start in exactly the half of the work that stopped being scarce.

So this is a release under pressure. I would rather write that down than let the announcement sound warmer than it was.

#What I do not write

I do not have the patience to write tests, and I have less of it for API documentation.

The reason is that I experiment far more than I finish. A question arrives, say whether a gateway can derive its own routing table from a manifest, or whether an append-only audit log can satisfy an erasure request without breaking its hash chain, and the fastest way to find out is to build the thing. By the time I have the answer I am interested in the next question, and the repository sits at the point where it works for me and for nobody else.

Cortex has tests for the scheduler and for nothing else. Weave's public API changed three times in six weeks last spring and its README still describes the first version. Chronicle has a doc comment on every exported symbol, because Go made me feel bad enough to write them, and no page anywhere explaining why the chain covers the previous entry's signature instead of its payload, which is the part a reader would actually need.

What a module ships is its interface, and the interface exists to hide a decision [2]. When the decision is never written down anywhere, the hiding is the entire product and nobody can tell whether it was a good call. Ousterhout makes a smaller version of the same point about comments, which is that anything written after the code describes the code rather than the intent behind it [3].

On tests I have no argument at all. Accelerate puts test automation among the practices that predict delivery performance [4], and the excuse that an experiment does not need tests holds right up until the experiment ends up in production.

#Why publishing is the repair

Publishing will not make me want to write documentation. It removes the option of skipping it. A README that lies is tolerable while I am the only reader of it, and it gets fixed within a week of somebody filing an issue. The gap between those two response times is embarrassing, and it is also reliable enough to build a plan on.

The pace is deliberately slow. Two or three repositories a month, in dependency order, starting with Confy because every other repository on the list imports it. Each one goes out with a licence, a README whose quick start actually runs, a changelog starting at the current version, and a supported configurations section naming the Go version and the databases I test against. Full API documentation follows for the ones people use, and does not follow for the ones nobody does.

#What I am worried about

I have done this before. Flutter Unity Widget crossed two thousand stars, and about sixty per cent of the issue tracker was somebody else's build configuration. Nineteen repositories is a much larger surface than one, and I do not have nineteen times the evenings.

What worked last time was drawing the edges early: supported configurations in the README from the first commit, a required reproduction, and declining anything I have no way to test. I trust that more than I trust my stated intention to keep up, which is the part of this plan with no evidence behind it.

The thing I cannot answer yet is whether a library published under this kind of pressure comes out better than one published out of confidence. My guess is that the code is the same and the documentation is thinner, because a deadline gets you a description of what shipped without the reasoning that would let a reader disagree with it. I should know by this time next year.