Capability Interfaces Instead of a Lowest Common Denominator

#Abstract

An abstraction over heterogeneous storage backends must either reduce to the intersection of what all backends support or fail at runtime when a driver cannot honour a call. This note describes a third option, in which optional behaviour is expressed as separate interfaces that calling code detects, and argues the same construction applies to object stores, query builders and message transports.

#The two unsatisfactory options

The intersection. Expose only what every backend supports. This is safe and it discards the features that motivated choosing a particular backend. Pre-signed URLs, server-side copy, versioning and range reads exist on some backends and not others, and an abstraction offering none of them forces every caller that needs one to bypass it.

The union with runtime failure. Expose everything and return an error when the backend cannot comply. This preserves capability and moves discovery to production, since nothing in the type tells a caller which operations are available.

#Construction

The core interface contains only universally supported operations. Each optional behaviour is a separate interface, and a caller that needs one asserts for it.

1type Store interface {
2    Get(ctx context.Context, bucket, key string) (io.ReadCloser, error)
3    Put(ctx context.Context, bucket, key string, r io.Reader) error
4}
5
6// Optional. Present on object stores, absent on a local filesystem.
7type PreSigner interface {
8    PreSignGet(ctx context.Context, bucket, key string, ttl time.Duration) (string, error)
9}
10
11if ps, ok := store.(PreSigner); ok {
12    url, err := ps.PreSignGet(ctx, bucket, key, time.Hour)
13    // ... hand the URL to the client
14} else {
15    // Deliberate, visible fallback: proxy the bytes ourselves.
16}
Copy

The difference from the union approach is where the branch appears. Here the caller writes the fallback at the point of use, with both paths visible in review. In the union approach the fallback does not exist and the failure is discovered by a user.

#A note on the word capability

The term is overloaded and the overload is worth clearing up before it causes confusion, because the older meaning is a security one and this is not it.

In the security literature a capability is an unforgeable token that both designates a resource and conveys the authority to use it, so that possessing the reference is possessing the permission [1] [2]. That is a strong idea with real consequences for how authority is delegated and confined [3] [4], and the interfaces described here convey no authority at all. Holding a PreSigner does not grant permission to pre-sign; it indicates that the backend implements the operation.

What is described here is feature detection with type system participation. The reason to name it carefully is that the two ideas compose badly if confused: an interface that indicates a feature is present says nothing about whether the caller is permitted to use it, and a design that treats possession of the interface as permission has an authorisation hole. Permission remains a separate concern above this boundary.

#Three kinds of variation, only one of which this handles

Backends differ in at least three ways, and treating them as one problem is why abstractions over storage tend to fail. Only the first is addressed by detectable interfaces.

Presence of an operation. Pre-signing, server-side copy, versioning, range reads. The operation either exists or it does not, the answer is fixed for a given backend, and it is exactly what an interface assertion establishes. This is the easy case and it is the one the construction solves.

Difference in guarantee. Two backends both implement Put, and one makes the object visible to a subsequent Get immediately while the other does so eventually. Both satisfy the interface. Nothing in the type distinguishes them, and a caller written against the strong one is subtly wrong against the weak one. Object stores have differed on exactly this point historically [5] [6], and it is the sort of difference behavioural subtyping is meant to rule out and which an interface without stated behavioural obligations does not [7]. The construction here does nothing about it, and I do not have a good answer beyond documenting the guarantee per driver and hoping it is read.

Difference in cost. An operation exists everywhere and is three orders of magnitude slower on one backend, a per-key listing on a filesystem against an object store being the usual example. A caller cannot detect this and the program is correct, merely unusable. Expressing cost in the type system is a longer-standing unsolved problem and is not attempted.

The interfaces make presence visible. They leave guarantee and cost exactly as invisible as they were, and a reader who takes the construction as a general solution to backend heterogeneity has taken more from it than it offers.

#Why this is the module criterion

Parnas asks what decision a module hides [8]. The core interface hides how bytes are transferred, which every backend genuinely decides for itself. It does not hide whether pre-signing is possible, because that is not a decision the module can make on the caller's behalf: the correct fallback depends on the application.

The end-to-end argument gives the same answer from the other direction [9]. A property the lower layer cannot guarantee should not be presented as though it can.

#Testing both branches

A capability check creates two paths and the untaken one rots. This is a genuine cost and it has a practical answer worth recording, since without it the construction trades a production failure for a silently broken fallback.

The test fixture provides two fake stores from the same core implementation, one implementing the optional interfaces and one not, and the application's test suite runs against both. That makes the fallback path exercised by default rather than by discipline. It is cheap because the fakes share everything except which methods they expose, and it catches the common failure, which is a fallback written once, never run, and wrong.

It does not catch the second and third kinds of variation above, since a fake has whatever guarantees the fake was written to have.

#Generalisation

The same shape appears in three other places examined:

• Query construction across relational and document stores, where set operations, window functions and upsert semantics differ. Exposing a per-driver builder rather than an intersection preserves them.
• Message transports, where ordering guarantees, delivery semantics and consumer group behaviour vary.
• Caches, where atomic operations and eviction policy differ.

In each case the workable boundary is below the point where backends diverge, which usually means lower than the abstraction's designer would prefer [10].

#Costs

Calling code is more verbose, because a capability check is a branch. Two paths must be tested rather than one, which the fixture above addresses at some cost of its own. A caller that forgets the check and asserts unconditionally still fails at runtime, so the construction reduces rather than eliminates the failure class.

A subtler cost appears when an operation needs two optional capabilities together. Asserting for each separately produces four branches, of which two are usually the same fallback, and asserting for a combined interface requires that combined interface to have been declared in advance by someone who anticipated the pairing. Neither is satisfactory. In practice the number of genuinely useful combinations turned out to be small, which is a statement about the backends examined rather than a property of the approach.

#Limitations

No measurement is offered. The claim that this reduces production incidents relative to the union approach is plausible and untested, and testing it would require two implementations of a comparable application, which was not done.

The construction assumes a language with runtime interface assertion. In a language without it, the equivalent is a capability descriptor queried at construction, which is weaker because the compiler does not participate: a descriptor can be consulted and ignored, whereas a failed assertion produces a variable the caller does not have.

The second and third kinds of variation are unaddressed, as stated above, and between them they account for the more damaging failures. An abstraction that makes feature presence visible while leaving consistency guarantees invisible may even be net harmful, by increasing confidence that heterogeneity has been handled. I do not think it is, because the alternative designs leave presence invisible as well, but the argument is comparative rather than absolute and a reader is entitled to weigh it differently.

Finally, the generalisation to query builders, transports and caches is asserted from having built variations of each rather than demonstrated. The three are not worked through here, and the query builder case in particular has a complication the storage case does not, since query construction has compositional structure and capability detection interacts with composition in ways a straight assertion does not capture.