XRAPH/Research/Technical note

Capability Interfaces Instead of a Lowest Common Denominator

Polyglot storage abstractions either reduce to what every backend supports or fail at runtime when a driver cannot honour a call. Opt-in capability interfaces let calling code degrade deliberately, and the same argument applies to ORMs and object stores.

Type
Technical note
Year
2022
Status
Draft
Length
2 min read

#Abstract

An abstraction over heterogeneous storage backends must either reduce to the intersection of what all backends support or fail at runtime when a driver cannot honour a call. This note describes a third option, in which optional behaviour is expressed as separate interfaces that calling code detects, and argues the same construction applies to object stores, query builders and message transports.

#The two unsatisfactory options

The intersection. Expose only what every backend supports. This is safe and it discards the features that motivated choosing a particular backend. Pre-signed URLs, server-side copy, versioning and range reads exist on some backends and not others, and an abstraction offering none of them forces every caller that needs one to bypass it.

The union with runtime failure. Expose everything and return an error when the backend cannot comply. This preserves capability and moves discovery to production, since nothing in the type tells a caller which operations are available.

#Construction

The core interface contains only universally supported operations. Each optional behaviour is a separate interface, and a caller that needs one asserts for it.

1type Store interface {
2 Get(ctx context.Context, bucket, key string) (io.ReadCloser, error)
3 Put(ctx context.Context, bucket, key string, r io.Reader) error
4}
5
6// Optional. Present on object stores, absent on a local filesystem.
7type PreSigner interface {
8 PreSignGet(ctx context.Context, bucket, key string, ttl time.Duration) (string, error)
9}
10
11if ps, ok := store.(PreSigner); ok {
12 url, err := ps.PreSignGet(ctx, bucket, key, time.Hour)
13 // ... hand the URL to the client
14} else {
15 // Deliberate, visible fallback: proxy the bytes ourselves.
16}

The difference from the union approach is where the branch appears. Here the caller writes the fallback at the point of use, with both paths visible in review. In the union approach the fallback does not exist and the failure is discovered by a user.

#Why this is the module criterion

Parnas asks what decision a module hides . The core interface hides how bytes are transferred, which every backend genuinely decides for itself. It does not hide whether pre-signing is possible, because that is not a decision the module can make on the caller's behalf: the correct fallback depends on the application.

The end-to-end argument gives the same answer from the other direction . A property the lower layer cannot guarantee should not be presented as though it can.

#Generalisation

The same shape appears in three other places examined:

  • Query construction across relational and document stores, where set operations, window functions and upsert semantics differ. Exposing a per-driver builder rather than an intersection preserves them.
  • Message transports, where ordering guarantees, delivery semantics and consumer group behaviour vary.
  • Caches, where atomic operations and eviction policy differ.

In each case the workable boundary is below the point where backends diverge, which usually means lower than the abstraction's designer would prefer .

#Costs

Calling code is more verbose, because a capability check is a branch. Two paths must be tested rather than one. A caller that forgets the check and asserts unconditionally still fails at runtime, so the construction reduces rather than eliminates the failure class.

#Limitations

No measurement is offered. The claim that this reduces production incidents relative to the union approach is plausible and untested, and testing it would require two implementations of a comparable application, which was not done.

The construction assumes a language with runtime interface assertion. In a language without it, the equivalent is a capability descriptor queried at construction, which is weaker because the compiler does not participate.

References

  1. [1]D. L. Parnas, On the Criteria To Be Used in Decomposing Systems into Modules, Communications of the ACM, vol. 15, no. 12, pp. 1053-1058, 1972doi:10.1145/361598.361623
  2. [2]J. H. Saltzer, D. P. Reed, D. D. Clark, End-to-End Arguments in System Design, ACM Transactions on Computer Systems, vol. 2, no. 4, pp. 277-288, 1984doi:10.1145/357401.357402
  3. [3]Martin Kleppmann, Designing Data-Intensive Applications, O'Reilly Media, 2017