FARP: A Manifest Protocol for Gateway Route Derivation

#Abstract

Gateway configuration duplicates information the service already holds, producing two authorities that agree only when written. This specification describes a manifest published by each service and consumed by gateways, in which the routing table is a pure function of the manifests currently visible. Two implementations exist, written against the specification rather than against each other, and their agreement is the evidence offered that it is complete. Both share one author, so that independence is partial.

#Problem statement

A service knows its routes, since they are its handler registrations. It knows its request and response shapes, since they are its types. It knows its protocols and ports. This information is then restated in a gateway configuration held in a different repository, owned by a different team, deployed on a different cadence.

The resulting failure is mundane and continuous: an endpoint ships, the gateway is unaware, a ticket is raised, configuration is edited, and a deployment occurs whose entire content is catching up with a previous deployment. This is the module criterion violated across a process boundary [1]: knowledge of a component's interface is held outside the component.

The duplication is not merely wasteful. Two authorities holding the same information will disagree, and the interesting question for any such design is what happens during the disagreement. Here the gateway wins, because it is the thing traffic passes through, which means the service's own view of its interface is the one that is wrong in practice however correct it is in principle.

The service is the only component that cannot be mistaken about its own routes. Every design that stores them elsewhere has chosen to trust a copy over the original.

#Scope and non-goals

This specification covers how a service describes its own interface and how a gateway derives a routing table from a set of such descriptions. It is deliberately narrow, and the exclusions are load-bearing rather than a matter of what has not been written yet.

Out of scope: traffic management, retry policy, rate limiting, authentication, authorisation, circuit breaking, canary weighting and observability configuration. Each of these is a gateway concern with its own operational owner, and folding any of them into the manifest would make the service's deployment the trigger for a change to the gateway's operational posture, which is exactly the coupling this protocol exists to remove in the other direction.

Also out of scope: how a gateway discovers which services exist. The protocol assumes a set of reachable services and describes what each publishes. Discovery mechanisms differ by environment and the protocol is intended to sit above all of them.

#Terminology

A service is a process that handles requests and publishes a manifest. A manifest is the document describing one service's interface at one instant. A gateway is a process that collects manifests and routes requests. A route is a path pattern, method set and upstream target. The routing table is the complete set of routes a gateway will serve. A path claim is a service's assertion, via its manifest, that it handles a given path pattern.

Requirement keywords are used in their conventional normative sense. A conforming implementation must satisfy every MUST and is expected to satisfy every SHOULD absent a stated reason.

#The manifest

A service publishes its API surface together with health semantics and capability hints. The document is generated from handler registrations rather than written by hand, which is what makes the claim of a single authority true rather than aspirational: a manifest maintained separately from the code is the gateway configuration problem relocated one repository.

1{
2  "farp": "1.0",
3  "service": { "name": "assets", "instance": "assets-7c4f", "version": "2.3.0" },
4  "health": { "path": "/healthz", "interval": "5s", "unhealthy_after": 3 },
5  "routes": [
6    {
7      "path": "/v2/assets/{id}",
8      "methods": ["GET", "PUT", "DELETE"],
9      "idempotent": ["GET", "PUT", "DELETE"],
10      "schema": { "format": "openapi-3.1", "ref": "/.well-known/openapi.json" },
11      "weight": 100
12    }
13  ],
14  "capabilities": { "streaming": false, "max_body_bytes": 1048576 }
15}
Copy

Schema references name their format rather than assuming one, since services in one estate are rarely written against a single description language. OpenAPI is the common case [2] and is not privileged by the protocol.

The idempotency list is the field that carries the most value per byte. Whether a given method on a given endpoint is safe to retry is knowledge the service has and the gateway needs, and in the absence of a channel like this it is supplied by human estimate, usually by assuming the HTTP method semantics hold [3]. They frequently do not: a PUT that appends rather than replaces is not idempotent whatever the specification says about the verb, and only the service knows that [4].

#Routing as a function of manifests

Because ρ is a function rather than a merge of two authorities, two conforming gateways given identical manifest sets must produce identical routing tables. That is the central conformance requirement and it makes the specification testable independently of either implementation.

Static routes participate in the same precedence rules and are not a compromise of the model. The bootstrap route by which a gateway reaches discovery cannot itself be discovered, which is a general property of reconciliation-based systems rather than a defect here [5].

#Precedence

Two services may legitimately claim one path at different versions. Resolution is declared rather than first-writer-wins.

Version, then declared weight, then recency as a tiebreak. Recency is last deliberately, so that a restarted instance does not win a contest it would otherwise lose. That ordering was not obvious: the first implementation used recency first, on the reasoning that the newest information is the best information, and the effect was that a crash-looping instance repeatedly seized a contested path from a healthy one.

#Manifest lifecycle

A manifest describes one instant and must therefore be refreshed. The lifecycle has four transitions and each has a normative requirement attached, because implementations that differ here produce routing tables that differ under identical inputs, which violates the central conformance property.

1. Publication. A service publishes on becoming ready to serve, not on process start. A manifest published by a process that is still warming produces routes to an upstream that will reject them.
2. Refresh. Gateways re-read manifests on an interval the service declares. A manifest that has not been refreshed within its declared interval is stale but not withdrawn, because a single missed refresh is far more likely to be a transient collection failure than a service disappearing.
3. Withdrawal. A service leaving gracefully withdraws its manifest and MUST continue serving until the gateway acknowledges, since the alternative is that in-flight requests are routed to a process that has already stopped listening.
4. Expiry. A manifest that has been stale beyond a multiple of its refresh interval is removed. This is the ungraceful path, and its duration is the window during which traffic is routed to something that is gone.

#Conformance requirements

1. A gateway MUST produce identical routing tables from identical manifest sets. This is the central requirement and every other one exists to make it achievable.
2. A gateway MUST reject a manifest failing validation in its entirety and MUST NOT apply the valid portion. Partial application makes the routing table a function of which fields happened to be malformed, which is not a function of the manifest set in any useful sense.
3. Health semantics MUST be taken from the manifest. A gateway MUST NOT substitute a default probe path or interval when the service has declared them.
4. Precedence MUST follow version, then weight, then recency, with no implementation-defined tiebreak.
5. A gateway MUST NOT admit a path claim that falls outside the service's authorised prefix set, as described below.
6. A gateway SHOULD continue serving from the last valid manifest set when collection fails, and MUST expose the age of that set.

Requirements two, four and six were added only after a second implementation made different and individually reasonable choices, which is the argument for two implementations being a completeness test rather than a duplication of effort. Each of the three was a place where the specification had said nothing and both authors had assumed their reading was the only one.

#Security considerations

Deriving the routing table from what services claim about themselves moves an authority from the gateway to the services, and that is a security-relevant change however good it is for maintenance. A service that is compromised, or merely misconfigured, can claim any path.

The concrete attack is short. A low-privilege service publishes a manifest claiming the path pattern used by an administrative service, at a higher declared version. Precedence resolves in its favour. Requests intended for the administrative service now arrive at the attacker's handler, complete with whatever credentials the caller attached.

The protocol's answer is that path claims are constrained rather than trusted. Each service identity is associated, outside the manifest, with a set of path prefixes it may claim, and a claim outside that set is rejected with the manifest. This is deliberately a static policy held by the gateway operator: it is the one piece of routing knowledge that must not come from the service, because it is precisely the knowledge that bounds what the service may assert. Complete mediation applies, and a check the claimant can influence is not a check [6].

Two residual exposures are worth stating. Prefix policy is coarse, so a service compromised within its own prefix can still shadow its own sibling routes, and the protocol offers nothing against that. And manifest transport integrity is out of scope, which means the protocol assumes an authenticated channel and says nothing about how to get one.

#Failure behaviour

Discovery moves onto the critical path for routing, and discovery has its own failure modes. Both implementations cache the last known good manifest set and continue serving from it, converting a discovery outage into staleness. This is the right trade and it is not free.

Staleness has a distinct failure of its own: an instance that is genuinely gone continues receiving traffic until health checks converge [7]. The specification requires health semantics to come from the service precisely so that this convergence time is declared and can be reasoned about, rather than being a property of whichever gateway happens to be deployed.

A second-order effect took longer to notice. When discovery is unavailable and gateways serve from cache, they are serving from caches populated at different times, so two gateways may hold different routing tables while both believe they are conforming. The conformance requirement is stated over identical manifest sets, and during a discovery outage the sets are not identical. Requirement six exists so that this condition is at least visible, since a gateway that exposes the age of its manifest set lets an operator see the divergence rather than infer it from inconsistent behaviour.

#What the protocol deliberately does not do

The end-to-end argument applies to what belongs in the gateway at all [8]. This protocol performs no traffic management, no rate limiting and no policy enforcement beyond the path claim check, and holding that boundary is what allows two structurally different gateways to consume the same manifests.

Static routes participate in the same precedence rules and are not a compromise of the model. The bootstrap route by which a gateway reaches discovery cannot itself be discovered, which is a general property of reconciliation-based systems rather than a defect here [5]. Keeping the bootstrap explicit and subject to the same precedence is preferable to a special case that behaves differently, which is a general design preference worth stating: a mechanism that handles the common case and the bootstrap case identically has one behaviour to understand rather than two [9].

#Evidence, and its limits

Two implementations exist, in different languages, written against the specification rather than against each other. Their agreement on identical manifest sets is the evidence offered that the specification is complete enough to implement from. That evidence is weaker than it sounds, in three specific ways.

No conformance suite is published alongside this specification. Until one exists, agreement rests on manual comparison across the cases the authors thought to compare, and the three requirements added after the second implementation are direct evidence that manual comparison finds gaps only where somebody looks. An executable suite, structured as manifest sets paired with expected routing tables, is the single most useful thing that could be added and is the obvious next piece of work.

Both implementations share one author, so the independence of the second is partial. Shared assumptions survive both. An implementation by an unrelated party would test the specification considerably harder, and the fact that three ambiguities surfaced even between two implementations by the same person suggests how many an unrelated implementer would find.

Finally, neither implementation has been operated at a scale where manifest collection itself becomes the bottleneck. The routing table is recomputed from the full manifest set on every change, which is correct and is O(number of services) per change. At a few hundred services this is unmeasurable. The scale at which it stops being unmeasurable has not been established, and an incremental derivation would need its own conformance argument to show it produces what the full recomputation would.