Structural Multi-Tenancy: Enforcing Isolation Below the Query Layer

#Abstract

In every cross-tenant disclosure the author has examined, the cause was not a flaw in the isolation design but one query missing one predicate, written by somebody competent, in code that reads as correct. That is an observed pattern across a handful of incidents rather than a measured population. This note argues that tenant isolation belongs in the storage engine rather than the application, gives the reasoning in terms of two long-standing security principles, and proposes a test whose failure indicates the boundary is not where the team believes it is.

#The problem

The common implementation of multi-tenancy is a column. Each row carries a tenant identifier, each query carries a predicate, and a review convention requires checking for it.

This holds for queries in the usual shape and fails for the ones outside it: administrative reports, migration backfills, aggregate counts, and joins that filter one side and not the other. In each case the correct query and the leaking query are visually similar, which is what makes the class persistent rather than a matter of care.

A safety property that depends on an author remembering something is not a property. It is a habit, and habits have a measurable failure rate.

#Two principles that decide the question

Saltzer and Schroeder give complete mediation, meaning every access passes through the check with no path around it, and fail-safe defaults, meaning the default decision is denial [1]. Application-level predicates satisfy neither. A query that omits the predicate has bypassed the check, and its default behaviour is to return everything.

The end-to-end argument identifies which layer can enforce a property completely [2]. For tenancy that layer is the storage engine, because it is the only component that observes every access regardless of which code path produced it.

#Construction

The tenant is established once, at the request boundary, from validated credentials, and is carried on the request context. No component downstream may set it.

Each engine then enforces the boundary using its own mechanism, since the mechanisms differ in strength and this should be stated rather than hidden:

1. A relational store applies a transaction-scoped setting plus row-level policies, with the application role holding no bypass privilege. A query with no predicate returns the empty set.
2. A graph store is partitioned per tenant, so a cross-tenant traversal has no edge to follow. This is the strongest of the four because it is not a policy at all.
3. A search index is provisioned per tenant behind an alias with routing.
4. A key-value cache uses prefixes produced by a single key-building function. This is the weakest and relies on discipline rather than enforcement.

#The four mechanisms are not equally strong

Listing them together invites the reading that the boundary is uniformly enforced, and it is not. The honest ordering, strongest first, is worth stating because it tells an operator where to look after an incident and where to spend effort before one.

Partitioning is the strongest because it is not a policy. A cross-tenant traversal in a per-tenant graph fails for the same reason a query cannot return rows from a database it is not connected to. There is no rule to misconfigure and no bypass privilege to grant by accident. Everything below this line is enforcement that could, in principle, be turned off.

Row-level policy is next and is genuinely strong, with one significant caveat treated in the following section: the enforcement is real but its correctness depends on a privilege configuration that is easy to get wrong and whose being wrong is invisible to the obvious test.

Per-tenant index provisioning behind an alias sits below that. The isolation is real, and the alias indirection is a piece of configuration that can be pointed at the wrong place. An index reachable by its underlying name bypasses the alias entirely.

Prefixed cache keys are the weakest by a wide margin and are not enforcement at all. They are a naming convention with a single choke point, which is better than a convention with many choke points and is not the same as a mechanism that denies. Any code path that constructs a key without going through the key builder has bypassed it, and nothing will report that it did.

A boundary is only as strong as its weakest layer, and one of these four is a naming convention. That should be stated in the design document rather than discovered during an incident.

Where cache contents are sensitive rather than merely convenient, the correct response is per-tenant cache instances, accepting the cost, rather than a better prefix scheme. Economy of mechanism argues for one strong mechanism over four of varying strength [1], and the reason this system has four is that the engines differ, not that four was preferable.

#The test

Row-level security has a specific failure that makes an incorrect implementation pass its own tests: the table owner bypasses the policy, so a test run as owner exercises nothing.

The proposed test connects as a role with no bypass privilege, writes as tenant A, switches to tenant B, and issues the broadest query the layer permits with no predicate. The assertion is that the result is empty, not that it is correctly filtered, because B has written nothing. A non-empty result indicates the boundary is above the engine.

1rows, err := nonSuperuserPool.Query(ctxTenantB, "SELECT id FROM assets")
2require.NoError(t, err)
3require.Empty(t, collect(rows)) // A's rows are invisible, not filtered
Copy

Three details of the test are load-bearing and are the ones most often dropped when it is reimplemented. The connection must use a role without bypass privilege, which is the whole point. Tenant B must have written nothing, so that an empty result is unambiguous; a test where both tenants have data passes if the filtering is merely correct, which is a weaker property than the boundary being below the application. And the query must carry no predicate, since a test that includes one is testing the predicate.

#What the test does not catch

It is a boundary-placement test, not an isolation test, and the distinction matters because passing it reads as more reassuring than it is.

It says nothing about the other three engines. A system whose relational boundary is enforced and whose cache keys are built by hand in one forgotten code path will pass. Each engine needs its own version of the test, and the cache version is the one that cannot be written convincingly, because the property being tested is that no code path constructs a key by hand, which is a statement about the whole codebase rather than about a running system.

It says nothing about a tenant identifier that is established incorrectly. If the request boundary derives the wrong tenant from a valid credential, every layer below enforces the wrong boundary perfectly. The construction places a great deal of weight on that single establishment point, and testing it is a separate exercise about credential validation rather than about storage.

And it says nothing about paths that legitimately need to cross tenants: billing rollups, platform administration, support tooling. Those exist in every real system, they run with elevated privilege, and they are where the boundary is deliberately absent. The value of pushing enforcement into the engine is partly that these paths become enumerable, since they are exactly the code that needs a different role, but enumerable is not the same as safe.

#Related approaches to tenant isolation

The design space is usually presented as a spectrum from shared schema through separate schema to separate database per tenant, with the trade being isolation strength against operational cost per tenant [3]. That framing is about where the data lives. The argument in this note is orthogonal to it: whichever layout is chosen, the enforcement of the boundary should sit in the engine rather than in the application, and a shared-schema deployment with row-level policy is stronger than a separate-schema deployment where the application selects the schema.

The maintenance consequences of multi-tenancy are their own subject, and the observation that tenant awareness spreads through a codebase over time matches what this note argues about predicates [4]. Spreading is the failure mode. A boundary in the engine does not spread, because there is nothing to remember at the call site.

Stepping back, this is an instance of a general point about security mechanisms: the ones that work are the ones that fail closed and cannot be forgotten, and the ones that depend on every developer doing something correctly every time have a failure rate that is a property of the organisation rather than of the design [5].

#Tenancy is not authorisation

Tenancy is not authorisation. Establishing which customer's data a request may reach says nothing about whether a given subject may perform a given action on a given object, which is the question role-based and attribute-based models address [6] [7], and which relationship-based systems answer at scale [8]. Conflating the two produces a function whose guarantee nobody can state.

It also does not address noisy neighbours, per-tenant backup granularity, or data residency, and it does not remove the need for identity controls above it [9].

The reason to keep insisting on this separation is that a single function called something like "canAccess" which folds both questions together has a guarantee nobody can state. Asked what it enforces, the honest answer becomes a description of its implementation. Two functions with two names, one answering which tenant's data is reachable and one answering whether this subject may perform this action, each have a statable guarantee and each can be tested against it.

#Limitations

The argument rests on the claim that leaks are dominated by omitted predicates rather than by flawed policy design. That matches the incidents I have examined and it is not a measured population. A survey of disclosed multi-tenant incidents classified by root cause would settle it, and that survey has not been done here. If the true distribution were dominated by policy design errors instead, the recommendation would change, because moving enforcement into the engine helps with omission and does nothing for a policy that is wrong.

The cache mechanism is materially weaker than the other three, as set out above, and the note does not offer a stronger one short of per-tenant instances.

No cost figures are given. Row-level policy has a query planning cost, per-tenant index provisioning has a per-tenant overhead that becomes significant with many small tenants, and graph partitioning multiplies connection and memory overhead. Each of these is real and none is measured here, so the note argues for a position without pricing it. A deployment with tens of thousands of small tenants may find the per-tenant mechanisms untenable, and for that shape the argument would need revisiting rather than restating.