Embedding Real-Time Rendering Engines in Declarative UI Frameworks

#Abstract

Compositing a real-time rendering engine into a declarative user interface framework involves three independent lifecycles, owned by the operating system, the interface framework and the engine, which no party coordinates. This report characterises the resulting failure modes, proposes an ownership model that resolves them, and reports the cost of running a second renderer as measured on one class of mobile hardware and one engine version.

#Motivation

Applications that require registered three-dimensional content, including augmented reality, product configuration and training simulation, need an engine. Applications that require conventional interface behaviour need the platform's own controls. Building either inside the other produces a known set of deficiencies: interface elements drawn by an engine ignore accessibility settings and platform conventions, while an interface framework cannot perform the tracking and rendering an engine provides [1] [2].

#The three lifecycles

The core difficulty is that three parties may independently invalidate state the others hold.

1. The operating system may destroy and recreate the hosting context at times of its choosing, including configuration change and memory pressure.
2. The interface framework may dispose the view while the hosting context persists, and may rebuild it for reasons unrelated to whether the engine should continue.
3. The engine holds a rendering context, audio focus and possibly a camera, any of which the operating system may reclaim independently.

A correct integration defines behaviour for every combination. The combinations that produce defects are the infrequent ones, which is why the resulting failures are timing dependent and why reproductions are the dominant cost in triage.

#Ownership model

Three rules, each of which corresponds to a decision being placed with the component that owns it [3].

One owner for the engine. A single object holds the engine and is solely responsible for pause, resume and teardown. Views attach and detach. They never own it, because a view is disposed for reasons that do not imply the engine should stop.

No assumption of visibility. Engine code that runs on resume rebuilds anything bound to a rendering surface rather than trusting that its previous surface remains valid.

Explicit state. The integration exposes whether the engine is created, paused or destroyed. This is not a convenience. A defect report that includes the state is worth an order of magnitude more than one reporting that the view went black.

#Measured cost

On mid-range mobile hardware, running a modest scene at 60 frames per second inside a declarative interface:

• Resident memory increased by approximately the engine's own footprint plus scene assets. The two runtimes do not share an allocator, so nothing is pooled between them.
• Cold start increased by roughly one and a half seconds, dominated by engine initialisation rather than by view creation.
• Scrolling a list containing the engine view was visibly worse than the same list without it, because the compositor synchronises two surfaces.

Three design consequences follow: keep the engine surface large and stationary, avoid repeated creation and destruction in favour of pausing, and implement all interface elements in the host framework.

#Deployment evidence

The reference implementation has been used in production for location-based augmented reality at public scale [5] and in a consumer mobile game with substantial engine content [6]. A practitioner write-up covers the build configuration in more detail [4].

#Limitations

The measurements are from one hardware class and one engine version, and both platforms have since changed their compositing mechanisms. The numbers should be read as an order of magnitude rather than a benchmark.

The ownership model does not resolve the case where the host application's configuration determines whether the context is recreated, since that is outside the integration's control. The only available response is documenting supported configurations, which is a mitigation rather than a solution.