Working principles
Algorithms earn trust by replay, not by reputation.
The leap of faith that lets an algorithm act on customer equipment is paid down by evidence the customer already owns. Replay produces that evidence: the past, as the algorithm would have written it, played against the customer's actual telemetry. The decision to deploy is made against history, not against a vendor's case study.
Purity is the precondition.
Replay only works when the function the algorithm computes is pure: (state, config, now) returning a decision, with no I/O, no hidden state, no implicit dependence on a real clock. Purity is what makes the same code path serve live operation and dry-mode preview without divergence. An impure executor produces a replay that resembles live but is not faithful.
State reconstruction is half the work.
A faithful replay requires the executor to see the same shape of input it sees live. Reconstructing historical state at the cadence the live executor runs (one minute, fifteen minutes, whatever the loop's tick is) is the data-engineering part of replay and often the part that is hardest to get right.
Sub-second tuning is the operator's interaction.
If a replay takes a minute to run, an operator tunes once and accepts the first plausible result. If it runs in under a second, the operator drags sliders, watches the timeline redraw, and converges on a configuration they would have approved in retrospect. The interaction shape is the trust-building mechanism, not just an optimisation.
Replay becomes the first verification's anchor.
The replay result that the operator approved at commissioning is recorded as the predicted savings the first live verification will be measured against. Predicted-versus-actual divergence in month one is one of the cheapest sanity checks on an algorithm whose behaviour just changed.
Replay does not prove the future.
A replay shows what would have happened under historical conditions. It does not capture regime changes (a tenant move-in, a process reformulation, a tariff redesign). Operators read replay as the strongest available pre-commissioning evidence, not as a forecast.
The trust problem
Replay-first is a response to a specific deployment failure mode. The conventional alternatives are slow, expensive, or both. The three patterns below name them.
The standard pilot is two-to-four weeks live
The conventional way to deploy a control algorithm onto customer equipment is to run it live in a non-binding configuration for two to four weeks, then evaluate. The customer pays in time and risk: time waiting for evidence, risk that the algorithm misbehaves on their actual equipment under real conditions. Replay closes the same loop in seconds without sending a single command.
Vendor case studies do not transfer
A case study from another building proves the algorithm performs on that building. It does not prove performance on this building's load shape, this building's occupancy schedule, this building's equipment-class mix. The operator who has to live with the decision wants evidence drawn from their own data.
First-month divergence is too late to catch the mistake cheaply
When a deployment goes wrong because the configuration was off, the cost is one month of suboptimal operation plus the operator's trust budget. Replay catches the same misconfiguration before commissioning at the cost of a few minutes of slider movement.
Purity is the precondition
Replay only works if the function the algorithm computes is pure. The contract below is what an executor has to satisfy. An executor that violates any of the four clauses produces a replay that is not faithful and the operator should not trust it.
Signature
The function the algorithm computes must be (state, config, now) returning a Decision, where state is the snapshot the algorithm would see live, config is the algorithm's tuning parameters, now is the wall-clock the algorithm is supposed to be reading, and Decision is the typed object that names the action. No side channels.
No I/O inside the function
Data lookups (telemetry queries, registry reads, baseline fetches) happen outside the function. The state argument carries everything the function needs. Inside the function, the body is pure computation against the inputs.
No hidden state
Two calls with the same arguments produce the same result. Memory that the algorithm carries between ticks (a vacancy timestamp, a model's hidden state) is part of state and is reconstructed on replay from the prior tick's output.
Clock is passed in
Time-of-day, day-of-week, season are computed from the now argument, not from a system clock. A replay over historical state passes the historical now; the function reads time exactly as it would have read it live.
State reconstruction
A faithful replay requires the executor to see the same shape of input it sees live. This is often the harder half of the engineering, because telemetry has gaps, devices have firmware changes, and memory variables thread sequentially across ticks.
Cadence parity
Replay snapshots are produced at the same cadence the live executor runs. A one-minute loop replays at one-minute resolution. A fifteen-minute loop at fifteen-minute. Skipping cadence steps means the historical decision stream the executor produces will not match the one it would have produced live.
Source of truth
For simulator-backed entities, snapshots are reads from the simulator's state table at each cadence step. For real-device entities, snapshots are reconstructed from the telemetry tables (load_telemetry for power, context_telemetry for environment, generation_telemetry for solar and battery) at the same cadence.
Memory carry-over
Memory variables (a vacancy timer, an exponential moving average, an RL agent's hidden state) are reconstructed by running the executor sequentially over the historical window and threading the prior decision's memory into the next call. This is what makes replay computationally non-trivial: the function is pure, but it is sequential.
Coverage gating
Replay refuses to start when telemetry coverage in the historical window is below the strategy's required threshold (eighty percent for most loops). A replay over a sparse window would mostly be filling in interpolated state and would not give the operator faithful evidence.
The replay flow
The replay engine reads historical telemetry, reconstructs state at the live cadence, runs the same executor the platform runs in production, and produces two artefacts: a hypothetical decision stream and an energy delta against the baseline model.
The tuning loop
The operator's interaction with replay is sliders and toggles, not forms. Each tweak re-runs the executor against cached historical state and redraws the timeline in under a second. Convergence on a configuration the operator would have approved in retrospect takes minutes rather than weeks.
Sub-second redraw
Each tuning interaction (a slider, a checkbox, a config field) triggers a fresh replay against the same historical state. The replay engine caches the state vector once per session and re-runs only the executor, which is pure CPU on a few hundred thousand decision evaluations. Sub-second is achievable for any well-shaped strategy on a thirty-day window.
Side-by-side comparison
The interaction surface lets an operator pin a configuration and tune a second one against it. The platform shows the two replay results overlaid so the operator sees the trade space directly rather than having to remember the prior result.
What gets shown
Energy delta against the baseline model. Comfort or quality excursions. Decision counts. Per-action histograms (how many setbacks, how many restores, how many no-ops). The set of metrics is per-strategy; the registry declares which charts the replay surface shows for a given strategy_type.
What does not get shown
Per-tick decision detail by default. The operator sees aggregates; the underlying decision stream is queryable but the surface refuses to overload it. An operator tuning thirty configurations would otherwise drown in twenty thousand decision rows per attempt.
What replay looks like to the operator
The tuning surface is the principal interaction. The second screen extends the same surface to compare two configurations side by side, which is how operators reason about the trade space between savings and comfort.
fcu_occupancy_optimizer · tuning preview
Replay · last 30 days · what the strategy would have done
rendered in 412msConfiguration · sub-second redraw
range 0 → 60
range 0 → 5
range 16 → 30
range 0 → 1
Replay result
Configuration A · 15 min window vs B · 10 min window
Overlay · day-of-week composite
B saves AUD 4.96 more per month but produces two comfort excursions at the room sensor. The replay shows the excursions occur during partial occupancy where the 10-minute window fires inside an unrecorded coffee-break return. Keep Config A unless the comfort manager approves the trade.
Replay as the verification anchor
The replay result the operator approved at commissioning is captured on the first verification row. The platform reads it back to ground the predicted-versus-actual figure that surfaces on the verification dashboard. The anchor is one of the cheapest ways to catch a misconfigured deployment before the second month of settlement.
Capture at commissioning
When the operator flips the strategy from Disabled to Enabled, the replay's predicted savings (energy delta with confidence band) is captured on the first row of the strategy's verification expectation block. The capture is immutable; subsequent tuning of the live strategy does not retroactively change the anchor.
First verification reads the anchor
The first month's intervention_verifications row carries both the verified savings (what actually happened) and the anchor (what the replay predicted). The ratio is the predicted-versus-actual figure that surfaces on the verification dashboard.
Divergence as a flag, not a failure
A first month that diverges from replay by more than the model's residual band does not block settlement, but it surfaces as a flag for the verifier to inspect. Common causes are named: a tenant move-in changed occupancy, a tariff shifted between commissioning and the first verification, the executor's configuration was tuned after commissioning without re-running the replay.
What replay cannot prove
Replay is the strongest available pre-commissioning evidence. It is not a forecast and it is not a guarantee. The four limits below should be named explicitly to operators before they over-weight a replay result.
Regime change
Replay assumes the historical conditions are representative of the period the strategy will run in. A facility that just took on a new tenant, retooled a process line, or signed a new tariff has a discontinuous history and replay is correspondingly weaker.
Equipment that has aged
A replay against the last thirty days of a chiller that just lost ten percent of its capacity will predict an outcome the rebuilt chiller will outperform. Replay is honest about the past; the future requires updates as the equipment changes.
Algorithms that learn
An online-learning executor (an RL agent that updates from experience) cannot have its future behaviour predicted by replaying its current strategy against history. Replay shows what the current snapshot would have done. The agent's future trajectory is a different artefact.
Replay is not a forecast
Replay shows what would have happened. Forecasting what will happen requires a model of future conditions (weather, occupancy, tariff). The platform forecasts separately; the replay engine deliberately stays inside historical evidence.
Generalisation beyond control policies
The principle is not specific to occupancy setbacks or to building controls. Any algorithm that decides from state, configuration, and clock can be replayed under the same contract. The three angles below cover what generalises.
Beyond control policies
Anything that decides on customer equipment from state, configuration, and clock can be replayed: a tariff-window load shifter, a DR dispatch agent, a demand-response participant, a battery dispatcher, a chiller plant sequencer. The same purity contract applies; the same state reconstruction problem applies; the same tuning surface applies.
Beyond commissioning
Replay is useful at three stages of a strategy's life. At commissioning it earns trust before action. After a tuning change it reproves the change against the same evidence base. After a regime change it lets the operator see whether the strategy still makes sense or needs retuning.
Beyond the operator
Replay artefacts are useful to the verifier (the anchor the first verification measures against) and to the regulator (the evidence base behind a claim). The same artefact serves three readers; the engineering investment to produce it is single-sourced.
Standards alignment
Replay-first commissioning extends established M&V practice with a pre-deployment evidence step. The references below ground the practice; an external review of the replay engine specifically is on the plan after the engine ships in production.
IPMVP Core Concepts (Volume I, 2022). Annex F treats uncertainty, including the predicted-versus-actual evidence base that replay-first commissioning extends.
Efficiency Valuation OrganizationASHRAE Guideline 14-2014: Measurement of Energy, Demand, and Water Savings. Informs the baseline model the replay engine uses to translate decision streams into energy deltas.
ASHRAESutton and Barto, Reinforcement Learning: An Introduction (2018). The challenge of evaluating a policy without deploying it is the off-policy evaluation problem; replay-first is one engineered solution for the building-controls domain.
MIT Press