Domestic Hot Water Validation Report · Test Matrix
Domestic Hot Water Test Matrix
Every parameter we varied, every gate we scored, every result we published — including the one that fell outside band. Three equipment families, 37 in-scope scenarios, 27 PASS against EnergyPlus 26.1 on annual and monthly energy. The single direct-fired exception is named and attributed rather than excluded from the count.
A note on the downloadable case codes. The stages here are numbered by equipment family, but the case codes inside the workbooks keep the prefixes they were run under — the hydronic-indirect files begin STAGE1_IHWH_. Those strings are the identifiers of the EnergyPlus run directories the results came from, and renaming them would break the link between a published workbook and the run that produced it. We kept the traceable name over the tidy one.
Stage 1 — Direct-fired water heaters (electric / gas / oil)
7 test cases, pass: 6/7
Reference engine: EnergyPlus 26.1.0 — WaterHeater:Mixed with an internal heater, a scheduled draw, and Site:WaterMainsTemperature (Correlation). EnergyPlus runs first and defines BOTH the draw and the mains water temperature, so the two engines are compared on fuel consumption against an identical thermal duty rather than against two separately-authored load assumptions. Scored against ASHRAE Guideline 14-2023.
Parameters varied
| Parameter | Levels | Values |
|---|---|---|
| Fuel type | 3 | Electric, natural gas, No. 2 oil — meter routing rather than physics; EnergyPlus returns byte-identical energy for gas and electricity at equal efficiency |
| Heater configuration | 2 | Storage tank (node-resolved stratified), tankless / instantaneous (algebraic, no tank state) |
| Control mode | 2 | Cycling (constant efficiency, no curve), modulating (part-load efficiency curve) |
| Jacket insulation / standby UA | 3 | Modern R-2.0 jacket, mid-grade jacket, EnergyPlus's own example value — the one term the defect touched, so it is swept rather than fixed |
| Simulation period | 1 | 8,760 hours (full year), EnergyPlus-defined draw and mains temperature |
| Simulation timestep | 4 | 60 / 30 / 20 / 15 min — every case re-run at all four as a control test (see below) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual fuel total | ±5% (ASHRAE G14-2023 annual sum) | 6/7 — worst +6.01% |
| Monthly NMBE | ±5% (ASHRAE G14-2023 monthly) | 6/7 — worst +6.01% |
| Monthly CV(RMSE) | ≤ 15% (ASHRAE G14-2023 monthly) | 7/7 — worst 6.03% |
| Peak hourly fuel | ±10% — REPORTED, NOT GATED | 1/7 — worst +17.18% |
| Hourly CV(RMSE) | ≤ 30% — REPORTED, NOT GATED | 1/7 — worst 98.65% |
| Timestep-sensitivity control | shift must require BOTH a curve AND tank state | Confirmed 7/7 — max |shift| 5.19%, no counter-example |
Stage 2 — Hydronic indirect water heaters (IHWH)
2 test cases, pass: 2/2
Reference engine: EnergyPlus 26.1.0 — WaterHeater:Mixed with its source-side coil on a hot-water PlantLoop fed by DistrictHeating:Water, and Heater Maximum Capacity set to zero so every joule entering the water arrives through the coil. This is EnergyPlus's own 'Indirect Water Heater' topology. The heat source is an ideal district-heating supply rather than a modelled boiler, so the comparison isolates the tank instead of blending a second model's part-load behaviour into the residual. Scored against ASHRAE Guideline 14-2023.
Parameters varied
| Parameter | Levels | Values |
|---|---|---|
| Tank size | 2 | 100 gallon residential, 300 gallon commercial |
| Heat source | 1 | Ideal DistrictHeating:Water on a hot-water PlantLoop, 71.1 °C loop setpoint |
| Internal heater | 1 | Disabled (Heater Maximum Capacity = 0) — all heat arrives through the source-side coil |
| Scored quantity | 1 | Source-side heat transfer, i.e. the demand the tank places on the boiler — exactly what the CogenS indirect model returns to the plant it sits in |
| Simulation period | 1 | 8,760 hours (full year) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual source-side energy | ±5% (ASHRAE G14-2023 annual sum) | 2/2 — −0.22% and −1.71% |
| Monthly NMBE | ±5% (ASHRAE G14-2023 monthly) | 2/2 — worst −1.71% |
| Monthly CV(RMSE) | ≤ 15% (ASHRAE G14-2023 monthly) | 2/2 — 0.29% and 1.85% |
| Peak hourly source demand | ±10% — REPORTED, NOT GATED | 0/2 — −31.35% and −36.39% |
| Hourly CV(RMSE) | ≤ 30% — REPORTED, NOT GATED | 0/2 — 84.22% and 258.93% |
| EnergyPlus reference integrity | 0 Severe, 0 Fatal; tank holds its setpoint | 2/2 — after the coil-flow sizing correction |
Stage 3 — Heat-pump water heaters (HPWH)
28 test cases, pass: 19/28
Reference engine: EnergyPlus 26.1.0 — WaterHeater:HeatPump:WrappedCondenser and :PumpedCondenser coupled to a 12-node WaterHeater:Stratified tank, with the annual stage driven by the DOE / ASHRAE 90.1-2019 Large Hotel prototype's own use-side heat delivery. Published in full as its own report.
Parameters varied
| Parameter | Levels | Values |
|---|---|---|
| Condenser architecture | 2 | Wrapped-condenser residential, pumped-condenser stratified |
| Climate | 4 | Chicago 5A, San Francisco 3C, Tampa 2A, Golden CO semi-arid |
| Evaporator ambient source | 2 | Indoor mechanical room, outdoor weather-file driven |
| Evaporator ambient temperature (Stage 1) | 3 | 10 °C, 20 °C, 30 °C constant |
| Simulation period | 2 | 720 hours constant-draw shakedown, 8,760 hours real weather |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual compressor electricity | ±5% (ASHRAE G14-2023) | Annual stage 15/16; shakedown stage 4/12 |
| Monthly NMBE | ±5% (ASHRAE G14-2023 monthly) | 16/16 on the annual stage |
| Monthly CV(RMSE) | ≤ 15% (ASHRAE G14-2023 monthly) | 16/16 — spanning 1.75% to 9.49% |
| Full breakdown | see the dedicated report | 19/28 in-scope scenarios inside band |
Timestep-sensitivity control test
A falsifiable prediction, tested on all seven direct-fired scenarios at all four supported timesteps against the same EnergyPlus reference. A part-load efficiency curve is evaluated at each step’s part-load ratio, so a coarser step averages the burner’s on-time into a different ratio and charges a different penalty — but only if there is a tank whose on-time can be averaged. Sensitivity should therefore require both a part-load curve and tank state, and any case missing either should be flat. That is a prediction which could have been wrong in three distinguishable ways.
| Scenario | Carries | 60 min | 30 min | 20 min | 15 min | Shift 60 → 15 |
|---|---|---|---|---|---|---|
| GAS_STORAGE_CYCLE_UAR2JACKET | cycling — none | +0.34% | −0.35% | −0.26% | −0.39% | −0.73% |
| GAS_STORAGE_CYCLE_UAMIDGRADE | cycling — none | −0.47% | −0.68% | −0.76% | −0.78% | −0.31% |
| GAS_STORAGE_CYCLE_UAEPEXAMPLE | cycling — none | −0.90% | −0.93% | −0.98% | −1.06% | −0.15% |
| ELECTRIC_STORAGE_CYCLE | cycling — none | +0.34% | −0.35% | −0.26% | −0.39% | −0.73% |
| OIL_STORAGE_CYCLE | cycling — none | +0.34% | −0.35% | −0.26% | −0.39% | −0.73% |
| GAS_STORAGE_MODULATE | curve + tank state | +1.98% | −1.75% | −2.59% | −3.21% | −5.19% |
| GAS_TANKLESS_MODULATE | curve, no tank state | +6.01% | +6.01% | +6.01% | +6.01% | +0.00% |
The prediction holds, with no counter-example. Cycling control carries no curve, so efficiency is constant and the shift is −0.15% to −0.73%. Tankless modulating carries a curve but no tank state — its algebraic branch reads the instantaneous served load, so the part-load ratio is identical at every step and the shift is exactly 0.00%. Storage plus modulating carries both, and is the only material mover at −5.19%. It stays inside the ±5% annual band at every timestep (+1.98% at 60 minutes, −3.21% at 15), so the timestep changes where in the band it sits rather than whether it passes. Every gated figure published here is the 60-minute one, the engine’s default. This is a control test on the model’s own timestep choice; sub-hourly agreement against EnergyPlus is not itself validated, and is listed as a known gap in the report.
Why the three cycling fuels return identical numbers
Electric, natural-gas and oil cycling storage all score +0.34% on annual energy, and the table above shows them moving together across every timestep. That is not a copied row. At equal efficiency these are the same physics with a different meter, and EnergyPlus itself returns byte-identical energy for natural gas and electricity at equal efficiency — fuel type is routing, not thermodynamics. Reporting them as three independent passes would inflate the count, so they are named for what they are: one validated model exercised at three nameplates. The axes that do carry different physics — storage against tankless, cycling against modulating, and three levels of jacket insulation — are the ones that produce different numbers, and they do.