Boiler Validation Report · Test Matrix
Boiler Test Matrix
Every parameter we varied, every gate we scored, every result we published. Eight stages, 280 in-scope scenarios, 278 PASS strict ASHRAE Guideline 14-2023 vs EnergyPlus.
Stage 1 — Core boiler model per equipment type
189 test cases, all pass: 187/189
Reference engine: EnergyPlus 26.1.0 Boiler:HotWater isolated topology. Single boiler on a PlantLoop driven by LoadProfile:Plant; Curve:Cubic for non-condensing / electric (CogenS FuelPLC Quadratic / Cubic mapped to PLF cubic); Curve:Cubic-from-BiQuadratic for condensing (PLF sampled at the scenario's design return temperature). Scored against ASHRAE Guideline 14-2023, with documented relaxed bands on Condensing (BiQuadratic 2D→1D collapse) and sub-minimum cycling (PLR ≤ 0.10 sinusoidal) regimes.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Equipment class | 3 | Non-condensing hot-water gas (Quadratic FuelPLC), electric hot-water (Quadratic FuelPLC), condensing hot-water gas (BiQuadratic FuelPLC) |
| Part-load ratio | 5 | 10%, 25%, 50%, 75%, 100% of nameplate |
| Nominal capacity | 3 | 100 kW, 500 kW, 2,000 kW |
| Hot-water supply temperature | 3 | 60 °C, 82 °C, 95 °C |
| Simulation period | 2 | 168-hour steady (1WEEK_STEADY), 720-hour sinusoidal variable load (1MONTH_VARIABLE) |
| Scope-out filters | 2 | PLR ≥ 100% + sinusoidal (peaks 125% capacity → engine-convention divergence); condensing at supply ≥ 82 °C (off-design — boiler can't condense at the corresponding return temps) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual fuel energy | ± 5% strict / ± 12% condensing / ± 15% sub-min cycling (ASHRAE G14-2023) | −1.32 to +2.13% strict (144/144 PASS); −6.86 to +11.09% condensing (27/27 PASS); +0.06 to +16.59% sub-min cycling (16/18 PASS, 2 documented FAIL at CAP100/PLR10/T82C) |
| NMBE on hourly fuel use | ± 10% strict / ± 15% relaxed regimes | Same ranges as annual fuel energy |
| CV(RMSE) on hourly fuel use | ≤ 30% strict / ≤ 40% sub-min cycling | +2.39 to +26.73% across the 4 sampled audit-trace cases — all within band; per-case CV(RMSE) is live-formula on the Comparison + Tolerances sheet |
| Annual thermal output delivered | ± 5% strict / ± 15% sub-min cycling (no condensing relaxation, separate gate) | +0.00 to +1.79% strict; +0.00 to +10.58% condensing; +2.18 to +18.69% sub-min cycling (the 2 FAIL cases hit +18.69% — fails this gate AND the fuel-energy gate) |
| Peak hour fuel rate | ± 10% strict / ± 15% condensing / ± 20% sub-min cycling | Sampled audit cases: −0.08% (condensing canonical) / +0.84% (sub-min electric) / +1.16% (sub-min noncond 100 kW) / +1.56% (strict canonical) — all comfortably within band |
| Efficiency profile NMBE + CV(RMSE) | ± 10% NMBE / ≤ 30% CV(RMSE) strict; ± 15% / ≤ 40% sub-min | Strict canonical −0.79% NMBE / +0.81% CV(RMSE); condensing canonical +0.01% / +0.23%; sub-min canonicals +2.00 to +6.20% NMBE / +2.13 to +21.56% CV(RMSE) |
Stage 1B — Full-year 8,760-hour realistic-load
3 test cases, all passing
Reference engine: EnergyPlus 26.1.0 Boiler:HotWater isolated topology driven by a Schedule:File reading the same 8,760-row hourly CSV CogenS consumes via load_profile_kw. Full annual ASHRAE Guideline 14-2023 scoring at annual / monthly (12-bucket) / hourly (8,760-row) resolution. Condensing equipment scored under the relaxed ± 12% annual / ± 15% peak / ± 12% monthly NMBE / ± 15% hourly NMBE band documented on the Cover sheet (BiQuadratic FuelPLC 2D → 1D collapse rationale).
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Equipment class | 3 | Non-condensing gas (Quadratic FuelPLC) at 82 °C, electric (Quadratic FuelPLC) at 82 °C, condensing gas (BiQuadratic FuelPLC) at 60 °C (in-design low-temp regime where the boiler condenses) |
| Nominal capacity | 1 | 500 kW (every case) |
| Load profile | 1 | 8,760 hours from Chicago O'Hare TMY3: heating-degree-hours envelope + DHW diurnal (morning + evening peaks) + weekday/weekend modulation, clipped to PLR [0.20, 0.95] |
| Run period | 1 | Full year (1/1 – 12/31) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual fuel energy | ± 5% strict / ± 12% condensing (ASHRAE G14-2023) | +1.25% (NONCOND), +1.25% (ELECTRIC), +0.93% (COND) — all PASS |
| Annual thermal output | ± 5% strict / ± 12% condensing | +1.06% (NONCOND), +1.06% (ELECTRIC), +1.01% (COND) — all PASS |
| Peak hour fuel rate | ± 10% strict / ± 15% condensing | −0.09% (NONCOND), −0.18% (ELECTRIC), −1.64% (COND) — all PASS |
| Monthly NMBE on fuel use | ± 5% strict / ± 12% condensing | +1.25% (NONCOND), +1.25% (ELECTRIC), +0.93% (COND) — all PASS |
| Monthly CV(RMSE) | ≤ 15% strict / ≤ 20% condensing | 1.31% (NONCOND), 1.31% (ELECTRIC), 0.94% (COND) — all PASS |
| Hourly NMBE on fuel use (8,760 h) | ± 10% strict / ± 15% condensing | +1.25% (NONCOND), +1.25% (ELECTRIC), +0.93% (COND) — all PASS |
| Hourly CV(RMSE) | ≤ 30% (both bands) | 1.45% (NONCOND), 1.48% (ELECTRIC), 1.04% (COND) — all PASS |
Stage 2 — Multi-unit identical staging
44 test cases, all passing
Reference engine: EnergyPlus 26.1.0 multi-unit Boiler:HotWater plant: N identical boilers on one PlantLoop, EP's 'Optimal' load-distribution scheme (loads each active unit to maximum PLR before adding the next). CogenS uses a band-fit staging rule that keeps operating units inside the optimum PLR range [0.44, 0.84]. Both are valid staging philosophies; this stage scores energy agreement under ASHRAE Guideline 14-2023 plus a units-operating discrepancy gate at ± 1 unit / hour from the CogenS validation plan.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Equipment class | 3 | Non-condensing gas (Quadratic FuelPLC) at 82 °C, electric (Quadratic FuelPLC) at 82 °C, condensing gas (BiQuadratic FuelPLC) at 60 °C (in-design) |
| Unit count (N identical boilers) | 3 | N = 2, N = 3, N = 4 |
| System part-load ratio | 5 | 10%, 25%, 50%, 75%, 100% of plant nameplate (N × 500 kW) |
| Per-unit capacity | 1 | 500 kW (every case) |
| Hot-water supply temperature | 1 per class | 82 °C non-cond / electric; 60 °C condensing (in-design) |
| Simulation period | 1 | 168-hour steady (1WEEK_STEADY) — staging signal isolated from sub-min cycling which is a Stage 1 concern |
| Documented scope exclusion | 1 | NONCOND N = 4 at PLR = 0.10 (200 kW load on 2 MW installed) — sub-minimum cycling regime not validatable head-to-head; documented at the Stage 5 gate |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual fuel energy | ± 5% strict / ± 12% condensing (ASHRAE G14-2023) | Strict regime (NONCOND + ELECTRIC, n = 29): −0.36% to +4.98%; Condensing (n = 15): −6.86% to +5.00% — all 44 PASS |
| Annual thermal output delivered | ± 5% strict / ± 12% condensing | Strict +0.63% to +4.45%; condensing +0.53% to +4.44% — all PASS |
| Peak hour fuel rate | ± 10% strict / ± 15% condensing | Tracks annual on steady-load matrix (per-hour constant); served case +0.76%, condensing canonical −1.23% |
| NMBE on hourly fuel use | ± 10% strict / ± 15% condensing | Equals annual deviation on steady-load matrix |
| CV(RMSE) on hourly fuel use | ≤ 30% strict (≤ 30% condensing) | Within band (steady-load matrix; CV(RMSE) = annual % deviation for constant-load cases) |
| Units-operating max discrepancy | ≤ 1 unit / hour (CogenS validation plan) | 0 units in 32 / 44 cases (exact match); 1 unit in 12 / 44 cases — the documented staging-philosophy difference (CogenS band-fit vs EP 'Optimal') stays within tolerance across the full matrix |
Stage 3 — Multi-size (2-type) dispatch
10 test cases, all passing
Reference engine: EnergyPlus 26.1.0 multi-size Boiler:HotWater plant: 2 boilers at size_1 capacity + 1 boiler at size_2 capacity on a single PlantLoop. EP's 'Optimal' load-distribution scheme picks the smallest available active boiler first and loads it to maximum PLR before adding the next. CogenS' min-fuel proportional dispatch loads all sizes simultaneously at proportional PLRs. Total system fuel scored against ASHRAE Guideline 14-2023 (the gating decision); per-size fuel split is reported as a dispatch-philosophy diagnostic only, mirroring the Stage 2 units-operating treatment.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Configuration | 2 | NONCOND_GAS_MECH 2×500 kW + 1×1000 kW (Quadratic FuelPLC), ELECTRIC 2×500 kW + 1×1000 kW (Quadratic FuelPLC). Condensing scoped out to isolate the 2-size dispatch signal from the BiQuadratic mapping noise characterized in Stage 1. |
| System part-load ratio | 5 | 20%, 40%, 60%, 80%, 100% of total installed capacity (2 MW per plant) |
| Per-unit capacities | 2 | size_1 = 500 kW (×2 units), size_2 = 1,000 kW (×1 unit) |
| Hot-water supply temperature | 1 | 82 °C (every case — Stage 1 already characterizes the temperature sweep) |
| Simulation period | 1 | 168-hour steady (1WEEK_STEADY) — dispatch signal isolated from sub-min cycling |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Total system fuel energy | ± 5% (ASHRAE Guideline 14-2023) | −0.68% to +2.05% across all 10 cases — all PASS |
| Total system thermal output | ± 5% | +0.63% to +2.20% — all PASS |
| Peak system fuel rate | ± 10% | Tracks annual on steady-load matrix; served case −0.33% |
| Hourly NMBE on system fuel | ± 10% | Equals annual deviation (steady-load matrix) |
| Hourly CV(RMSE) | ≤ 30% | Within band (steady-load matrix; CV(RMSE) = annual % deviation for constant-load cases) |
| Per-size fuel split (diagnostic, NOT a gate) | documented dispatch-philosophy difference | At low system PLR (0.20–0.40), CogenS picks size_2 (1×1000 kW) while EP picks size_1 (2×500 kW); at higher PLRs both engines run all sizes but split fuel differently. Per-size split deviation up to ±234% on the diagnostic gate at PLR=0.60, while total system fuel stays within ±5% strict — bankable TEA depends on total fuel, not the per-size split. Documented Tier 2 in the Cover sheet. |
Stage 4 — Hot-water thermal energy storage (standby decay)
6 test cases, all passing
Reference engine: EnergyPlus 26.1.0 WaterHeater:Stratified isolated standby tank — heaters disabled, matched geometry (cylindrical tank, height = 4 × radius, 5 axial nodes), matched U-value, matched ambient (20 °C), initial uniform 90 °C, 168-hour decay. Tank-average temperature trajectory scored against ASHRAE Guideline 14-2023 hourly NMBE ± 10% / CV(RMSE) ≤ 30%. RE-SCORED July 2026 against the current stratified-tank engine; the previous engine’s figures are quoted alongside each gate. The shell-loss column was previously reported here as a ~20% surface-area convention difference with EnergyPlus. It was not a convention difference but an under-count: the engine integrated the cylindrical side wall and not the two end caps, which for this geometry (height = 4 × radius) is 8/10 of the surface. The current engine integrates the full surface. Stage 4b (separate row) covers the charge / discharge cycling regime and has NOT been re-scored.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Tank volume | 2 | 10 m³, 50 m³ |
| U-value (insulation envelope) | 3 | 0.3, 0.5, 1.0 W/m²K |
| Number of stratification nodes | 1 | 5 (axially stratified) |
| Initial uniform temperature | 1 | 90 °C (fully charged tank) |
| Ambient temperature | 1 | 20 °C (constant) |
| Simulation period | 1 | 168-hour standby decay (heaters disabled, no charge/discharge) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Tank temperature trajectory NMBE | ± 10% (ASHRAE Guideline 14-2023) | −1.02% to −0.22% across all 6 cases — all PASS (was +0.34% to +1.86%) |
| Tank temperature trajectory CV(RMSE) | ≤ 30% | 0.25% to 1.10% — all PASS (was 0.40% to 2.11%) |
| Total tank temperature drop (168 h) | — | CogenS 4.91–23.46 °C vs EP 4.58–22.33 °C. The previous engine read 3.97–19.62 °C — it under-decayed because it was losing heat through the side wall only. |
| Cumulative shell-loss energy (DIAGNOSTIC, not a gate) | ± 2% of the EnergyPlus shell loss | −1.3% to −0.3% across all 6 cases. Previously −17.6% to −19.6%, reported as a surface-area convention difference; it was an under-count of the two end caps, now corrected. |
Stage 4b — Hot-water TES charge / discharge cycling
4 test cases, all passing
Reference engine: EnergyPlus 26.1.0 WaterHeater:Stratified TES with phased charge / discharge cycling — source side is an indirect boiler coil whose heater-1 setpoint schedule gates charge (high setpoint during the night window so the source feeds the tank) versus discharge (low setpoint during the day window so the source stops while the use-side LoadProfile:Plant draws). CogenS receives the matching signed charge/discharge schedule. Tank temperature trajectory + cumulative discharge energy are the gating metrics; charge energy and shell-loss are documented diagnostics carrying the Stage 4 sidewall-vs-full-surface convention.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Tank volume | 2 | 10 m³, 50 m³ |
| Cycling depth | 2 | DEEP (2.2 kW/m³ draw, deep state-of-charge swing), MOD (1.2 kW/m³ draw, moderate SoC swing) |
| Charge window | 1 | 8 hours nightly (heater setpoint 82 °C) |
| Discharge window | 1 | 12 hours daily (heater setpoint 30 °C, source idle, LoadProfile:Plant draws) |
| Idle window | 1 | 4 hours (setpoint low, no draw — tank coasts at shell-loss only) |
| Simulation period | 1 | 168-hour 1-week cycle (7 daily cycles) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Tank temperature trajectory NMBE | ± 10% (ASHRAE Guideline 14-2023) | −4.75% to −5.13% across all 4 cases — all PASS |
| Tank temperature trajectory CV(RMSE) | ≤ 30% | 5.17% to 5.57% — all PASS |
| Cumulative discharge energy (useful round-trip throughput) | ± 5% | +0.6% on all 4 cases (reproduces the prior published claim exactly) — all PASS |
| Tank cycling range | — | CogenS 57–82 °C vs EP 58–85 °C across the matrix (both engines actively cycling) |
| Charge energy (DIAGNOSTIC, not a gate) | carries Stage 4 shell-loss surface-area convention | +1.2% to +26.7% (charge = discharge + shell losses; the +26.7% on the MOD-depth cases is the shell-loss share dominating the smaller charge total) |
| Shell-loss energy (DIAGNOSTIC, not a gate) | Stage 4 sidewall-vs-full-surface convention | −25.0% to −25.6% (within the documented Stage 4 ratio for the cylindrical geometry) |
Stage 5 — Boiler + storage coupling (setpoint-buffer)
12 test cases, all passing
Reference engine: EnergyPlus 26.1.0 coupled Boiler:HotWater + WaterHeater:Stratified plant: two PlantLoops with a Boiler:HotWater feeding the heater side of the tank under IndirectHeatPrimarySetpoint control, and a LoadProfile:Plant drawing the tank-use side. CogenS' setpoint-buffer harness composes the Stage 1 boiler engine + Stage 4 TES engine with matched parameters. Scored against ASHRAE Guideline 14-2023 on boiler fuel + thermal (annual) plus tank-temperature trajectory (hourly NMBE / CV(RMSE)). RE-SCORED July 2026 with the storage engine swapped and the boiler engine held fixed, so the movement is attributable to the tank rather than to two things at once; re-running with the previous storage engine reproduces all four published gate ranges exactly. The Electric variant adds a financial layer on a served annual canonical: 20-year time-of-use cost of ownership with and without storage, and the storage-incremental NPV, gated against an analytical NIST HB135 closed-form present value under an off-peak $0.05 / on-peak $0.18 per kWh schedule. Internal rate of return and payback are reported but NOT gated, and on this canonical they are undefined because the storage NPV is negative.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Equipment class | 2 | Non-condensing gas (Quadratic FuelPLC), electric (Quadratic FuelPLC) |
| Tank volume | 2 | 10 m³, 50 m³ |
| System part-load ratio | 3 | 30%, 50%, 70% of boiler nameplate (system load fraction) |
| Boiler capacity | 1 | 500 kW (every case) |
| Tank setpoint / supply | 1 | 82 °C (boiler holds tank at setpoint; load draws to ~71 °C return) |
| Simulation period | 1 | 168-hour steady (1WEEK_STEADY) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Boiler annual fuel energy | ± 5% (ASHRAE Guideline 14-2023) | +1.48% to +2.95% across all 12 cases — all PASS (was +1.43% to +2.76%; the boiler now supplies the tank’s end-cap losses too) |
| Boiler annual thermal output | ± 5% | +1.44% to +2.49% — all PASS (was +1.39% to +2.27%) |
| Boiler peak fuel rate | ± 10% | Tracks annual on the steady-load matrix; +0.14% on the served annual canonical (was +0.11%) |
| Tank temperature trajectory NMBE | ± 10% (Guideline 14 hourly) | +0.48% to +1.64% — all PASS (was +0.50% to +1.65%) |
| Tank temperature trajectory CV(RMSE) | ≤ 30% | 0.48% to 1.64% — all PASS (was 0.50% to 1.65%) |
| Delivered load (DIAGNOSTIC) | — | +0.00% on all 12 cases (both engines deliver the exact requested load — the setpoint-buffer harness conserves energy) |
| Shell-loss diagnostic | ± 5% of the EnergyPlus shell loss | +0.9% to +2.9% across all 12 cases (was −19.2% to −17.6%; the end-cap under-count is corrected) |
| 20-year TCO with storage (served annual canonical) | ± 5% vs analytical NIST HB135 PV | +1.838% — PASS (was +1.771%, published as +1.78%) |
| 20-year TCO without storage (served annual canonical) | ± 5% vs analytical NIST HB135 PV | 0.000% — PASS. The no-storage case does not involve the storage engine, so it is unchanged by construction. |
| Storage-incremental NPV, absolute agreement | within $100k absolute | $66,930 apart — PASS (was $64,483). CogenS reports −$54,704 and the analytical reference +$12,226. |
| Storage feasibility verdict (DIAGNOSTIC, not a gate) | NPV-sign agreement; reported, not gated | The two engines DISAGREE on this canonical: CogenS makes the storage marginally unattractive, the analytical reference marginally attractive. Both NPVs sit near zero, and the setpoint-buffer control validated here does not shift load to off-peak hours, so the storage has little financial value in this regime by design. The same disagreement is present on the previous engine. |
Stage 6 — Total Cost of Ownership and financial KPIs
12 test cases, all passing
Reference engine: EnergyPlus 26.1 LifeCycleCost (NIST Handbook 135 reference implementation) for the operating present value gate; independent polynomial-root IRR solver and closed-form analytical NPV / payback recompute for the financial KPI gates; IRS Publication 946 MACRS GDS depreciation tables for the tax-on cases. All gates score CogenS' production financial engines against three independent reference implementations on the same scenario inputs.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Cost basis (boiler + TES sizing) | 2 | SMALL: 500 kW boiler + 10 m³ tank; LARGE: 2 MW boiler + 50 m³ tank |
| Financial profile | 6 | D07_I02_N20 (7% discount, 2% inflation, 20 yr); D05_I02_N20 (5% discount, isolates discounting); D10_I02_N20 (10% discount); D07_I00_N20 (0% inflation, real-discount only); D07_I03_N25 (3% inflation, 25 yr); D07_I02_N20_TAXMACRS (base + MACRS depreciation enabled) |
| Total scenarios | 12 | 2 bases × 6 profiles = 12 cases (2 of which are tax-on with MACRS active) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Operating present value vs EnergyPlus | ± 0.5% | +0.0000% / −0.0000% across all 12 cases (exact to 4 dp) |
| NPV vs analytical recompute | ± 0.5% | +0.0000% / +0.0000% across all 12 cases (exact) |
| IRR vs polynomial-root solver | ± 0.05 percentage points | ± 0.0000 pp across all 12 cases (exact) |
| Simple payback vs analytical | ± 0.5% | +0.0000% across all 12 cases (exact) |
| MACRS depreciation schedule vs IRS Publication 946 GDS | ≤ $1 max absolute | $0.00 max absolute on the 2 tax-on cases (exact match against published GDS tables) |
| TCO (CAPEX + operating PV) — diagnostic | — | +0.0000% across all 12 cases (CAPEX is an exact t=0 input; TCO % deviation tracks the operating PV deviation by construction) |