Heat-Pump Boiler Validation Report · Test Matrix
Heat-Pump Boiler Test Matrix
Every parameter we varied, every gate we scored, every result we published. Six stages, 587 scenarios, 5,407 individually scored gates, 587 of 587 PASS against EnergyPlus 26.1 and published manufacturer operating envelopes.
Stage 1 — Core heat-pump model, air-source and water-source
336 test cases, all passing · 2,688 gates scored
Reference engine: EnergyPlus 26.1.0 HeatPump:PlantLoop:EIR:Heating on an isolated PlantLoop driven by LoadProfile:Plant. Air-source cases use the AirSource condenser reading outdoor dry-bulb; water-source cases add a PlantComponent:TemperatureSource loop. Both engines read the identical capacity, efficiency and part-load curve coefficients, so any deviation is a difference in how the two models apply them rather than in the data they were given. Scored against ASHRAE Guideline 14-2023.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Source type | 2 | Outdoor air (air-to-water); water or waste-heat stream (water-to-water) |
| Leaving-water setpoint | 4 | 35 °C, 45 °C, 55 °C, 65 °C |
| Source temperature | 8 | −20, −15, −10, −5, 0, +5, +10, +25 °C |
| Part-load ratio | 4 | Four load levels from light part-load to full output on each machine size |
| Compressor control | 3 | Variable speed (no cycling penalty), staged (Cd = 0.15), single speed (Cd = 0.25, AHRI 210/240 default for untested equipment) |
| Machine size | 2 | 80 kW, 400 kW rated heating capacity |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Total heat delivered | ± 5% (ASHRAE Guideline 14-2023) | +0.01 to +0.12% across all 336 scenarios |
| Total electricity consumed | ± 5% | +0.01 to +0.12% |
| Peak heat and peak electricity | ± 10% | worst 0.12% |
| Hourly NMBE on heat and electricity | ± 10% | worst 0.13% |
| Hourly CV(RMSE) on heat and electricity | ≤ 30% | worst 0.12% |
Stage 1B — Full-year 8,760-hour run on real weather
8 test cases, all passing · 112 gates scored
Reference engine: EnergyPlus 26.1.0 running a two-module heat-pump plant with a natural-gas auxiliary in series for a full year on TMY3 weather, with the hourly heating load delivered to both engines through Schedule:File from the identical 8,760-row CSV. The plant is sized to each climate's peak load, and the hydronic loop to an 8 °C design temperature difference at that peak, exactly as a real installation would be. No climate is excluded. Scored against ASHRAE Guideline 14-2023 at annual, peak and hourly resolution.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Climate (TMY3) | 4 | Chicago O'Hare (217.6 MWh annual heating, 1,788 h below freezing); Golden, Colorado (208.8 MWh, cold and dry at altitude); San Francisco (67.2 MWh, mild marine); Tampa (17.9 MWh, the low-load extreme) |
| Leaving-water setpoint | 2 | 45 °C, 55 °C |
| Bank | 1 per climate | Two modules, sized to 1.08 × the climate's peak load and rounded to 5 kW — 2 × 25 kW (San Francisco) through 2 × 80 kW (Chicago, Golden) |
| Hydronic loop flow | 1 per climate | Sized for an 8 °C design temperature difference at the climate's peak, 0.00146 to 0.00449 m³/s |
| Auxiliary | 1 | Natural-gas boiler in series downstream, 85% thermal efficiency |
| Simulation period | 1 | 8,760 hours, hourly timestep, every hour scored |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Annual heat delivered by the heat pump | ± 5% | +0.07 to +0.92% |
| Annual heat-pump electricity | ± 5% | −0.85 to +0.47% |
| Share of annual load carried by the heat pump | ± 5% | −0.08 to −0.01% — the two engines agree on the heat-pump-versus-gas split to under a tenth of a percent |
| Annual auxiliary heat and fuel | absolute-energy gate at 1% of the annual heating load (trivial-energy regime, named on the gate) | worst 0.10% of annual load |
| Peak heat-pump heat and electricity | ± 10% | worst 3.44% |
| Hourly NMBE on delivered heat | ± 10% | worst 0.94% |
| Hourly CV(RMSE) on delivered heat | ≤ 30% | worst 14.57% |
Stage 2 — Multi-module bank staging
96 test cases, all passing · 768 gates scored
Reference engine: EnergyPlus 26.1.0 multi-module plant: N HeatPump:PlantLoop:EIR:Heating machines on parallel branches with splitter, mixer and bypass, dispatched by PlantEquipmentOperation:HeatingLoad under Load control. Delivered heat is scored strictly against ASHRAE Guideline 14-2023; bank electricity is scored under the federal FEMP Measurement and Verification band for the documented load-sharing difference between the two engines' control laws.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Modules in the bank | 3 | 2, 3, 4 identical modules |
| Bank capacity | 3 | 160 kW, 240 kW, 320 kW |
| Load as a fraction of bank capacity | 4 | 20%, 45%, 70%, 95% |
| Source temperature | 4 | −15, −5, +5, +15 °C |
| Compressor control | 2 | Variable speed, single speed |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Bank heat delivered | ± 5% | +0.04 to +0.19% — the two engines stage the same number of modules |
| Bank electricity | ± 15% (FEMP Measurement and Verification, applied for the documented load-sharing difference) | −4.20 to +9.71% |
| Peak bank heat | ± 10% | worst 0.19% |
| Peak bank electricity | ± 15% | worst 9.71% |
| Hourly NMBE on bank heat | ± 10% | worst 0.20% |
| Hourly CV(RMSE) on bank heat | ≤ 30% | worst 0.19% |
Stage 3 — Operating envelope against published vendor data
9 test cases, all passing · 21 gates scored
Reference engine: Published manufacturer operating envelopes and datasheets read directly from the engineering catalogs and submittals of eight vendors: Trane (SYS-APG003B-EN operating envelope), Mitsubishi (QAHV databook), Nyle (e360 submittal SB-e360-103123L), Mayekawa (unimo AW datasheet), Aermec (NSH correction curves), Swegon (OMICRON Zero envelope chart), Ochsner (IWWDS waste-heat) and H.Stars (S31 engineering catalog). EnergyPlus is not the reference here because its maximum-supply-temperature curve field was found across eight separate plant configurations not to restrict output in the released version.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Vendor machines | 8 | Trane ACX 140-230 Ton, Mitsubishi QAHV-N560YA-HPB, Nyle e360, Mayekawa unimo AW, Aermec NSH 1251-3602, Swegon OMICRON Zero S4, Ochsner IWWDS 430 R4b, H.Stars S31 90C Water-Source Series |
| Published envelope points scored | 18 | Source temperatures from −25 °C to +55 °C, maximum leaving water from 48 °C to 130 °C |
| Envelope forms | 2 | Linear derate with source temperature (air-source subcritical); flat hold to the envelope floor (transcritical carbon dioxide and waste-heat machines) |
| Refrigerant classes covered | 4 | Legacy halocarbon air-source, R290 propane air-source, R744 transcritical carbon dioxide, water and waste-heat source |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Maximum leaving water temperature at each published source temperature | ± 1.0 °C | 0.00 °C across all 18 published points |
| The envelope caps leaving water below setpoint when it should | the cap must engage | engages correctly |
| Auxiliary covers the shortfall when the envelope caps supply | energy balance closes to 0.1% | closes exactly |
| A fixed-temperature model understates auxiliary demand | the difference must be demonstrable | demonstrated |
Stage 4 — Defrost
96 test cases, all passing · 1,272 gates scored
Reference engine: EnergyPlus 26.1.0 HeatPump:PlantLoop:EIR:Heating with TimedEmpirical defrost control, scored against EnergyPlus's dedicated defrost meters. EnergyPlus reports defrost electricity and defrost heat load separately from compressor electricity, so each term is scored on its own rather than inferred from a total that could be right for the wrong reasons.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Source temperature | 8 | −20, −15, −10, −5, 0, +5, +10, +15 °C — through and below the frosting band, and above it as a control |
| Leaving-water setpoint | 2 | 45 °C, 60 °C |
| Machine size | 2 | 80 kW, 400 kW |
| Compressor control | 3 | Variable speed, staged, single speed |
| Load level | 2 | Part-load and near-full-load, so the part-load coupling of the defrost term is exercised |
| Scenarios inside the frosting band | 84 of 96 | The 12 scenarios above the defrost cut-off temperature are scored on the non-defrost gates only, because there is no defrost term to compare |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Defrost electricity | ± 5% | +0.02 to +0.10% across the 84 scenarios inside the frosting band |
| Defrost heat load taken back from the water | ± 5% | +0.02 to +0.10% |
| Total electricity including defrost | ± 5% | −0.19 to +0.10% |
| Total heat delivered | ± 5% | worst 0.10% |
| Peak defrost electricity and heat load | ± 10% | worst 0.10% |
| Hourly NMBE on defrost electricity and heat load | ± 10% | worst 0.10% |
| Hourly NMBE and CV(RMSE) on total electricity | ± 10% NMBE, ≤ 30% CV(RMSE) | worst 0.20% NMBE |
Stage 5 — Auxiliary heater coupling
42 test cases, all passing · 546 gates scored
Reference engine: EnergyPlus 26.1.0 HeatPump:PlantLoop:EIR:Heating and Boiler:HotWater on the same plant loop under SequentialLoad operation, so the heat pump is dispatched first and the gas boiler covers the remainder. This is how a real heat-pump boiler plant is piped, and it is what makes the two heat streams sum to the load rather than stack on top of it. Scored against ASHRAE Guideline 14-2023.
Parameter sweeps
| Parameter | Levels | Values |
|---|---|---|
| Source temperature | 7 | −20, −15, −10, −5, 0, +5, +10 °C — spanning the range where the handoff moves from capacity-bound to temperature-bound |
| Load | 3 | 60 kW, 90 kW, 120 kW |
| Leaving-water setpoint | 2 | 45 °C, 55 °C — the higher setpoint pushes the envelope limit into play at warmer ambients |
| Auxiliary | 1 | Natural-gas boiler in series downstream, 85% thermal efficiency |
| Shortfall modes exercised | 2 | Capacity-bound (the heat pump cannot make enough kilowatts) and temperature-bound (it has capacity but the envelope caps how hot it can send the water) |
Gates scored on every case
| Gate | Tolerance | Result |
|---|---|---|
| Heat delivered by the heat pump | ± 5% | −0.00 to +0.07% |
| Heat delivered by the auxiliary | ± 5% | +0.00 to +4.96% |
| Share of the load carried by the heat pump | ± 5% | worst 0.07% — the handoff point matches |
| Auxiliary fuel | ± 5% | worst 4.96% |
| Peak auxiliary heat and fuel | ± 10% | worst 4.96% |
| Peak heat-pump heat and electricity | ± 10% | worst 4.41% |
| Hourly NMBE on every stream | ± 10% | worst 5.22% |