Hardware-in-the-loop verification of a safety-certified ignition controller
I owned the verification suite and the automated bench for a two-stage gas furnace ignition controller certified to UL/CSA, ANSI Z21.20 and NFPA 54. This is how the bench works, how a single test decides pass or fail, and why a green result on it means something.
The client's firmware, state table and code are confidential. Names below are generic, code is illustrative, and client measurements are left out; the claims are the ones on my CV.
The problem
A furnace ignition controller is a safety control. It sequences the inducer, pressure switches, hot-surface igniter, gas valves and flame sensing, and it has to fail safe on every fault. The firmware is a table-driven state machine, and every transition has to be demonstrated on real hardware with evidence a certification reviewer can follow.
What I started with: hand-written, duplicated tests; a whole family of cases where the stimulus window was never actually measured; and bench failures nobody could attribute to firmware, harness or instrument.
Ignition sequence, simplified
- 01 Idle No heat demand
- 02 Pressure-switch proving Inducer starts; switches must change state
- 03 Prepurge Clear the combustion chamber
- 04 Igniter warm-up Hot-surface igniter heats
- 05 Trial for ignition Gas valve opens; flame must be proven in seconds
- 06 Flame stabilisation Hold before declaring run
- 07 Run Low or high fire
- 08 Post-purge Inducer clears, back to idle
Every phase also has fault exits (limits, rollout, pressure switches, flame and gas-valve faults, supply disturbances) into post-purge, soft lockout, hard lockout or wait-for-recovery. Those exits are most of the transitions in the catalogue.
The bench
Each instrument is owned by a single thread and stamped on one monotonic clock, so a command sent to the AC source, a relay edge and a state change reported by the controller can be put on one axis. That timeline is what made root-cause analysis across three independent devices possible.
One test: arrange → act → assert
Drive the controller into the source state through a data-driven navigation chain. Then ask the controller which state it is actually in, which catches the silent misroute where navigation reported success but the firmware had parked somewhere else.
Apply the stimulus: a relay, a flame change or an AC-source excursion. Every observation the test expects is armed before the stimulus starts.
Eight independent gates run on the captured evidence. They all run even after one fails, so a single bench run gives the whole diagnostic picture.
Some source states last only a few seconds, and the gap between confirming arrival and firing the stimulus used up most of that window. For those transitions the stimulus is handed to the navigation step and fires the moment the state is confirmed. For transient source states the post-arrange check is a short probe, so the check itself doesn't use up the window.
The eight gates
- 01 Preconditions
Inputs, relays and active alarms are as required and the source state was validly reached before anything is stimulated.
- 02 Stimulus timing
The stimulus was applied and the response landed inside its not-before / not-after window.
- 03 State sequence
The controller reports the expected target state, confirmed by a live poll and by the captured message log.
- 04 Alarms
The expected fault codes are raised and cleared, and each alarm name matches the catalogue.
- 05 Signal history
Edge counts and timings of every output match the model. Outputs that are not expected on must stay off.
- 06 Cross-layer consistency
What the controller reports over the bus agrees with the output levels measured at its pins. Warning-only until calibrated over enough bench runs.
- 07 Timer accuracy
Timed transitions fire within a tolerance of the configured timer; a long timer that fires far too early is flagged.
- 08 Requirement traceability
Each result maps back to the specification clauses it proves.
Timing you can trust
A safety control is judged on when it reacts, not just whether it does. The expectation engine makes every deadline in the suite an assertion.
Every observation in a sequence is armed before the first stimulus action, so an event that lands mid-stimulus is captured rather than missed.
Each deadline starts when the sequence reaches that expectation. Concurrent expectations share one clock, are waited tightest-deadline first, and the group fails fast.
Latency is measured from when the event arrived, not from when the waiter collected it. Otherwise one slow member of a group inflates its neighbours and fails deadlines they actually met.
A waiter could return “seen” after its deadline, so an event that arrived late still counted as a pass. Late now means missed; the only slack is a small allowance for thread scheduling.
Bus events are reported as measured latencies, polled pin levels as upper bounds (≤), because every input-card read takes time. A timeout shorter than the sampling floor is rejected when the test is defined, not discovered on the bench.
How a test states it
# Illustrative only: simplified names, not client code.
act = (
OpenSwitch("limit"), # drive the bench
ExpectFault("LIMIT_OPEN", within=2.0), # observe, in sequence
( # observe, concurrently
ExpectState("LIMIT_POSTPURGE", within=5.0),
ExpectOutput("igniter", level=False, within=1.0),
),
)
engine.arm(act) # every expectation armed before the first action runs
run(act, engine) # each deadline starts when the sequence reaches it Verdicts that mean something
A red result should point at the firmware, and a green one should be earned. Five verdicts keep those apart.
Target state, alarms, output signals and timing all verified.
The firmware did not do what the catalogue says. Evidence goes to the firmware team with a repro script.
The source state was never validly reached. Kept separate from FAIL so bench and navigation noise never reads as a firmware regression; “helper missing” and “helper ran and failed” are distinct messages.
A documented bench limitation. New skip reasons need explicit approval, so a skip cannot quietly hide a regression.
No reaction without a proven stimulus. “Nothing happened” is not a pass.
Before a sweep may claim a state is unreachable under supply dips, the campaign has to prove the AC source actually powers the controller. That gate showed the source barely registering while the controller stayed powered, so the controller was not being fed from it. The campaign stopped there instead of publishing a green negative sweep.
Relay read-back can look fine while the controller's own supply rails are not. The rule is that this precondition has to hold before any downstream result is classified, so one bad power-up cannot contaminate a whole run's verdicts.
Running unattended
- Nightly pipeline with per-key locking, crash and abort recovery, result journalling, Excel evidence workbooks, bilingual HTML dashboards and Mermaid failure graphs. Report generation became dramatically faster.
- One instrument session per process, shared by every test. A failed open used to be cached for the rest of the run, and one transient hiccup cascaded into dozens of failures across unrelated transitions. Now a failed open is retried by the next test, while an instrument disabled by configuration stays off.
- A “hardware fault” reported for months was a leaked USB handle. Fixing it restored a whole blocked test family.
Defects and traceability
- Every open issue that can be reproduced from the workspace has a standalone reproduction or diagnostic script.
- Each finding is classified before anyone acts on it: firmware defect, catalogue drift, bench limitation or harness defect. Family-wide problems are narrowed by confirming one representative transition per source state on the bench before any full sweep.
- The state model is regenerated from the firmware's C state tables with a content hash per source header, so drift between firmware and test catalogue shows up as a diff, not a bench surprise.
- Failing transitions are rendered as a Mermaid graph for triage.
Agentic tooling on the bench
Bench procedures are written down as agent skills and prompts, so AI-assisted work follows the same gates as a person would. The supply-dip campaign prompt, for example, refuses to run a sweep until the positive-control gate passes. The harness of agents, skills, policy hooks and MCP servers was part of daily delivery.
The agentic harness case study →Results
- Replaced hand-written, duplicated tests with a comprehensive suite generated from a YAML catalogue of the full state, event and transition model.
- Closed a critical coverage gap where the stimulus window was never actually measured.
- Traced a “hardware fault” reported for months to a leaked USB handle, which unblocked an entire test family.
- Investigated numerous firmware and harness defects against the C/C++ firmware sources, each with a standalone reproduction script.
- Delivered an unattended nightly pipeline and made evidence report generation dramatically faster.
- Moved a C#/.NET WinForms serial-bus monitor into version control, fixed a log-parser crash and reduced build and deployment to one command.
- Owned delivery end to end: traceability, issue tracking, branch / PR workflow, Conan packaging, CI and monthly client reporting.
Need this kind of verification?
Happy to walk through the bench and the evidence model on a call.