Agreeing with the reference to 1.1 dB

A GPU port is only worth its speed if it returns the same answer. This is what checking that cost — including the 15 dB error the first honest comparison exposed.

Measured 4 August 2026 · RTX 3060 (sm_86), CUDA 12.9 · against the NTIA/ITS reference implementation, built from public source

The Longley-Rice model has a reference implementation: the ITM code published by the U.S. NTIA's Institute for Telecommunication Sciences, which is also the code the FCC's OET-69 procedure rests on. It is public, it is buildable, and it is not ours. That makes it the only yardstick worth using — a ratio or an agreement figure means nothing when the vendor controls both ends of it.

So the question is not "does BALROG look right". It is: given the same terrain and the same parameters, does BALROG return what the reference returns? Two experiments answer it, and they have to be done in that order.

First, ground with no relief

Flat ground removes terrain from the comparison entirely, so anything left is the model itself: free-space loss, refraction, the variability quantiles. If two implementations disagree here, nothing measured on real relief can be trusted.

Sixteen distances from 3.0 to 25.5 km, 800 MHz, receiver at 2 m, Δh = 0, at the median quantile:

At 1000 m the two implementations agree exactly, at all sixteen distances. That is the strongest statement available: not "close enough", but bit-level identical arithmetic on the model's own terms.

One correction worth recording, because it cost an entire round of analysis. The reference's variability routine defaults to the 0.5 % tail, not the median, and returns corrections of twenty decibels. The median is 50.0. A comparison run against the default is not a comparison of models — it is a comparison of two different questions.

Then, ground that is not flat

Flat ground is necessary, not sufficient. The comparison that matters is on real relief, and it needs one precaution: feeding the reference its own reading of the terrain tile would measure a difference in terrain sampling, not a difference in model. Two engines can disagree simply because they interpolate an elevation grid differently, and that tells you nothing about whether the physics matches.

So the engine was made to dump the elevation profile it actually uses — 851 points at 30 m spacing along one probe profile, elevations from 44 to 152 m — and the reference was driven in point-to-point mode on exactly that array. Same relief, same parameters, only the model differs.

A method note that cost a day. The first attempt dumped the profile with a device-side printf. That silently loses data — the FIFO flushes per block and drops the rest — and returned 270 lines out of 1333, in clumps of about thirty. Nothing warns you. The array is copied back host-side instead, the way the engine already does for attenuation.

The first result was bad, and one-sided

The bias was systematic: the engine under-predicted loss almost everywhere. A scattered error suggests numerical noise; a one-sided one points at a term. Comparing the irregularity parameter Δh directly against the reference's showed which: BALROG's was 20 to 45 m low at every distance, and at 25.5 km it was exactly 0.0000 on terrain carrying 108 m of relief. A Δh of zero on a mountain is not a rounding error.

Two defects, not one

Reading the reference's own ComputeDeltaH.cpp settled the first. It does three things the GPU version did not:

1. it resamples the interval to n points, n between 35 and 245;
2. it fits a straight line and subtracts it — the deciles are taken on the departure from the slope, so a steadily rising valley does not read as roughness;
3. it takes the deciles exactly, by nth_element, rather than iterating toward them.

All three are now transcribed. The exact order statistics fit in a bitonic sort of 256 elements — 36 block barriers, against up to a hundred passes over the interval for the loop it replaced. The replacement is shorter by 188 lines and faster; it was not a trade.

That still left the worst probe. The second defect was narrower and nastier: Δh is computed one point in incr, and the fill routine copies, across the whole block, the value taken at the start of the next block. Near the edge of the domain that point does not exist. Measured: the domain ends at index 853, the block covering 848–855 reaches for 856, and Δh collapsed to 0 on three points where the neighbourhood reads 116.7 m. That single fault was the whole of the 15 dB. The fill now falls back to the block's own start, which is always a computed point.

After

Flat ground is unchanged — 0.000 dB at 1000 m, 0.055 dB at 23 m — so nothing regressed to buy this.

What is still open

Two probes sit above 0.2 dB: +1.12 dB at 13.5 km and +0.40 dB at 4.5 km, both with the receiver in the clear. Here is what they are.

Where the remaining gap lives, measured

Both outliers are in the line-of-sight regime, and that is not a coincidence. In ITM the line-of-sight answer is a blend between a two-ray term and the extended diffraction line, weighted by a factor that depends on the terrain-irregularity parameter Δh. On smooth ground Δh is zero, the weight collapses to exactly 1, and the blend disappears. The 0.000 dB smooth-ground result is therefore structurally blind to any disagreement about Δh — which is worth saying plainly, because it is our most flattering number.

So the question becomes: how much is a metre of Δh worth, in decibels, at these distances? That is measurable on the reference implementation alone, with no BALROG code involved. Sweeping Δh at the sixteen probe distances, 800 MHz, 23 m over 2 m:

The sensitivity is almost flat — 0.083 to 0.108 dB per metre across the whole range. That result refuted the first hypothesis we tested, which was that the two outliers sat at a distance where the model becomes hypersensitive. It does not.

What it does give is a conversion. At 0.102 dB/m, the +0.40 dB at 4.5 km is 3.9 m of Δh. At 0.088 dB/m, the +1.12 dB at 13.5 km is 12.8 m. On ground carrying 108 m of relief, and given that ITM recomputes Δh per receiver point over a window that moves with distance, a disagreement of a few metres at two particular windows is the ordinary size of thing — not a sign of broken arithmetic, which the smooth-ground result already rules out.

What is still open, precisely. The measurement above establishes the mechanism and the magnitude. It does not prove the engine's Δh is the term that differs at those two windows rather than another terrain-derived quantity. Closing it means dumping the engine's Δh at exactly those two probes and comparing it against ComputeDeltaH over the same interval — a small experiment, and the next one we will run.

The honest summary is therefore: identical on smooth ground, within 1.122 dB worst-case on 108 m of relief, with fourteen of sixteen probes inside 0.2 dB — and two line-of-sight points whose disagreement is bounded, converted into terrain metres, and traced to the one part of the model the smooth-ground control cannot exercise.

Why this is on the website

Because speed is worth nothing if the answer is wrong, and because a validation claim you cannot reproduce is marketing. The reference is public source. The tile is a standard SRTM1 tile. The parameters are in five plain-text files. If your evaluation repeats this comparison and gets a different answer, that is a finding we want.

The PDF is this note plus the benchmark method, formatted for a technical review or a procurement file. No form, no email required — it is the same evidence either way.