Taking the Haze Back Out

Air lays haze over a photo by distance: each pixel is the scene seen through a transmittance t, plus the colour of the haze, the airlight A, filling in what t lets go. Clear runs the same model backwards. If a hazy photo is I = J·t + A·(1 − t), then the scene behind it is J = (I − A·(1 − t)) / t, and the whole problem is knowing t. This article is about where Clear gets it, and about the one photograph out of five where it got it wrong.

What a dehaze slider has to guess

A dehaze tool working from the photo alone has to guess t. The simplest guess is one amount for the whole frame. The published method behind most dehaze tools is He, Sun and Tang’s dark channel prior, from 2009: in a clear outdoor scene almost any small patch has some pixel that is nearly black in one channel, so how far haze has lifted that darkest pixel measures the haze in front of the patch.

The weakness is the patch. Where one straddles a near branch and a far hillside, it reads one t for both, and leaves a halo along the edge.

Rake has something the slider does not: a depth map for every photo, from the camera or estimated by the model. Haze thickens with distance, so t should depend on depth and on nothing else. Clear keeps the dark channel’s idea of measuring haze from the darkest pixels, but it measures it across whole bands of depth rather than in patches, and then lets every pixel read t from its own distance.

A curve over inverse depth

First the airlight. Clear takes the haziest 0.1 % of the photo by its patch-minimum dark channel, and averages the brightest tenth of those. Then it sorts every pixel that is not sky by depth and cuts them into 24 bands of equal pixel count. In each band it divides each pixel’s darkest channel by the airlight’s and takes the value 5 % of the way up: the darkest pixels in the band, without letting a few stray ones decide it. One minus that, kept no lower than 0.05, is the band’s transmittance.

while k < samples.count {
    let band = samples[k..<min(k + per, samples.count)]
    let depths = band.map(\.0), ratios = band.map(\.1).sorted()
    let lower = ratios[Int(Float(ratios.count - 1) * 0.05)]
    ds.append(depths[depths.count / 2])
    ts.append(1 - min(max(lower, 0), 0.95))
    k += per
}
curveDisparity = ds
curveT = nonIncreasing(ts)

The last line matters. Haze cannot thin out with distance, so the 24 measurements are forced into the closest sequence that never rises, by pool-adjacent-violators: wherever a farther band reads clearer than a nearer one, the two are averaged until the order holds.

The first version built the curve over metres, and it did not work. The phone’s map stores disparity, which is one over distance, as the first series showed, and the estimate is linear in it too; in metres every distant ridge crowds together at the far end where the curve has nothing left to tell them apart. Over inverse depth they spread out, and each ridge gets its own share of the curve. Between bands t is interpolated linearly in inverse depth, and it is held flat past either end.

The sky, from depth alone

The sky is already the colour of the airlight, and dividing it by a small t pushes it towards dark noise. Clear has to find it and leave most of it alone.

The first attempt found it by brightness, and it broke the sky into blobs and blotched the ridges underneath. The sky now comes from depth only: the farthest 0.3 % of the range of inverse depth is sky, and from there to 1.2 % it feathers out to none. A wider band, from 1.5 to 4 %, was tried, and it swallowed the Kathmandu mountains whole. Where a pixel is sky, its t is not allowed below 0.75:

let t = pow(max(scene.transmittance(atDepth: z), 1e-3), s.strength)
let k = scene.sky[i]
return t * (1 - k) + max(t, 0.75) * k

Strength is applied as a power of t, which scales the optical depth rather than the result: 0 removes nothing, 1 removes all the measured haze, and the control goes up to 1.5 for more. The other control, Protect in the app, is the floor: the least t that is ever divided by, 0.25 by default, so the far distance is not pushed into noise.

Seams, and a filter against the photo

An estimated depth map has edges the photo does not. Two bands can meet in the middle of one hillside, and a step in t there became a step in the result: a dark contour line drawn across the slope, seen in the fading-distance and Kathmandu photos. The fix is the same guided filter the dark channel prior uses, but pointed the other way. Clear filters t against the photo’s luma, with a radius of a forty-eighth of the long side and no less than 4 pixels. The filtered t keeps the edges the photo has and loses the ones only the depth map has.

Putting the light back

Subtracting haze removes light, and a dehazed photo comes out darker than it went in. Clear compares the median brightness of the ground, everything that is not sky, before and after, and lifts the result by the ratio. It never darkens, and it never lifts by more than 1.6×.

a.sort(); b.sort()
return min(max(a[a.count / 2] / max(b[b.count / 2], 1e-4), 1), 1.6)

That gain is one number for the whole photo, and it has to be worked out once. In the test rig it was at first computed inside the per-pixel map, so the whole measurement ran again for every pixel, and the run hung for ten minutes. Now render computes t, dehazes, measures the gain and multiplies, in that order.

The kill test

The plan for Clear was five hazy landscapes against Lightroom’s Dehaze, and Clear would die if the depth could not separate the ridges or if Dehaze won. Lightroom cannot be run where the rebuild was done, so the test was run against the two things a dehaze slider is built from: one global amount of haze removed everywhere, and the dark channel prior with its guided-filter refinement. The same five photos through Lightroom’s Dehaze, by hand, are still to do.

The five are public-domain and CC0 photos from Wikimedia Commons: smoke over Mount Rainier, smog over Lower Manhattan, ridges fading into the distance, blue slopes, and the Kathmandu valley under haze. All five ran on estimated depth, and the rig wrote one sheet for each with the photo, the global dehaze, the prior and Clear side by side.

On the Rainier photo, the prior turns the smoke layer pink; Clear keeps it white, with a crisp forest and a blue sky. Over Manhattan the skyline comes back without the prior’s colour noise across the sky, though Clear leaves a slightly over-dark strip along the far left shore. In the fading distance every ridge separates, with no noise, though the prior gets a little more contrast in the clouds. The blue slopes were subtle both ways, with the prior adding noise to the sky.

Kathmandu went to the prior. The depth estimate puts the distant mountains at the sky’s depth, Clear protects them as sky, and they stay hazy. Everything Clear does rides on the estimate separating the far layers, and where it does not, Clear does too little rather than something wrong.

The global dehaze was never close. One amount removed everywhere crushes the near ground to black and leaves the distance hazy, which is exactly the problem depth exists to solve. The verdict was continue. There were no Portrait photos in the test; portraits are rarely hazy, and their depth maps reach only a few metres.

First in the stack

Haze was in the air before any light Rake adds, so in the app Clear runs first. The tool switcher reads Clear, Relight, Air in the order they apply, and the stacked composite on the GPU takes the haze out, then relights what is left, then lays Air’s fog over the result.

float t = tk.x, k = clamp(tk.y, 0.0f, 1.0f), v = clamp(tk.z, 0.0f, 1.0f);
float3 j = max((albedo - clearParams.rgb * (1.0f - t)) / max(t, clearParams.w) * params.w, 0.0f);
float3 c = j * (params.x * g.veil.a + e) * a.a + a.rgb;

The transmittance map is not made on the GPU: nothing in it changes while a light is dragged. It is computed on the CPU, on the render queue, only when Clear’s settings change, and uploaded with the airlight, the floor and the gain; the shader only applies the formula. Against the CPU reference the mean difference is 0.24 of an 8-bit step. The worst is 25 steps, where dividing by a small t magnifies the rounding of an 8-bit photo.

In the Simulator, Clear was checked on the vase photo for the interface and the pipeline only. The hazy test photos need estimated depth, which the Simulator could not make, and nothing here has been timed on a device.

What the tests hold

• Haze added to the synthetic test scene by the model itself, with an airlight of 0.8, 0.82, 0.85 and t falling as e^{−0.5·depth}, is taken out again by Clear: the cleared photo’s error against the clean scene, in luma over everything nearer than 4 m, is less than half the hazy photo’s.
• With no lights and no air, the GPU’s stacked composite at the working size matches the CPU Clear to a mean of less than one 8-bit step.
• The kill test: against a global dehaze and the dark channel prior, Clear won three of five hazy photos, was about even on one, and lost Kathmandu. The comparison with Lightroom’s Dehaze has not been done.

Clear moves a photo’s far distance closer by taking out what lay in front of it. The next article moves the other thing a camera fixed at the moment of the shot, the plane of focus.