Shadows That Something Blocks

There are two separate questions in lighting a scene, and they are easy to run together. The first is which way a surface faces, which the previous article covered. The second is whether anything is standing in the way. A surface can be turned directly toward a light and still be pitch dark, because someone is standing between them. That is a shadow, and it is a question about the whole scene rather than about one point.

Walking toward the light

With a depth map, the question can be asked directly. Start at the point being lit and step along the line toward the light. At each step you know where you are in space, and you know — by looking up the depth map at that position — how far away the nearest surface in that direction is. If the surface is nearer than you are, something is in front of you, and the light cannot reach. Keep stepping until you arrive at the light or run out of picture.

It is a simple idea and it produces real occlusion: a raised hand casts a shadow on a chest, a nose casts one on a cheek, a person casts one on the wall behind them. Not an effect that resembles a shadow, but a shadow, arising from the same fact about the world that makes a real one.

Soft edges come from size

If the light is treated as a single point, every spot is either blocked or not, and the shadow has a hard border. Real light sources have area, and near the edge of a shadow you can see part of the source and not the rest — the amount visible falling off gradually as you move further in. That gradient is the penumbra, and it is what a large light produces and a small one does not.

Getting it means asking the question several times across the width of the source rather than once at its center, and using how many of those attempts got through. A bigger source spreads those samples further apart, the answers disagree over a wider band, and the shadow edge softens. The same control, from the same physics, as the one in the last article.

Contact is what sells it

The most valuable shadows in any image are the small dark ones where two things touch — under a chin, where a hand rests on a table, at the base of anything standing on anything. They are the cue that the eye trusts above all others for deciding whether an object is actually in a scene or floating in front of it.

They are also the shadows this method is best at, because the geometry involved is short-range, near the camera, and exactly what the depth map records most accurately. The far end of the scene is where depth is unreliable; contact is where it is at its best.

Missing backs, and where rays get lost

The limitation from the previous article arrives here in force. The depth map records only the surfaces the camera could see. There is no back to anything, and nothing at all is known about the region behind a person.

So a ray that passes behind a surface is in territory that does not exist. Is it inside an object, or in empty space beyond one? The map cannot say, and different assumptions produce different wrong answers: assume solidity and objects cast shadows far longer than they should, as though everything were a wall extending backward forever; assume emptiness and shadows stop short or disappear where they ought to continue. There is a middle course — treat a surface as having a plausible thickness and no more — and it is a fudge that works most of the time and visibly fails some of the time.

The other artifact is a shadow that trails off at the edge of the frame, because the occluder that should have produced it is outside the picture. A real room continues past the borders of a photograph. A depth map does not.

The cost, and why it shapes the app

Every lit pixel, for every light, walks a path of many steps, each one a lookup. At full resolution that is an enormous number of operations for a single frame, and it has to happen again every time a finger moves.

This is why the work belongs on the GPU, where all those pixels are handled at once, and why interactive editing happens at screen resolution rather than at the photo's own. What you drag around is a preview, sized to the display. The full-size render is done once, at the end, when the result is being exported — which is the subject of the last article in this series.

A smaller light tells you more

One practical note that follows from all of the above. When a shadow lands somewhere strange, the instinct is to move the light. Usually the light is fine and the depth under it is not.

A small light makes that diagnosable. Its hard edge traces precisely where the app believes an object stops, so a boundary drawn through the middle of someone's hair is immediately visible as a depth problem rather than a lighting one. A large light hides the same error under a soft gradient — which is a perfectly good way to finish a photograph, and a bad way to work out what is wrong with one.