The Device, the Queue, the Frame

This series is about Metal, Apple's GPU API, built up from the smallest thing that draws to the shaders that paint a whole racing world in flat colour. The code is from Veta, a kart racer for the Mac I am building — a Mode 7 game whose ground is one triangle and whose karts are small low-poly meshes — and every listing is what actually ships in it.

Twelve posts, in four parts: the basics, then the things you reach for once the first triangle works, then Metal 4 and the fallback that keeps a game running on GPUs that do not have it, then three posts on shaders.

Metal is not a drawing API

The first thing to unlearn, coming from Core Graphics or SwiftUI, is that Metal calls do not draw. They record. You build a list of commands on the CPU, hand the finished list to a queue, and the GPU works through it whenever it gets to it. Your function has returned long before any pixel exists.

That is the reason for nearly every piece of ceremony in this post. Four objects, in a hierarchy:

let device: MTLDevice          // the GPU itself: makes everything else
let queue = device.makeCommandQueue()!     // a stream of work, made once
let commands = queue.makeCommandBuffer()!  // one frame's list, made per frame
let encoder = commands.makeRenderCommandEncoder(descriptor: pass)!  // writes into the list

The device is the GPU. It is a factory — buffers, textures, pipelines, queues all come from it — and on a Mac you usually want the one MTKView already picked.

The queue is created once, at startup, and lives as long as the renderer. Making one per frame is a common early mistake; it is an expensive object.

The command buffer is one frame's worth of recorded work. Make it, fill it, commit it, throw it away.

The encoder is what actually writes commands into the buffer, and it is exclusive: one encoder at a time per command buffer, and you must call endEncoding() before making another. A render encoder records draws; a compute encoder records dispatches; a blit encoder records copies.

The render pass descriptor decides what you are drawing into

An encoder is created from a MTLRenderPassDescriptor, and that descriptor is where the attachments live: which texture the colour goes to, whether to clear it first, whether to keep the result. With MTKView you get one for free, already pointed at this frame's drawable:

guard let pass = view.currentRenderPassDescriptor, let drawable = view.currentDrawable,
      let commands = queue.makeCommandBuffer(),
      let render = commands.makeRenderCommandEncoder(descriptor: pass) else { return }

Every one of those is optional, and the reason is worth knowing: currentDrawable is a texture from a small pool the display system owns, and if all of them are still in use, asking blocks and can return nothing. A frame that cannot get a drawable should be skipped, not waited for. That guard ... else { return } is not defensive programming; it is the normal path under load.

Settings that belong to the view rather than the frame are set once, when the renderer is built:

view.colorPixelFormat = .bgra8Unorm_srgb
view.depthStencilPixelFormat = .depth32Float
view.sampleCount = Launch.sampleCount        // 4× multisampling unless -samples says otherwise
view.preferredFramesPerSecond = 120
view.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)

Two of those decide things you cannot change later without rebuilding every pipeline, which is the subject of the next post. bgra8Unorm_srgb in particular is a decision about colour, not about storage: with the _srgb suffix the hardware converts what your shader writes from linear light into sRGB on the way to the screen, so blending happens in linear light where it is physically meaningful. Without it, every blend and every anti-aliased edge is computed in the wrong space and looks subtly muddy.

The frame, start to finish

Here is a real one, minus the parts later posts are about:

func frame(in view: MTKView, capture: URL?, times: GPUFrameTimes, encode: (DrawEncoder) -> Void) {
    guard let pass = view.currentRenderPassDescriptor, let drawable = view.currentDrawable,
          let commands = queue.makeCommandBuffer(),
          let render = commands.makeRenderCommandEncoder(descriptor: pass) else { return }
    inFlight.wait()
    arena.nextFrame()
    encode(Encoder(encoder: render, backend: self))
    render.endEncoding()

    commands.present(drawable)
    times.record(commands)
    let inFlight = self.inFlight
    commands.addCompletedHandler { _ in inFlight.signal() }
    commands.commit()
}

Read the order carefully, because it is the order of a promise rather than of events. present(drawable) does not present anything; it records "when you reach this point, put this texture on screen". addCompletedHandler registers a closure that runs on some other thread, later, when the GPU is finished. commit() hands the list to the queue and returns immediately.

By the time frame returns, nothing has been drawn. That is not a problem to be solved; it is the thing that makes a GPU fast, and the rest of this series is mostly about arranging your program so that it is true safely.

How do you know it was fast?

Because the work is asynchronous, timing it on the CPU measures how long it took to write down the work. The GPU's own time comes back through the completion handler:

/// Adds a completed handler to `commands` that records its GPU time. Call before commit.
public func record(_ commands: MTLCommandBuffer) {
    commands.addCompletedHandler { [self] buffer in
        let elapsed = buffer.gpuEndTime - buffer.gpuStartTime
        if elapsed > 0 { add(elapsed) }
    }
}

One measurement finding is worth carrying from the start, because it will otherwise make every optimisation you do look better than it is:

/// Measure a budget with the frame paced as the game runs it, not back to back. A spike
/// found the same frame costs about three times as much at 60 Hz as flat out, because the
/// GPU drops its power state between paced frames.

A benchmark that renders the same frame in a tight loop keeps the GPU at its highest clocks and reports a number your game will never see. Measure with the frame paced the way it actually runs.

Next

A device, a queue and an empty pass clear the screen to black. The next post draws something into it — a triangle that covers the whole viewport, with no vertex buffer at all.