How does AVAudioEngine work: nodes, formats, buffers, and when do you use it instead of AVAudioPlayer?

18.2.2 · L5 · Interview

Predict first

What do the standard format, a buffer’s duration, and a freshly attached engine report?

import AVFoundation

let fmt = AVAudioFormat(standardFormatWithSampleRate: 44100, channels: 2)!
print(fmt.sampleRate, fmt.channelCount, fmt.isInterleaved, fmt.commonFormat == .pcmFormatFloat32)
let buf = AVAudioPCMBuffer(pcmFormat: fmt, frameCapacity: 4410)!
buf.frameLength = 4410
print(Double(buf.frameLength) / fmt.sampleRate)
let eng = AVAudioEngine()
let pl = AVAudioPlayerNode()
eng.attach(pl)
eng.connect(pl, to: eng.mainMixerNode, format: fmt)
print(eng.attachedNodes.contains(pl), eng.isRunning)

44100.0 2 false true, then 0.1, then true false
44100.0 1 true false, then 1.0, then true true
48000.0 2 false true, then 0.1, then false false

Mental model. The engine is a mixing desk: each source is a channel strip, each effect a pedal, all wired with cables of a stated format. Nothing is heard until you switch the desk on, and a cable of the wrong kind either needs an adapter (a mixer) or does not fit at all.

Use AVAudioPlayer to play a file with minimal code, and AVAudioEngine when you need a processing graph: mixing several sources, effects (reverb, EQ, delay, time-pitch), real-time input processing, metering, synthesis, or spatial audio. The engine is a graph of nodes: source nodes (AVAudioPlayerNode for files and buffers, inputNode for the microphone, AVAudioSourceNode for synthesised samples), processing nodes (AVAudioUnitEffect subclasses, AVAudioMixerNode) and the output (mainMixerNode → outputNode). You attach a node, connect it to the next with a format, prepare() and start() the engine, then play() the player nodes. A new engine is not running (isRunning is false) until you start it, and attaching a node does not start anything. Every connection has an AVAudioFormat: a sample rate (44,100 or 48,000 Hz), a channel count and a sample type (the standard format is 32-bit float, deinterleaved), and formats must match or be converted by a mixer; a mismatch (for example an input node at 48 kHz connected at 44.1 kHz) is a classic crash. Audio data is carried in an AVAudioPCMBuffer with a frameCapacity and a frameLength: 4,410 frames at 44,100 Hz is exactly 0.1 seconds, which is how you convert between frames and time.

Things that matter. Real-time constraints: audio runs on a high-priority real-time thread that must finish each render cycle in a few milliseconds, so in a tap or source-node callback do not allocate memory, take locks, call Objective-C or Swift runtime features that may block, or do I/O; copy into a pre-allocated ring buffer and process elsewhere. Latency is set by the session’s preferredIOBufferDuration and the hardware (Bluetooth adds a lot). Lifecycle: the engine stops on interruptions, route changes and configuration changes (AVAudioEngineConfigurationChange notification), so rebuild or restart the graph, and you must not touch nodes while the engine is running in ways that need reconfiguration. Offline and manual rendering (enableManualRenderingMode) lets you process audio faster than real time into buffers or files, which is how you export or test. Schedule playback precisely with AVAudioTime (host-time based) to start several players in sync, use installTap for metering and visualisation (the tap callback runs off the main thread), prefer scheduleBuffer or scheduleFile with completion handlers, and for 3D audio use AVAudioEnvironmentNode or the system’s spatial audio support. Keep the session category consistent with what the engine does (record needs .playAndRecord).

The standard format is 32-bit float and deinterleaved. 4,410 frames at 44,100 Hz is a tenth of a second, and an attached engine is not running until started.

Common traps

Allocating or locking inside a render or tap callback. It runs on a real-time thread; a stall causes glitches. Use pre-allocated buffers.
Connecting nodes with mismatched formats. Sample rate or channel count differences crash or distort; match them or insert a mixer.

They might follow up with

How long is a 4,410-frame buffer at 44.1 kHz? 0.1 seconds: frames divided by the sample rate.
What happens to the engine on a route change? It can stop and post a configuration change; restart or rebuild the graph.

How does AVAudioEngine work: nodes, formats, buffers, and when do you use it instead of AVAudioPlayer?

Predict first

Common traps

They might follow up with

Related questions