How do AVMutableComposition, tracks and time ranges work for editing, and how do you export the result?

18.1.2 · L5 · Interview

Predict first

What does a time range’s end and containment give, and what does an empty composition with a new track hold?

import AVFoundation

let r = CMTimeRange(start: CMTime(seconds: 1, preferredTimescale: 600), duration: CMTime(seconds: 4, preferredTimescale: 600))
print(CMTimeGetSeconds(r.end), r.containsTime(CMTime(seconds: 3, preferredTimescale: 600)), r.containsTime(CMTime(seconds: 5, preferredTimescale: 600)))
let comp = AVMutableComposition()
print(comp.duration.seconds, comp.tracks.count)
let tr = comp.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)!
print(comp.duration.seconds, comp.tracks.count, tr.mediaType.rawValue)

5.0 true false, then 0.0 0, then 0.0 1 vide
4.0 true true, then 0.0 0, then 4.0 1 vide
5.0 true false, then 1.0 1, then 0.0 0 vide

Mental model. A composition is an edit decision list: “from reel A take seconds 2 to 6, and put it at the 10-second mark”. No film is cut, the list is only followed when you play or export.

An AVMutableComposition is an AVAsset you build in code by stitching segments of other assets on a timeline, without copying any media: it only records which time range of which source track appears where. A new composition is empty (duration 0, no tracks). You add tracks (addMutableTrack(withMediaType: .video, preferredTrackID:), one per kind of media you will lay down: one for video, one or more for audio) and insert content with track.insertTimeRange(_ range: CMTimeRange, of: sourceTrack, at: time): take seconds 2 to 6 of the source clip and place it at second 10 of the composition. Time is expressed with CMTime and CMTimeRange (a start and a duration, whose end is start + duration, and which can contain a time), so a cut is arithmetic on rationals. Note that adding a track does not add time: a composition’s duration comes from media actually inserted. To layer, transition or transform you add a video composition (AVMutableVideoComposition with AVMutableVideoCompositionInstruction and layer instructions that set opacity, transform and crop ramps, or a custom AVVideoCompositing that renders each frame with Core Image or Metal) and an AVMutableAudioMix for volume ramps and ducking. Orientation, frame size and frame rate must be set deliberately (renderSize, frameDuration), and source tracks’ preferredTransform must be applied or portrait clips come out rotated.

Playing a composition is just AVPlayerItem(asset: composition, ...) with the video composition and audio mix attached, so you can preview edits live without rendering. Exporting uses AVAssetExportSession (a preset such as AVAssetExportPresetHighestQuality or a specific resolution; call export() async and observe the status or progress; check supportedFileTypes and the output URL must not already exist) or, for full control of codec and bitrate, AVAssetReader and AVAssetWriter (read sample buffers, process them, write with chosen compression settings). Practical concerns: load tracks and durations asynchronously before building; keep long exports in the background with a background task assertion and handle interruption; mind memory with 4K sources (process frame by frame, release buffers); test with HDR, variable frame rates and different orientations; check available storage first; and provide cancellation (cancelExport()). For simple trims use the export session’s timeRange; for editors use compositions; for processing every frame (filters, ML) use a custom compositor or the reader and writer pair. Metal or Core Image in a custom compositor keeps rendering on the GPU and avoids copying frames to the CPU.

The range 1 s + 4 s ends at 5 s and does not contain 5 s itself. A new composition is empty, and adding a video track does not give it any duration.

Common traps

Ignoring each clip’s preferredTransform. Portrait videos come out sideways; apply it in a video composition instruction.
Exporting to a URL that already exists. The export fails; remove the file first.

They might follow up with

Does adding a track lengthen a composition? No, only inserted media does.
When do you use AVAssetWriter rather than an export session? When you need precise control of codec, bitrate or per-frame processing.

How do AVMutableComposition, tracks and time ranges work for editing, and how do you export the result?

Predict first

Common traps

They might follow up with

Related questions