What makes a test non-deterministic, and how do you control time zones, locales, randomness, UUIDs and the file system?

22.4.2 · L3 · Interview

Predict first

The same instant is formatted with different zones and locales, and a seeded generator is run twice. What do the checks report?

import Foundation

let date = Date(timeIntervalSince1970: 1_700_000_000)
func format(_ zone: String, _ locale: String) -> String {
    let f = DateFormatter()
    f.locale = Locale(identifier: locale)
    f.timeZone = TimeZone(identifier: zone)
    f.dateStyle = .medium
    f.timeStyle = .short
    return f.string(from: date)
}
print("zone matters:", format("UTC", "en_US") != format("Asia/Kolkata", "en_US"))
print("locale matters:", format("UTC", "en_US") != format("UTC", "de_DE"))
print("same inputs repeat:", format("UTC", "en_US") == format("UTC", "en_US"))

struct Seeded: RandomNumberGenerator {
    var state: UInt64
    mutating func next() -> UInt64 { state = state &* 6364136223846793005 &+ 1442695040888963407; return state }
}
var a = Seeded(state: 42), b = Seeded(state: 42)
let first = (0..<3).map { _ in Int.random(in: 0..<100, using: &a) }
let second = (0..<3).map { _ in Int.random(in: 0..<100, using: &b) }
print("seeded generators agree:", first == second)

zone matters: true, locale matters: true, same inputs repeat: true, seeded generators agree: true
every line prints false
only the seeded line prints true

Mental model. A science experiment that is not repeated is not evidence. Fix the temperature, the ingredients and the timer, or you cannot tell what changed.

A test is deterministic if the same code and inputs give the same result on every machine and every run. It stops being so when it silently depends on ambient inputs: the current Date, the machine’s time zone, locale and calendar, the language, random numbers, generated UUIDs, the contents of UserDefaults or the file system, network, environment variables, and the order other tests ran in. A test that formats a date and passes in London may fail in Mumbai, and one that checks a localised price string fails on a developer’s German-region simulator. The check below shows that the same instant formats differently by time zone and by locale, while identical inputs repeat.

Control each input explicitly. Pass the time zone, locale and calendar into formatting and date logic (set them on the formatter, Calendar and Locale instead of using the current ones), or set them for the test run in a test plan configuration, so the same suite runs for several regions. Inject randomness as a RandomNumberGenerator (a seeded generator reproduces the sequence) and identifiers as a () -> UUID. Give each test its own temporary directory and an isolated UserDefaults(suiteName:) (Q106) and clean them up, or use in-memory stores. Replace the network with a stub (Q326). Avoid depending on test order; Swift Testing and XCTest can run in a random order, which is a cheap way to find hidden coupling. When a test must be non-deterministic (property-based testing), print the seed on failure so it can be replayed.

Formatting depends on the zone and locale you give it, repeats for identical inputs, and a seeded generator reproduces its sequence.

Common traps

Using Date(), Locale.current or TimeZone.current inside logic that a test asserts on. The result changes with the machine.
Sharing UserDefaults.standard or a fixed file path between tests. They pollute each other and run order decides the outcome.

They might follow up with

How do you run a suite in several locales? Test plan configurations that set language and region, so the same tests run for each.
How do you reproduce a random failure? Log the seed and re-run with it.

What makes a test non-deterministic, and how do you control time zones, locales, randomness, UUIDs and the file system?

Predict first

Common traps

They might follow up with

Related questions