---
name: speech-recognition
description: "Speech framework speech-to-text: live microphone transcription with AVAudioEngine, recorded audio files, speech and microphone permission, on-device versus server SFSpeechRecognizer, iOS 26 SpeechAnalyzer with SpeechTranscriber, DictationTranscriber, SpeechDetector, AssetInventory and async results. Use when transcribing live or recorded speech or migrating to SpeechAnalyzer."
metadata:
  source: multi-agent-pipeline
---

# Speech recognition

The Speech framework has two generations. `SpeechAnalyzer` with
`SpeechTranscriber` arrived in iOS 26 and runs entirely on device with async
result streams. `SFSpeechRecognizer` dates back to iOS 10 and uses callbacks.
Code here assumes Swift 6.3 and an iOS 26 deployment target, with notes for
older targets.

## Scope

This skill covers speech permissions, microphone capture plumbing,
transcription and result handling. Hand off:

| Need | Skill |
|---|---|
| Analyzing the transcript: language, sentiment, entities, embeddings, translation | `natural-language` |
| Playback UI and captions | `avkit` |
| Summaries or other generation over a transcript | `apple-on-device-ai` |

## Which API

Choose `SpeechAnalyzer` on iOS 26+ for long recordings, live transcription,
transcripts indexed by time, and flows that must stay on the device.

Keep `SFSpeechRecognizer` when the app still supports older systems, when a
locale is only covered by server recognition, or when a working
callback-based implementation does not need to change yet.

### SpeechAnalyzer checklist

1. **Pick the module.** `SpeechTranscriber` is the newer general-purpose
   on-device model. Fall back to `DictationTranscriber` when the transcriber
   is unavailable for the device or locale and dictation-style output is
   acceptable. Add `SpeechDetector` (voice activity detection) only alongside
   a transcriber, and only if the power saving is worth the risk to accuracy.
2. **Check support.** `SpeechTranscriber.isAvailable` and
   `SpeechTranscriber.supportedLocale(equivalentTo:)`; language pickers read
   `installedLocales` and `supportedLocales`.
3. **Pick a preset.** `.transcription` for plain accurate text,
   `.progressiveTranscription` for live UI, `.timeIndexedProgressiveTranscription`
   when playback should highlight words through `audioTimeRange`.
4. **Install assets** with `AssetInventory.assetInstallationRequest(supporting:)`.
5. **Convert live audio** to `SpeechAnalyzer.bestAvailableAudioFormat(compatibleWith:)`
   before yielding `AnalyzerInput` values.
6. **Read results** from the module's async sequence in a separate task.
7. **Finish on purpose** with `finalizeAndFinish(through:)`,
   `finalizeAndFinishThroughEndOfInput()` or `cancelAndFinishNow()`.

There is no `.offlineTranscription` preset, whatever some samples claim.
Closing your `AsyncStream` of input does not finish the analyzer either.
Full code is in [analyzer recipes](references/speechanalyzer-patterns.md).

## SFSpeechRecognizer setup

```swift
import Speech

let deviceLocale = SFSpeechRecognizer()                               // current locale
let british = SFSpeechRecognizer(locale: Locale(identifier: "en-GB"))  // specific locale

guard let recognizer = british, recognizer.isAvailable else {
    // show "speech input unavailable"
    return
}
```

The initializer is failable. Availability also changes at runtime (network
loss, locale switch), so adopt `SFSpeechRecognizerDelegate`. The callback runs
on the recognizer's `queue`, the main queue unless you change it, so a
main-actor model can conform with `@preconcurrency`; without it Swift 6
rejects the conformance as crossing into main-actor code.

```swift
@MainActor
final class DictationModel: NSObject, @preconcurrency SFSpeechRecognizerDelegate {
    var recordButtonEnabled = false

    func speechRecognizer(_ source: SFSpeechRecognizer, availabilityDidChange isUsable: Bool) {
        recordButtonEnabled = isUsable
    }
}
```

## Authorization

Live transcription needs **two** permissions: speech recognition and the
microphone. Add both Info.plist strings:

- `NSSpeechRecognitionUsageDescription`
- `NSMicrophoneUsageDescription`

```swift
func speechAllowed() async -> Bool {
    await withCheckedContinuation { continuation in
        SFSpeechRecognizer.requestAuthorization { status in
            continuation.resume(returning: status == .authorized)
        }
    }
}

func microphoneAllowed() async -> Bool {
    await AVAudioApplication.requestRecordPermission()   // iOS 17+
}
```

Before iOS 17, ask through `AVAudioSession.sharedInstance().requestRecordPermission(_:)`.

## Live microphone transcription

`AVAudioEngine` captures audio, a tap appends each buffer to an
`SFSpeechAudioBufferRecognitionRequest`, and the recognition task reports
results.

```swift
import AVFoundation
import Speech

@MainActor
final class LiveCaptioner {
    private let engine = AVAudioEngine()
    private let recognizer = SFSpeechRecognizer(locale: Locale(identifier: "en-AU"))
    private var request: SFSpeechAudioBufferRecognitionRequest?
    private var task: SFSpeechRecognitionTask?
    private(set) var caption = ""

    func begin() throws {
        task?.cancel()
        task = nil

        let session = AVAudioSession.sharedInstance()
        try session.setCategory(.record, mode: .measurement, options: [.duckOthers])
        try session.setActive(true, options: .notifyOthersOnDeactivation)

        let request = SFSpeechAudioBufferRecognitionRequest()
        request.shouldReportPartialResults = true
        self.request = request

        task = recognizer?.recognitionTask(with: request) { [weak self] result, error in
            let text = result?.bestTranscription.formattedString
            let done = error != nil || (result?.isFinal ?? false)
            Task { @MainActor in
                guard let self else { return }
                if let text { self.caption = text }
                if done { self.end() }
            }
        }

        feedMicrophone(engine.inputNode, into: request)
        engine.prepare()
        try engine.start()
    }

    // nonisolated so the tap closure is not main-actor isolated: it runs on the audio thread
    private nonisolated func feedMicrophone(_ mic: AVAudioInputNode,
                                            into request: SFSpeechAudioBufferRecognitionRequest) {
        let native = mic.outputFormat(forBus: 0)
        mic.installTap(onBus: 0, bufferSize: 2048, format: native) { buffer, _ in
            request.append(buffer)
        }
    }

    func end() {
        engine.stop()
        engine.inputNode.removeTap(onBus: 0)
        request?.endAudio()
        request = nil
        task?.cancel()
        task = nil
    }
}
```

The tap block runs on a real-time audio thread. Under Swift 6 a closure
written inside a `@MainActor` method inherits that isolation and traps when
called elsewhere, which is why the tap is installed from a `nonisolated`
helper.

## Transcribing a recorded file

```swift
func transcribeFile(at url: URL, with recognizer: SFSpeechRecognizer) async throws -> String {
    let request = SFSpeechURLRecognitionRequest(url: url)
    request.shouldReportPartialResults = false

    let text: String = try await withCheckedThrowingContinuation { continuation in
        var settled = false
        recognizer.recognitionTask(with: request) { outcome, failure in
            guard !settled else { return }
            if let failure {
                settled = true
                continuation.resume(throwing: failure)
            } else if let outcome, outcome.isFinal {
                settled = true
                continuation.resume(returning: outcome.bestTranscription.formattedString)
            }
        }
    }
    return text
}
```

The `settled` flag stops a second resume if the handler fires again.

## On device or on the server

- Since iOS 13 many locales can be recognized on the device. When
  `supportsOnDeviceRecognition` is `false`, recognition needs the network.
- `requiresOnDeviceRecognition = true` is only honoured when the recognizer
  supports it, so check first:

```swift
if recognizer.supportsOnDeviceRecognition {
    request.requiresOnDeviceRecognition = true
} else {
    // tell the user audio will be sent for recognition, or pick another locale
}
```

- Server-backed recognition has limits, including roughly one minute of
  audio per task. On iOS 26+ prefer `SpeechAnalyzer` for long recordings;
  otherwise split the audio and restart before the limit, carrying the
  transcript forward.

## Results

- `shouldReportPartialResults` defaults to `true`. Partial text can still
  change; `isFinal` marks the last result of a task.
- `result.transcriptions` holds alternatives, most confident first. Each
  transcription's `segments` expose `substring` and `confidence`.
- `request.addsPunctuation = true` (iOS 16+) inserts punctuation.
- `request.contextualStrings = ["Kubernetes", "Grafana"]` nudges the
  recognizer toward domain vocabulary.

## Common mistakes

- **Asking only for speech permission.** Live audio also needs the
  microphone, or the engine fails to start. Request both before recording.
- **Ignoring availability changes.** Watch the delegate; network loss or a
  locale change can make the recognizer unavailable.
- **Leaving the engine running.** On a final result or an error, stop the
  engine, remove the tap and end the audio.
- **Forcing on-device without support.** The flag is ignored; check
  `supportsOnDeviceRecognition` and fall back or inform the user.
- **Running into the one-minute ceiling.** Roll over before it, for example
  with a timer at 55 seconds: keep the latest partial text, stop through the
  single cleanup path (the only place that calls `endAudio()`, removes the tap
  and cancels), then start a fresh task.
- **Inventing task APIs.** `SFSpeechRecognitionTask` offers `finish()`,
  `cancel()`, `state` and `error`, nothing to restart itself. Hold the live
  request as a property of the owning object, and let exactly one code path
  call `endAudio()`.
- **Assuming the analyzer finishes when input ends.** Finishing your input
  builder does not close result streams. After `analyzeSequence(_:)` returns
  the last sample time, call `finalizeAndFinish(through:)`; if there is none,
  `cancelAndFinishNow()`.
- **Duplicating volatile text.** When a result is final, clear the volatile
  part and append the final text; otherwise replace the volatile part.
- **Overlapping tasks.** Cancel the previous recognition task before creating
  a new one; concurrent tasks behave unpredictably.

## Review checklist

- [ ] `NSSpeechRecognitionUsageDescription` present, plus `NSMicrophoneUsageDescription` for live audio
- [ ] Authorization requested before any recognition
- [ ] Delegate reacts to `availabilityDidChange`
- [ ] Engine stopped, tap removed and `endAudio()` called when finished
- [ ] Previous task cancelled before a new one starts
- [ ] `supportsOnDeviceRecognition` checked before requiring on-device
- [ ] Partial and final results handled differently
- [ ] Task duration and service limits planned for
- [ ] iOS 26+: assets installed through `AssetInventory`, `isAvailable` and locale support checked, live buffers converted, sessions finalized or cancelled, volatile text replaced rather than duplicated

## References

- [analyzer recipes](references/speechanalyzer-patterns.md)
- https://developer.apple.com/documentation/speech
- https://developer.apple.com/documentation/speech/speechanalyzer
- https://developer.apple.com/documentation/speech/speechtranscriber
- https://developer.apple.com/documentation/speech/speechtranscriber/preset
- https://developer.apple.com/documentation/speech/dictationtranscriber
- https://developer.apple.com/documentation/speech/speechdetector
- https://developer.apple.com/documentation/speech/assetinventory
- https://developer.apple.com/documentation/speech/sfspeechrecognizer
- https://developer.apple.com/documentation/speech/sfspeechrecognitionrequest
- https://developer.apple.com/documentation/speech/sfspeechaudiobufferrecognitionrequest
- https://developer.apple.com/documentation/speech/sfspeechurlrecognitionrequest
- https://developer.apple.com/documentation/speech/sfspeechrecognitionresult
- Apple articles on requesting speech-recognition permission and on recognizing live audio, in the Speech documentation
- WWDC25 session 277 (SpeechAnalyzer introduction)