# Media Accessibility

Back to [SKILL.md](../SKILL.md).

How to make audio and video accessible with AVFoundation: tagging and choosing
caption and audio-description tracks, following system settings, and building
player controls that assistive technologies can use.

## Captions, SDH and subtitles

AVFoundation marks accessibility-related tracks with media characteristics
(`AVMediaCharacteristic`,
https://developer.apple.com/documentation/avfoundation/avmediacharacteristic).

| Characteristic | Meaning |
|---|---|
| `.transcribesSpokenDialogForAccessibility` | Text of what is said, as captions or SDH, for viewers who are deaf or hard of hearing |
| `.describesMusicAndSoundForAccessibility` | The text track also covers music and sound effects, not just speech |
| `.describesVideoForAccessibility` | An audio track that narrates the picture for viewers who are blind or have low vision |
| `.easyToRead` | Captions written in simpler language, for cognitive accessibility |
| `.containsOnlyForcedSubtitles` | Only the forced lines, shown where the content's language differs from the device's |
| `.languageTranslation` | Subtitles translating speech into another language |

## Picking an accessible caption track

```swift
import AVFoundation

@MainActor
func makeCaptionedItem(for movieURL: URL) async throws -> AVPlayerItem {
    let movie = AVURLAsset(url: movieURL)
    let item = AVPlayerItem(asset: movie)

    guard let legible = try await movie.loadMediaSelectionGroup(for: .legible) else {
        return item
    }
    let sdhOptions = AVMediaSelectionGroup.mediaSelectionOptions(
        from: legible.options,
        withMediaCharacteristics: [.transcribesSpokenDialogForAccessibility]
    )
    if let sdh = sdhOptions.first {
        item.select(sdh, in: legible)
    }
    return item
}
```

`AVPlayerItem` is a main-actor class. Its `select(_:in:)` is exempt only when
the deployment target is iOS 16 or later, so the function is `@MainActor` to
build for older targets too.

## Picking an audio description track

Same approach on the audible group:

```swift
@MainActor
func enableAudioDescription(on item: AVPlayerItem, source: AVAsset) async throws {
    guard let audible = try await source.loadMediaSelectionGroup(for: .audible) else { return }
    let described = AVMediaSelectionGroup.mediaSelectionOptions(
        from: audible.options,
        withMediaCharacteristics: [.describesVideoForAccessibility]
    )
    if let track = described.first {
        item.select(track, in: audible)
    }
}
```

## Following the system settings

When a user turns on captions or audio descriptions in Settings >
Accessibility > Subtitles & Captioning, `AVPlayer` picks matching tracks on its
own. Select options by hand only when you build your own media-selection UI.

To read the caption preference yourself:

```swift
import MediaAccessibility

let displayType = MACaptionAppearanceGetDisplayType(.user)
let captionsShowByDefault = displayType == .alwaysOn || displayType == .automatic
```

UIKit apps can also check `UIAccessibility.isClosedCaptioningEnabled` and
observe `UIAccessibility.closedCaptioningStatusDidChangeNotification`.

## SwiftUI `VideoPlayer`

`VideoPlayer` from AVKit wraps `AVPlayer`, so it gets the same automatic track
selection.

```swift
import AVKit
import SwiftUI

struct LessonPlayer: View {
    let lessonPlayer: AVPlayer
    let lessonTitle: String

    var body: some View {
        VideoPlayer(player: lessonPlayer)
            .accessibilityLabel("Video: \(lessonTitle)")
            .accessibilityHint("Plays or pauses the lesson")
    }
}
```

## Custom player controls

A custom transport bar has to provide what the system player gives you for
free:

- play/pause, seek and volume are focusable and labelled;
- current time and duration are spoken when focus reaches them;
- a captions toggle exists and is labelled;
- the scrubber reports the time position through `accessibilityValue`;
- controls stay visible and reachable while captions are on screen.

```swift
struct CaptionsButton: View {
    @Binding var captionsOn: Bool

    var body: some View {
        Button(action: { captionsOn.toggle() }) {
            Image(systemName: "captions.bubble")
                .symbolVariant(captionsOn ? .fill : .none)
        }
        .accessibilityLabel(captionsOn ? "Captions on" : "Captions off")
        .accessibilityHint("Turns subtitles and captions on or off")
    }
}

struct Scrubber: View {
    @Binding var position: Double      // seconds
    let duration: Double

    var body: some View {
        Slider(value: $position, in: 0...duration)
            .accessibilityLabel("Playback position")
            .accessibilityValue(
                "\(Duration.seconds(position).formatted(.time(pattern: .minuteSecond))) of "
                + "\(Duration.seconds(duration).formatted(.time(pattern: .minuteSecond)))"
            )
    }
}
```