# VisionKit scanners

Camera UI that VisionKit provides on top of Vision: the live
`DataScannerViewController` and the paper-oriented
`VNDocumentCameraViewController`. Summary in `../SKILL.md`.

## Contents

- [Camera permission](#camera-permission)
- [DataScannerViewController](#datascannerviewcontroller)
- [Delegate callbacks](#delegate-callbacks)
- [SwiftUI](#swiftui)
- [Custom overlay](#custom-overlay)
- [VNDocumentCameraViewController](#vndocumentcameraviewcontroller)
- [Performance](#performance)

## Camera permission

Both scanners use the camera, so Info.plist needs `NSCameraUsageDescription`
before either is shown. Ask for access explicitly when you want to control the
moment of the prompt:

```swift
import AVFoundation

func cameraAllowed() async -> Bool {
    let status = AVCaptureDevice.authorizationStatus(for: .video)
    if status == .authorized {
        return true
    }
    if status == .notDetermined {
        let granted = await AVCaptureDevice.requestAccess(for: .video)
        return granted
    }
    return false
}
```

`.denied`, `.restricted` and any status added in a future release all fall
through to `false`.

## DataScannerViewController

A full-screen live scanner with built-in highlighting and tap handling.
Available from iOS 16 on devices with an A12 Bionic chip or newer.

Two class properties gate it, and both must be true before presenting:

- `isSupported`: the hardware can run it.
- `isAvailable`: supported, plus camera access is granted and not blocked by
  restrictions such as device management.

### Initializer options

| Parameter | Default | Meaning |
|---|---|---|
| `recognizedDataTypes` | required | Set of text and barcode types to find |
| `qualityLevel` | `.balanced` | Speed versus detail |
| `recognizesMultipleItems` | `false` | Track many items or only the most prominent |
| `isHighFrameRateTrackingEnabled` | `true` | Smoother highlight tracking, more power |
| `isPinchToZoomEnabled` | `true` | Pinch gesture zooms the camera |
| `isGuidanceEnabled` | `true` | On-screen hints such as "slow down" |
| `isHighlightingEnabled` | `false` | System highlight boxes around found items |

Three configurations for different jobs:

```swift
import VisionKit

@MainActor
enum ScannerPresets {
    static func signReader() -> DataScannerViewController {
        DataScannerViewController(
            recognizedDataTypes: [.text()],
            qualityLevel: .accurate,
            recognizesMultipleItems: true,
            isHighFrameRateTrackingEnabled: false,
            isPinchToZoomEnabled: true,
            isGuidanceEnabled: false,
            isHighlightingEnabled: true
        )
    }

    static func boardingPassReader() -> DataScannerViewController {
        DataScannerViewController(
            recognizedDataTypes: [.barcode(symbologies: [.pdf417, .aztec, .qr])],
            qualityLevel: .fast,
            isHighFrameRateTrackingEnabled: false,
            isPinchToZoomEnabled: false
        )
    }

    static func parcelReader() -> DataScannerViewController {
        DataScannerViewController(
            recognizedDataTypes: [
                .text(textContentType: .shipmentTrackingNumber),
                .barcode(symbologies: [.code128, .qr])
            ],
            qualityLevel: .balanced,
            isHighlightingEnabled: true
        )
    }
}
```

### Data types

`DataScannerViewController.RecognizedDataType` values:

- `.text(languages:textContentType:)`: both arguments are optional, so
  `.text()`, `.text(languages: ["fr"])` and
  `.text(textContentType: .flightNumber)` all work.
- `.barcode(symbologies:)`: takes `VNBarcodeSymbology` values.

Text content types narrow recognition to one kind of string: `.URL`,
`.emailAddress`, `.telephoneNumber`, `.fullStreetAddress`, `.flightNumber`,
`.shipmentTrackingNumber`, `.dateTimeDuration`, and `.currency` (iOS 17).

A retail product scanner usually needs `.ean13`, `.ean8`, `.upce` and
`.code128`.

### Quality levels

- `.fast`: lowest latency; barcodes and large, quick text.
- `.balanced`: the sensible default for most scanners.
- `.accurate`: small or dense text; slower to respond.

### Start and stop

Scanning can only start once the controller is on screen. Set the delegate,
present, and start in the presentation completion. To finish, call
`stopScanning()` and then dismiss.

```swift
import UIKit
import VisionKit

@MainActor
final class ScanCoordinator {
    private var live: DataScannerViewController?

    func open(from host: UIViewController, delegate: DataScannerViewControllerDelegate) {
        let passReader = ScannerPresets.boardingPassReader()
        passReader.delegate = delegate
        live = passReader
        host.present(passReader, animated: true) {
            try? passReader.startScanning()
        }
    }

    func close() {
        guard let live else { return }
        live.stopScanning()
        live.dismiss(animated: true)
        self.live = nil
    }
}
```

## Delegate callbacks

`DataScannerViewControllerDelegate` methods, all main-actor isolated:

| Method | When it fires |
|---|---|
| `dataScanner(_:didTapOn:)` | The user tapped a recognized item |
| `dataScanner(_:didAdd:allItems:)` | New items appeared |
| `dataScanner(_:didUpdate:allItems:)` | Items moved or their content changed |
| `dataScanner(_:didRemove:allItems:)` | Items left the frame |
| `dataScanner(_:becameUnavailableWithError:)` | Scanning stopped being possible, for example camera access was revoked; dismiss or fall back |
| `dataScannerDidZoom(_:)` | Zoom factor changed |

`RecognizedItem` is an enum. Handle both cases and keep an `@unknown default`:

```swift
import VisionKit

func describe(_ item: RecognizedItem) -> String {
    switch item {
    case .text(let recognized):
        return recognized.transcript
    case .barcode(let code):
        return "\(code.observation.symbology.rawValue): \(code.payloadStringValue ?? "?")"
    @unknown default:
        return ""
    }
}
```

### Async stream of items

`recognizedItems` is an `AsyncStream<[RecognizedItem]>`: every element is the
full set of items currently in view.

```swift
import VisionKit

@MainActor
func watch(_ scanner: DataScannerViewController,
           onText: ([String]) -> Void,
           onCodes: ([String]) -> Void) async {
    for await visible in scanner.recognizedItems {
        var lines: [String] = []
        var codes: [String] = []
        for item in visible {
            switch item {
            case .text(let recognized): lines.append(recognized.transcript)
            case .barcode(let code): codes.append(code.payloadStringValue ?? "")
            @unknown default: break
            }
        }
        onText(lines)
        onCodes(codes)
    }
}
```

### Still photo

`try await scanner.capturePhoto()` returns a `UIImage` of the current camera
frame, useful for a high-resolution OCR pass or for attaching to a record.

## SwiftUI

A representable that reports tapped items through bindings:

```swift
import VisionKit
import SwiftUI

struct LiveScanner: UIViewControllerRepresentable {
    let kinds: Set<DataScannerViewController.RecognizedDataType>
    var quality: DataScannerViewController.QualityLevel = .balanced
    var many = false
    @Binding var lines: [String]
    @Binding var payloads: [String]

    func makeUIViewController(context: Context) -> DataScannerViewController {
        let liveView = DataScannerViewController(
            recognizedDataTypes: kinds,
            qualityLevel: quality,
            recognizesMultipleItems: many,
            isHighlightingEnabled: true
        )
        liveView.delegate = context.coordinator
        Task { @MainActor in
            try? liveView.startScanning()
        }
        return liveView
    }

    func updateUIViewController(_ scannerController: DataScannerViewController, context: Context) {}

    func makeCoordinator() -> Tapper {
        Tapper(owner: self)
    }

    static func dismantleUIViewController(_ scannerController: DataScannerViewController, coordinator: Tapper) {
        scannerController.stopScanning()
    }

    @MainActor
    final class Tapper: NSObject, DataScannerViewControllerDelegate {
        private let owner: LiveScanner

        init(owner: LiveScanner) {
            self.owner = owner
        }

        func dataScanner(_ dataScanner: DataScannerViewController, didTapOn item: RecognizedItem) {
            switch item {
            case .text(let recognized):
                owner.lines.append(recognized.transcript)
            case .barcode(let code):
                if let value = code.payloadStringValue {
                    owner.payloads.append(value)
                }
            @unknown default:
                break
            }
        }
    }
}
```

`startScanning()` must wait until the controller is presented. The deferred
`Task` above runs after `makeUIViewController` returns; stopping belongs in
`dismantleUIViewController`.

A screen that offers the scanner only where it can run:

```swift
import VisionKit
import SwiftUI

struct ShelfScanScreen: View {
    @State private var scanning = false
    @State private var lines: [String] = []
    @State private var payloads: [String] = []

    var body: some View {
        Form {
            if DataScannerViewController.isSupported {
                Button("Scan labels") { scanning = true }
            } else {
                ContentUnavailableView("Scanning unavailable",
                                       systemImage: "camera.fill",
                                       description: Text("Live scanning needs a newer iPhone or iPad."))
            }
            Section("Text") { ForEach(lines, id: \.self, content: Text.init) }
            Section("Codes") { ForEach(payloads, id: \.self, content: Text.init) }
        }
        .fullScreenCover(isPresented: $scanning) {
            NavigationStack {
                LiveScanner(kinds: [.text(), .barcode()], lines: $lines, payloads: $payloads)
                    .ignoresSafeArea()
                    .toolbar {
                        ToolbarItem(placement: .cancellationAction) {
                            Button("Close") { scanning = false }
                        }
                    }
            }
        }
    }
}
```

## Custom overlay

Layer your own guidance on top of the scanner with a `ZStack`:

```swift
import VisionKit
import SwiftUI

struct FramedScanner: View {
    @State private var lines: [String] = []
    @State private var payloads: [String] = []

    var body: some View {
        ZStack {
            LiveScanner(kinds: [.barcode()], lines: $lines, payloads: $payloads)
                .ignoresSafeArea()
            VStack(spacing: 24) {
                Text("Fit the code inside the frame")
                    .padding(.horizontal, 16)
                    .padding(.vertical, 8)
                    .background(.ultraThinMaterial, in: Capsule())
                RoundedRectangle(cornerRadius: 16)
                    .stroke(.white.opacity(0.7), lineWidth: 3)
                    .frame(width: 260, height: 260)
                if let latest = payloads.last {
                    Text(latest)
                        .padding()
                        .background(.thinMaterial, in: .rect(cornerRadius: 12))
                }
            }
        }
    }
}
```

Sizes in these examples are placeholders; take them from the design system.

## VNDocumentCameraViewController

Presents a camera that fills the screen, snaps pages on its own, straightens
them, and collects several pages per session. It works on every device that runs iOS 13 or later.

A small helper used by both versions below:

```swift
import VisionKit

extension VNDocumentCameraScan {
    var allPages: [UIImage] {
        var collected: [UIImage] = []
        for index in 0..<pageCount {
            collected.append(imageOfPage(at: index))
        }
        return collected
    }
}
```

UIKit:

```swift
import UIKit
import VisionKit

@MainActor
final class PaperScanFlow: NSObject, @MainActor VNDocumentCameraViewControllerDelegate {
    private weak var host: UIViewController?
    var onPages: ([UIImage]) -> Void = { _ in }
    var onFailure: (Error) -> Void = { _ in }

    func start(from host: UIViewController) {
        self.host = host
        let camera = VNDocumentCameraViewController()
        camera.delegate = self
        host.present(camera, animated: true)
    }

    func documentCameraViewController(_ camera: VNDocumentCameraViewController,
                                      didFinishWith scan: VNDocumentCameraScan) {
        camera.dismiss(animated: true)
        onPages(scan.allPages)
    }

    func documentCameraViewControllerDidCancel(_ camera: VNDocumentCameraViewController) {
        camera.dismiss(animated: true)
    }

    func documentCameraViewController(_ camera: VNDocumentCameraViewController,
                                      didFailWithError error: Error) {
        camera.dismiss(animated: true)
        onFailure(error)
    }
}
```

`VNDocumentCameraViewControllerDelegate` carries no actor annotation, so the
conformance is written as `@MainActor VNDocumentCameraViewControllerDelegate`
(an isolated conformance, Swift 6.2 and later). The presenter is held `weak`
so the flow object never keeps a dismissed screen alive.

SwiftUI:

```swift
import VisionKit
import SwiftUI

struct PaperCamera: UIViewControllerRepresentable {
    @Binding var pages: [UIImage]
    @Environment(\.dismiss) private var close

    func makeUIViewController(context: Context) -> VNDocumentCameraViewController {
        let camera = VNDocumentCameraViewController()
        camera.delegate = context.coordinator
        return camera
    }

    func updateUIViewController(_ camera: VNDocumentCameraViewController, context: Context) {}

    func makeCoordinator() -> Receiver {
        Receiver(owner: self)
    }

    @MainActor
    final class Receiver: NSObject, @MainActor VNDocumentCameraViewControllerDelegate {
        private let owner: PaperCamera

        init(owner: PaperCamera) {
            self.owner = owner
        }

        func documentCameraViewController(_ camera: VNDocumentCameraViewController,
                                          didFinishWith scan: VNDocumentCameraScan) {
            owner.pages = scan.allPages
            owner.close()
        }

        func documentCameraViewControllerDidCancel(_ camera: VNDocumentCameraViewController) {
            owner.close()
        }

        func documentCameraViewController(_ camera: VNDocumentCameraViewController,
                                          didFailWithError error: Error) {
            owner.close()
        }
    }
}
```

### Scan then read

A model that runs OCR over every captured page, and a view that drives it:

```swift
import Observation
import SwiftUI
import Vision
import VisionKit

@MainActor
@Observable
final class PaperReader {
    var pages: [UIImage] = []
    var transcripts: [String] = []
    var busy = false

    func transcribe() async {
        busy = true
        defer { busy = false }
        var results: [String] = []
        for page in pages {
            guard let bitmap = page.cgImage else {
                results.append("(page could not be read)")
                continue
            }
            var request = RecognizeTextRequest()
            request.usesLanguageCorrection = true
            request.recognitionLanguages = [Locale.Language(identifier: "en-GB")]
            request.recognitionLevel = .accurate
            do {
                let found = try await request.perform(on: bitmap)
                let body = found.compactMap { $0.topCandidates(1).first }.map(\.string)
                results.append(body.joined(separator: "\n"))
            } catch {
                results.append("(recognition failed)")
            }
        }
        transcripts = results
    }
}

struct PaperReaderScreen: View {
    @State private var reader = PaperReader()
    @State private var capturing = false

    var body: some View {
        Form {
            Button("Scan pages") { capturing = true }
            if reader.busy {
                ProgressView()
            }
            ForEach(reader.transcripts.indices, id: \.self) { page in
                Section("Page \(page + 1)") {
                    Text(reader.transcripts[page]).textSelection(.enabled)
                }
            }
        }
        .sheet(isPresented: $capturing) {
            PaperCamera(pages: $reader.pages)
        }
        .onChange(of: reader.pages) {
            Task { await reader.transcribe() }
        }
    }
}
```

User-facing strings here are literals for brevity; ship them as localized keys.

## Performance

Live scanner:

- Use `.fast` when only barcodes are scanned.
- Set `recognizesMultipleItems = false` when one result is enough.
- Turn off `isHighFrameRateTrackingEnabled` for barcode-only scanners to save
  power.
- Ask only for the `recognizedDataTypes` the feature needs.
- Call `stopScanning()` while you process a result, and resume afterwards.

Document camera:

- Pages arrive as full-resolution `UIImage` values; downscale them if memory
  is tight.
- Process pages one after another rather than all at once, and wrap
  synchronous per-page loops in `autoreleasepool` when there are many.