# Rich Notifications

Two app extensions shape how a notification looks:

- **Notification Service Extension**: edits the content of a remote
  notification before the user sees it.
- **Notification Content Extension**: draws a custom interface when the user
  expands the notification.

## Contents

- [Service Extension](#service-extension)
- [Attachments](#attachments)
- [Content Extension](#content-extension)
- [Communication Notifications](#communication-notifications)
- [Troubleshooting Extensions](#troubleshooting-extensions)
- [A Complete Service Extension](#a-complete-service-extension)

## Service Extension

The extension is launched only for a remote notification that alerts the user
(its `alert` has a title, subtitle or body) and carries `mutable-content: 1`.
It then has about 30 seconds.

When a design gets this wrong, correct these four points:
1. A silent push never launches the extension.
2. Attachments are supported files on disk; the system validates them and
   moves them into its own storage.
3. A communication notification needs the capability, an
   `NSUserActivityTypes` entry, a donated `INInteraction`, and a call to
   `content.updating(from:)`.
4. The content handler runs exactly once on every path: success, partial
   failure, bad payload and `serviceExtensionTimeWillExpire()`.

If the handler is never called in time, the user sees the original,
unmodified notification.

### Creating it

Add a target in Xcode (File > New > Target) and pick the Notification Service
Extension template. It adds a `NotificationService` class that subclasses
`UNNotificationServiceExtension`.

The template's shape has a trap: the async work and the timeout callback can
both try to deliver. Guard delivery so only the first caller wins:

```swift
import UserNotifications

final class NotificationService: UNNotificationServiceExtension, @unchecked Sendable {
    private let lock = NSLock()
    private var handler: ((UNNotificationContent) -> Void)?
    private var draft: UNMutableNotificationContent?

    override func didReceive(
        _ request: UNNotificationRequest,
        withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void
    ) {
        let original = request.content
        guard let copy = original.mutableCopy() as? UNMutableNotificationContent else {
            contentHandler(original)
            return
        }
        lock.withLock {
            handler = contentHandler
            draft = copy
        }
        let headline = request.content.userInfo["headline"] as? String

        Task {
            let cleaned = await HeadlineFormatter.clean(headline)
            self.deliver { content in
                if let cleaned { content.title = cleaned }
            }
        }
    }

    override func serviceExtensionTimeWillExpire() {
        deliver()   // hand over whatever the draft holds now
    }

    /// Calls the content handler at most once, whichever path gets here first.
    private func deliver(_ edit: (UNMutableNotificationContent) -> Void = { _ in }) {
        let pending: (((UNNotificationContent) -> Void), UNMutableNotificationContent)? = lock.withLock {
            guard let handler, let draft else { return nil }
            self.handler = nil
            self.draft = nil
            return (handler, draft)
        }
        guard let (deliverTo, content) = pending else { return }
        edit(content)
        deliverTo(content)
    }
}
```

### Downloading media

```swift
enum MediaFetcher {
    static func attachment(from remote: URL) async -> UNNotificationAttachment? {
        let session = URLSession.shared
        guard let (bytes, reply) = try? await session.data(from: remote) else {
            return nil   // show the notification without media
        }
        let suffix = fileExtension(for: reply.mimeType)
        let local = FileManager.default.temporaryDirectory
            .appendingPathComponent(UUID().uuidString)
            .appendingPathExtension(suffix)
        do {
            try bytes.write(to: local)
            return try UNNotificationAttachment(identifier: "media", url: local, options: nil)
        } catch {
            return nil
        }
    }

    static func fileExtension(for mime: String?) -> String {
        switch mime {
        case "image/jpeg": "jpg"
        case "image/png": "png"
        case "image/gif": "gif"
        case "video/mp4", "video/mpeg4": "mp4"
        case "audio/mpeg", "audio/mp3": "mp3"
        case "audio/wav", "audio/x-wav": "wav"
        default: "jpg"
        }
    }
}
```

A failed download is not an error to the user: deliver the text without media.

### Decrypting the payload

For end-to-end encrypted messages the server sends a base64 ciphertext and the
extension decrypts it before display. If decoding or decryption fails, show
neutral text such as "New message" rather than nothing.

```swift
if let sealed = info["sealed"] as? String,
   let bytes = Data(base64Encoded: sealed),
   let key = try? SharedKeychain.messageKey(),
   let plain = try? MessageCrypto.open(bytes, using: key) {
    content.body = plain
} else {
    content.body = "New message"
}
```

The key belongs in a Keychain access group that the app and the extension both
use. Never put it in `UserDefaults`, and never in the push payload.

### Sharing data with the app

The extension is a separate process.

- **App Groups**: turn on the capability for both targets with the same
  identifier (for example `group.com.example.garden`). Use
  `UserDefaults(suiteName:)` for small values and
  `FileManager.default.containerURL(forSecurityApplicationGroupIdentifier:)` for
  files.
- **Keychain Sharing**: turn it on for both targets. The
  `kSecAttrAccessGroup` value has to appear in the target's list of keychain
  access groups; the App Group identifier does not work there.

Service extensions get little memory. Keep dependencies small; if the system
kills the extension, the original notification is shown.

## Attachments

`UNNotificationAttachment` handles pictures (GIFs included), sound and movies. The collapsed
notification shows a thumbnail; the expanded one shows the media.

| Kind | Size limit | Formats |
|------|------------|---------|
| Image | 10 MB | JPEG, GIF, PNG |
| Audio | 5 MB | WAV, AIFF, M4A, MP3 |
| Video | 50 MB | MPEG-4, MPEG-2, MPEG, AVI |

Thumbnail options:

```swift
let options: [AnyHashable: Any] = [
    // Normalized rect: show the centre half of the image as the thumbnail.
    UNNotificationAttachmentOptionsThumbnailClippingRectKey:
        CGRect(origin: CGPoint(x: 0.25, y: 0.25), size: CGSize(width: 0.5, height: 0.5))
            .dictionaryRepresentation,
    // For video: the frame at 3 seconds.
    UNNotificationAttachmentOptionsThumbnailTimeKey: 3
]
```

Rules:
- The URL points to a supported file on disk. A remote URL or an unsupported
  type fails.
- For a local notification, the file must be readable when you schedule it.
- The system validates the file and moves it into its own data store. Files
  inside the app bundle are copied instead.
- Several attachments are allowed; the first one is the thumbnail.
- GIFs can animate in the expanded view; check the real presentation on a
  device.

## Content Extension

A content extension supplies a view controller that appears when the user
long-presses or expands the notification. Use it when an attachment is not
enough.

Add it the same way (File > New > Target), choosing the Notification Content
Extension template; Xcode adds
a storyboard and a `NotificationViewController`.

### Info.plist

```xml
<key>NSExtension</key>
<dict>
    <key>NSExtensionAttributes</key>
    <dict>
        <!-- One category as a string, or several as an array -->
        <key>UNNotificationExtensionCategory</key>
        <array>
            <string>ORDER_UPDATE</string>
            <string>DELIVERY_PHOTO</string>
        </array>
        <!-- Height as a fraction of width; 1.0 when omitted -->
        <key>UNNotificationExtensionInitialContentSizeRatio</key>
        <real>0.6</real>
        <!-- Hide the standard title/body under the custom view -->
        <key>UNNotificationExtensionDefaultContentHidden</key>
        <true/>
        <!-- Let the custom view receive touches -->
        <key>UNNotificationExtensionUserInteractionEnabled</key>
        <true/>
    </dict>
    <key>NSExtensionMainStoryboard</key>
    <string>MainInterface</string>
    <key>NSExtensionPointIdentifier</key>
    <string>com.apple.usernotifications.content-extension</string>
</dict>
```

### The view controller

```swift
import UIKit
import UserNotifications
import UserNotificationsUI

final class NotificationViewController: UIViewController, @preconcurrency UNNotificationContentExtension {
    @IBOutlet private var statusLabel: UILabel!
    @IBOutlet private var photoView: UIImageView!

    func didReceive(_ notification: UNNotification) {
        let content = notification.request.content
        statusLabel.text = content.body

        if let file = content.attachments.first?.url, file.startAccessingSecurityScopedResource() {
            defer { file.stopAccessingSecurityScopedResource() }
            photoView.image = UIImage(contentsOfFile: file.path)
        }
    }

    func didReceive(_ response: UNNotificationResponse,
                    completionHandler completion: @escaping (UNNotificationContentExtensionResponseOption) -> Void) {
        switch response.actionIdentifier {
        case "CONFIRM_DELIVERY":
            statusLabel.text = "Thanks, noted."
            Task {
                try? await Task.sleep(for: .milliseconds(900))
                completion(.dismiss)
            }
        default:
            completion(.dismissAndForwardAction)
        }
    }
}
```

The conformance is marked `@preconcurrency` because the protocol has no
concurrency annotations while the view controller is main-actor isolated; the
system drives a content extension on the main thread.

Attachment files are security scoped inside the extension: wrap access in
`startAccessingSecurityScopedResource()` and
`stopAccessingSecurityScopedResource()`.

### Response options

| Option | Result |
|--------|--------|
| `.doNotDismiss` | Stays on screen; for multi-step interactions |
| `.dismiss` | Closes; the extension handled the action |
| `.dismissAndForwardAction` | Closes and passes the action to the app's `didReceive` |

### Media playback

For audio or video, ask the system for its play/pause button and respond to it.

```swift
import AVFoundation

final class ClipViewController: UIViewController, @preconcurrency UNNotificationContentExtension {
    private var player: AVPlayer?

    var mediaPlayPauseButtonType: UNNotificationContentExtensionMediaPlayPauseButtonType { .overlay }
    var mediaPlayPauseButtonFrame: CGRect { CGRect(x: 16, y: 16, width: 44, height: 44) }

    func didReceive(_ notification: UNNotification) {
        guard let url = notification.request.content.attachments.first?.url else { return }
        let player = AVPlayer(url: url)
        let layer = AVPlayerLayer(player: player)
        layer.frame = view.bounds
        view.layer.addSublayer(layer)
        self.player = player
    }

    func mediaPlay() { player?.play() }
    func mediaPause() { player?.pause() }
}
```

`mediaPlayPauseButtonType` can be `.none`, `.default` or `.overlay`.

### Interactive views

With `UNNotificationExtensionUserInteractionEnabled` set, buttons and gesture
recognizers in the custom view work like anywhere else in UIKit:

```swift
private func makeReactionBar(for orderID: String) -> UIStackView {
    let reactions = ["Great", "Okay", "Late"].map { title in
        UIButton(configuration: .tinted(), primaryAction: UIAction(title: title) { _ in
            Task { await FeedbackAPI.submit(orderID: orderID, reaction: title) }
        })
    }
    let bar = UIStackView(arrangedSubviews: reactions)
    bar.axis = .horizontal
    bar.distribution = .fillEqually
    bar.spacing = 8
    return bar
}
```

## Communication Notifications

These show the sender's name and picture and are built on the SiriKit intents
`INSendMessageIntent` and `INStartCallIntent`. They change how Focus and the
notification summary treat the app, so use them only for messages or calls
between real people.

Setup:
1. Add the Communication Notifications capability to the app target.
2. List the intent classes (for example `INSendMessageIntent`) under
   `NSUserActivityTypes` in the app's Info.plist.
3. Link the `Intents` framework into the service extension.
4. In the extension: build the intent, wrap it in
   `INInteraction(intent:response:)`, set `direction = .incoming`, call
   `donate()`, then pass `content.updating(from:)` to the handler.

```swift
import Intents

enum SenderStyling {
    static func apply(to content: UNNotificationContent,
                      senderID: String,
                      senderName: String,
                      avatar: Data?,
                      conversationID: String) async -> UNNotificationContent {
        let sender = INPerson(
            personHandle: INPersonHandle(value: senderID, type: .unknown),
            nameComponents: PersonNameComponentsFormatter().personNameComponents(from: senderName),
            displayName: senderName,
            image: avatar.map { INImage(imageData: $0) },
            contactIdentifier: nil,
            customIdentifier: senderID
        )
        let intent = INSendMessageIntent(
            recipients: nil,
            outgoingMessageType: .outgoingMessageText,
            content: content.body,
            speakableGroupName: nil,
            conversationIdentifier: conversationID,
            serviceName: nil,
            sender: sender,
            attachments: nil
        )
        let donation = INInteraction(intent: intent, response: nil)
        donation.direction = .incoming
        try? await donation.donate()

        do {
            let styled = try content.updating(from: intent)
            return styled
        } catch {
            return content   // fall back to the plain notification
        }
    }
}
```

### In the service extension

Attach media first. Then, if the payload names a sender, run the styling
above and deliver its result; if not, deliver the plain content. The
[complete example](#a-complete-service-extension) shows the whole flow.

## Troubleshooting Extensions

**Service extension never runs**
- The payload includes an alert and sets `mutable-content: 1`.
- The extension is embedded in the app and has a valid bundle ID and profile.
- Both targets carry the shared App Group and Keychain entitlements they use.

**Content extension never appears**
- `UNNotificationExtensionCategory` matches the notification's
  `categoryIdentifier` exactly.
- The storyboard or view is actually set up, and the size ratio is sensible.

**Memory**
- The system kills extensions that use too much memory. Keep dependencies
  small and use `URLSession` directly rather than a large networking library.
  A service extension has no UI, so a UI framework buys nothing there; a
  content extension may host SwiftUI through `UIHostingController`, but keep
  it light.

**Fallbacks**
- The handler runs exactly once. If a download, decryption, donation or
  `updating(from:)` fails, deliver the original or the best attempt so far.
- In `serviceExtensionTimeWillExpire()`, stop waiting and deliver the best
  content right away.

**Network**
- Extensions may use the network; a `URLSession` with the `.default`
  configuration is fine. Stay well inside the roughly 30-second budget and
  fall back to the payload text on failure.

**Debugging**
- Run the extension's scheme, or attach from the Debug menu (Attach to
  Process by PID or Name) using the extension's name.
- Log with `os_log` or `Logger` and filter Console.app by the extension's
  bundle ID.
- Set breakpoints, then send a notification to trigger the extension.

## A Complete Service Extension

The steps run in order: decrypt the body with a key from the shared Keychain,
attach media, style as a communication notification, then deliver once. The
timeout path delivers whatever is ready.

```swift
import Intents
import UserNotifications

final class NotificationService: UNNotificationServiceExtension, @unchecked Sendable {
    private let lock = NSLock()
    private var handler: ((UNNotificationContent) -> Void)?
    private var bestAttempt: UNNotificationContent?

    override func didReceive(
        _ request: UNNotificationRequest,
        withContentHandler contentHandler: @escaping (UNNotificationContent) -> Void
    ) {
        lock.withLock {
            handler = contentHandler
            bestAttempt = request.content
        }
        let payload = IncomingPayload(request.content.userInfo)

        Task {
            guard let draft = self.snapshot()?.mutableCopy() as? UNMutableNotificationContent else {
                self.finish(with: nil)
                return
            }

            // 1. Decrypt, falling back to neutral text.
            draft.body = payload.decryptedBody() ?? "New message"
            self.remember(draft)

            // 2. Media, skipped silently on failure.
            if let url = payload.mediaURL, let media = await MediaFetcher.attachment(from: url) {
                draft.attachments = [media]
                self.remember(draft)
            }

            // 3. Sender styling, only when the payload names a sender.
            guard let sender = payload.sender else {
                self.finish(with: draft)
                return
            }
            let styled = await SenderStyling.apply(
                to: draft,
                senderID: sender.id,
                senderName: sender.name,
                avatar: await payload.avatarData(),
                conversationID: sender.conversationID
            )
            self.finish(with: styled)
        }
    }

    override func serviceExtensionTimeWillExpire() {
        finish(with: nil)   // best attempt so far
    }

    private func snapshot() -> UNNotificationContent? {
        lock.withLock { bestAttempt }
    }

    private func remember(_ content: UNNotificationContent) {
        lock.withLock { if handler != nil { bestAttempt = content.copy() as? UNNotificationContent } }
    }

    /// Single delivery point. `nil` means "use the best attempt".
    private func finish(with ready: UNNotificationContent?) {
        let delivery: (((UNNotificationContent) -> Void), UNNotificationContent)? = lock.withLock {
            guard let handler, let fallback = bestAttempt else { return nil }
            self.handler = nil
            return (handler, ready ?? fallback)
        }
        guard let (deliverTo, result) = delivery else { return }
        deliverTo(result)
    }
}
```

`IncomingPayload` is the app's own parser over `userInfo`: it reads the sealed
body and decrypts it with the shared-Keychain key, and exposes `mediaURL`, the
sender fields and an avatar download. Keep it free of UI and large
dependencies.
