/** * useStitcherWorklet — exposes the lib's first-party stitching as a * callable worklet function for host-composed Frame Processors. * * v0.11.0 — closes the v0.8.0 Phase 5 either-or constraint by letting * hosts COMPOSE: write ONE `useFrameProcessor` worklet body that calls * BOTH your custom logic AND the lib's first-party stitching, instead * of one displacing the other. See `docs/host-app-integration.md` * § Tier 3 composition for the pattern. * * ## Why this is a separate hook * * vision-camera v4 lets a `` mount accept exactly ONE frame * processor. Pre-v0.11.0, hosts that passed a `frameProcessor` prop * to the lib's `` REPLACED the lib's first-party stitching * processor in non-AR mode. Composing required hand-writing both * worklet bodies in the host's processor. v0.11.0 extracts the * lib's worklet body into this hook so hosts can compose with a * single call: * * const stitcher = useStitcherWorklet(); * const fp = useFrameProcessor((frame) => { * 'worklet'; * hostPreLogic(frame); * stitcher.call(frame); // ← lib's first-party stitching * hostPostLogic(frame); * }, [stitcher.call]); * return ; * * AR mode is unaffected — the AR-session dispatch path (v0.8.0 Phase * 4b.i / 4b.iii) already composes natively. * * ## What this owns * * - vc Frame Processor plugin acquisition for * `cv_flow_gate_process_frame` (the same plugin the legacy * `useFrameProcessorDriver` used; reentrant by construction). * - Shared values backing pose (yaw / pitch / roll), throttle * counter, every-N gate, and FoV-derived intrinsics scalars. * - Gyro subscription on the JS thread (always-on between mount * and unmount; subscription cost is tiny). * - The worklet body itself: throttle → pose synthesis → * `plugin.call(frame, params)`. * * ## Lifecycle * * - Gyro auto-subscribes on mount, auto-unsubscribes on unmount. * Composed hosts get pose tracking for free. * - `reset()` zeros the accumulated yaw / pitch / roll between * captures. `useFrameProcessorDriver` calls this on `start()` to * preserve pre-v0.11.0 per-capture pose-reset behaviour; * composed hosts should call it at the start of each capture too * (otherwise pose drifts across captures). * * ## Behaviour delta from pre-v0.11.0 * * Before: `useFrameProcessorDriver.start()` subscribed the gyro; * `stop()` unsubscribed. The subscription was tied to the * capture lifecycle. * * After: the gyro is subscribed for the lifetime of this hook * (i.e., as long as the component using it is mounted). In the * default `` integration the hook mounts when the camera * screen mounts, so the practical effect is the same; in * custom-composed integrations the host controls mount/unmount * by mounting/unmounting the component that calls * `useStitcherWorklet`. The battery delta is small: gyroscope * sampling at 33ms costs ≪1% CPU on every Android/iOS device * the lib supports. * * `pose reset` semantics are preserved via the new explicit * `reset()` method. Hosts that previously relied on `start()` * to zero pose now call `stitcher.reset()` at the capture start. * * ## Pose synthesis (verbatim from `useFrameProcessorDriver`) * * Quaternion: q = q_yaw * q_pitch * q_roll (Tait-Bryan YPR, body * frame). Expanded: * qx = cy*sp*cr + sy*cp*sr * qy = sy*cp*cr - cy*sp*sr * qz = cy*cp*sr - sy*sp*cr * qw = cy*cp*cr + sy*sp*sr * * When roll=0 this collapses to the legacy 2-axis form so captures * held level produce bit-identical poses to the pre-v0.6 driver * (and bit-identical to v0.10.x's `useFrameProcessorDriver`). * * ## Throttling (verbatim) * * `evalEveryNFrames` controls how often the worklet calls the * plugin. Default 1. Independent of — and stacks on top of — * the stitcher's own internal `flowEvalEveryNFrames` in * `KeyframeGate.swift`; effective cadence is the product. * * ## Pairing with `IncrementalStitcher.start` * * The plugin's per-frame call into `consumeFrameFromPlugin` is * gated by `IncrementalStitcher.frameProcessorIngestEnabled`, * which is TRUE only when the stitcher was started with * `frameSourceMode === 'frameProcessor'`. Hosts MUST call * `incrementalStitcher.start({ frameSourceMode: 'frameProcessor', * ... })` to actually get frames into the engine — otherwise the * worklet runs to completion but the wrapper drops the call. * `Camera.tsx` does this wiring automatically when the host opts * into the lib's `useFrameProcessorDriver`. Hosts that compose * their own worklet via this hook must do the wiring themselves. */ import type { Frame } from 'react-native-vision-camera'; import type { CameraFrame } from './CameraFrame'; /** * Frames the lib's stitching worklet accepts. Accepting either a * vc `Frame` (what the host's `useFrameProcessor` body sees) or the * lib's `CameraFrame` (what the lib's `useFrameProcessor` body * sees) keeps the same `useStitcherWorklet` usable from both kinds * of host worklet bodies without a cast on the call site. The * worklet only reads `width` / `height`; the rest of the frame * object is forwarded verbatim to the native plugin. */ export type StitcherWorkletInput = Frame | CameraFrame; export interface UseStitcherWorkletOptions { /** * Gyro sample interval in ms (~30 Hz default). Drives the JS- * thread pose integration loop; not the producer-thread plugin * call rate. */ gyroIntervalMs?: number; /** * Approximate horizontal FoV of the device camera, used to * synthesise `fx` from frame width. Default 65° matches a typical * mid-tier smartphone main camera. */ fovHorizDegrees?: number; /** * Approximate vertical FoV of the device camera, used to * synthesise `fy` from frame height. Default 50° matches a typical * 4:3 phone camera in landscape; for 16:9 portrait you probably * want ~75°. */ fovVertDegrees?: number; /** * Evaluate the plugin every Nth producer-thread frame. Default 1 * (every frame). Clamped to [1, 10] to match native's cadence clamp * (IncrementalStitcher.kt `evalCadence.coerceIn(1, 10)`) so an out-of-range host setting * keeps its effective cadence once the throttle runs JS-side. */ evalEveryNFrames?: number; /** * perf-3a change 1 — initial ingest-gate state. `true` (default) = * gate OPEN, so a bare `useStitcherWorklet()` behaves exactly as before * (every frame runs pose synthesis + `plugin.call`). Managed * integrations (`useFrameProcessorDriver`) pass `false` and drive the * gate via `setActive()` so idle / stitch-phase frames cost only a * shared-value read instead of a full JSI→JNI plugin dispatch. */ initialIngestActive?: boolean; } export interface StitcherWorkletHandle { /** * Worklet function: pass a `CameraFrame` to perform one frame of * the lib's first-party stitching (throttle + pose synthesis + * native plugin call). Safe to call from inside another * `'worklet'`-prefixed function (this is the canonical * composition pattern). * * The returned function reference is stable across re-renders as * long as the plugin reference doesn't change (which happens at * most once — at the moment the JSI plugin finishes * registering). Include `stitcher.call` in your `useFrameProcessor` * deps so the host worklet rebuilds when the plugin acquires. * * Safe to invoke before the plugin is ready: the worklet * internally short-circuits (the frame is silently skipped). * Hosts that want to display a "stitcher initialising…" UI can * read `isReady` to gate their own behaviour. */ call: (frame: StitcherWorkletInput) => void; /** * Zero accumulated yaw / pitch / roll. Call at the start of each * capture so the pose stream starts from `(0, 0, 0)` instead of * carrying drift from the previous capture or from idle time * between captures. Idempotent; safe to call from JS. */ reset: () => void; /** * perf-3a change 1 — open/close the ingest gate. While closed, `call` * returns after a single shared-value read (no pose synthesis, no * plugin dispatch) and the gyro handler skips its accumulator writes. * The native `AtomicBoolean` fast-exit stays authoritative; this is the * cheap JS-side gate and may lag native by a frame or two. Bare-hook * users leave the gate open (`initialIngestActive` default `true`); * `useFrameProcessorDriver` drives it from `start()`/`stop()`. */ setActive: (active: boolean) => void; /** * perf-3a change 2 — zero ONLY the decimation frame counter (not pose), * re-anchoring the every-Nth grid. Managed drivers call this right after * the native `start()` await so the grid anchors at native-ingest-enable * (frame-identical decimation) despite the gate opening before the await. */ resetCadence: () => void; /** * `true` once the JSI Frame Processor plugin * (`cv_flow_gate_process_frame`) has resolved. Before this flips * `true`, `call(frame)` is a no-op (the plugin reference is * `null`). Hosts integrating via `useFrameProcessorDriver` use * this to decide whether to render the frame-processor at all — * the driver returns `null` for `frameProcessor` until ready, so * `` falls back gracefully. */ isReady: boolean; /** * v0.24.3 — `true` once the SDK has determined the * `cv_flow_gate_process_frame` plugin can NEVER be acquired in this * build (vision-camera reported frame processors disabled, or the * plugin never registered within ~3 s). Distinguishes a permanent * build defect from the normal ~1-frame acquisition window, so callers * can fail a non-AR capture fast instead of running one that cannot * ingest frames. See the console.error this flag is set alongside. */ acquisitionFailed: boolean; } export declare function useStitcherWorklet(options?: UseStitcherWorkletOptions): StitcherWorkletHandle; //# sourceMappingURL=useStitcherWorklet.d.ts.map