openapi: 3.0.3 info: title: Repzo API - AI Object Detection Session Analysis version: 1.0.0 description: | **Session analysis** is the cross-frame fusion stage of the Object Detection Sessions feature. Given a capture `session` (a stream of frames materialized as `ai-object-detection-task` rows) plus a `model_version`, it produces a versioned, audit-friendly analysis document with the fused 3D objects on the shelf and a kept/ignored breakdown per label. **Asynchronous.** `create` is non-blocking: it opens a `pending` analysis document and returns it **immediately** (with a `message`), then runs the heavy work in the background. Inferring every frame can take a while, so the client should not wait on the request — poll the returned document (or list analyses for the session) until `status` flips to `success`/`failed`. On completion the backend emits `inferred-od-session`, which notifies reps (`update-object-detection-task`) so the dashboard can refresh. **What the background job does.** 1. Loads the session and all its tasks, sorted in CAPTURE order (`frame_meta.frame_id`, then device timestamp). 2. Infers any task not yet placed — delegates to the per-task `ai-object-detection-inference` path with the given `model_version` (or the zero-shot VLM when `engine` requests it / nothing is trained). Already-inferred tasks are reused as-is. 3. Collects annotations FRAME BY FRAME; un-placeable ones become `unplaced` ledger entries carrying the placement failure reason. 4. **MERGE (frame walk)** — walks the frames in capture order: detections matching an object seen in an EARLIER frame (same label, within the merge radius, one per object per frame) merge into it as re-observations; the rest spawn new objects. Same-frame drops happen only for explicit quality edge cases (duplicate box, same world spot, low confidence, implausible size) — a frame's own detection count is authoritative, so two adjacent identical facings stay distinct. 5. **ARBITRATION** — scores every member of a cluster with the weighted 6-factor view score (detector confidence, depth confidence, tracking, box centering, box size, label agreement) and picks the best VIEW as the winner; the full per-member breakdown is persisted. 6. Persists `objects[]` (kept + merged), `concluded_labels[]` (per-label kept/merged/ignored/dropped counts), the per-detection `detections[]` ledger and the `funnel` summary, then emits `inferred-od-session`. **Storage & multi-tenancy.** Each run is a new document in the `ai.objectDetectionSessionAnalyses` collection, scoped by `company_namespace` (injected from the caller's token) and soft-deleted via `disabled: true`. The run is opened with `status: pending`, flips to `in_progress` when the background job starts, then to `success` (or `failed` with `_errors`) on completion. References `aiObjectDetectionSession`, `ai.objectDetectionModelVersions`, and `sv.aiObjectDetectionLabels`. **Who calls it.** Admins / reps with a valid JWT. `create` triggers analysis (or, with `recompute: compose`, synchronously recomposes the shelf structure of an existing analysis); `find` / `get` are standard reads; `update` (PUT) is a raw document update with a server-stamped `editor`; `patch` is the bulk `writeQuery` update over the find filters; `remove` soft-deletes. **Population.** `?populatedKeys[]=` supports `session`, `model_version`, `objects.label_id`, `objects.winning_image_media` and `concluded_labels.label_id`; every key is populated IN PLACE (the id is replaced by the referenced document). servers: - url: https://sv.api.repzo.me security: - ApiKeyAuth: [] - JwtAuth: [] paths: /ai-object-detection-session-analysis: get: summary: List session analyses description: Paginated list of analysis documents, newest first. operationId: findAiObjectDetectionSessionAnalysis parameters: - in: query name: _id description: Filter by document `_id`. schema: oneOf: - type: string - type: array items: { type: string } - in: query name: session description: Filter by the parent session `_id` (one or many). schema: oneOf: - type: string - type: array items: { type: string } - in: query name: model_version description: Filter by the model version used (one or many). schema: oneOf: - type: string - type: array items: { type: string } - in: query name: status description: Filter by run status (one or many). schema: oneOf: - type: string enum: [pending, in_progress, success, failed] - type: array items: type: string enum: [pending, in_progress, success, failed] - in: query name: creator._id description: Filter by the id of the rep/admin who triggered the analysis. schema: oneOf: - type: string - type: array items: { type: string } - in: query name: from_createdAt description: Return analyses created at/after this Unix timestamp (ms). schema: { type: number } - in: query name: to_createdAt description: Return analyses created at/before this Unix timestamp (ms). schema: { type: number } - in: query name: from_updatedAt description: Cursor — analyses with `updatedAt` greater than this Unix timestamp (ms). schema: { type: number } - in: query name: to_updatedAt description: Cursor — analyses with `updatedAt` less than this Unix timestamp (ms). schema: { type: number } - in: query name: disabled description: Include disabled (soft-deleted) analyses. Defaults to `false`. schema: { type: boolean, default: false } - in: query name: per_page description: Page size. Defaults to the server's configured pagination limit. schema: { type: integer, minimum: 1, maximum: 500 } example: 50 - in: query name: page description: 1-indexed page number. schema: { type: integer, minimum: 1 } example: 1 - in: query name: sort description: Field to sort by. Defaults to `_id`. schema: { type: string, default: _id } - in: query name: sortPageOrder description: Sort direction. Defaults to descending. schema: { type: string, enum: [asc, dsc], default: dsc } - in: query name: populatedKeys description: | Embed referenced documents (populated in place). Supported: `session`, `model_version`, `objects.label_id`, `objects.winning_image_media`, `concluded_labels.label_id`. schema: type: array items: type: string enum: - session - model_version - objects.label_id - objects.winning_image_media - concluded_labels.label_id responses: "200": description: Paginated list of analysis documents (standard envelope). content: application/json: schema: $ref: "#/components/schemas/AnalysisFindResult" post: summary: Start cross-frame analysis on a session (async) description: | Opens a `pending` analysis document and returns it immediately, then infers any not-yet-placed tasks, clusters the kept world detections, and arbitrates a winner per cluster **in the background**. Poll the returned document (or list analyses for the session) until `status` becomes `success`/`failed`. **Model-version semantics.** When `model_version` is given, tasks whose auto annotations came from a *different* version are re-inferred with the requested one; tasks already inferred by the same version are reused. Without `model_version`, any existing auto annotations are reused as-is. **Concurrency & staleness.** Only one live run per session: creating an analysis while another is `pending`/`in_progress` returns `400`. Zombie runs (a server restart lost the background job) are detected via `heartbeat_at` — any run without a heartbeat for 15 minutes is automatically marked `failed` by the next create, so a stuck run never blocks the session permanently. **Side effects on success.** The parent session's `status` transitions `infer_in_progress → inferred` (or `failed`), its `detections_count` / `objects_count` aggregates are refreshed, each contributing task annotation gets its `cluster_id` set to the concluded object's `_id` (stale links from earlier runs are cleared first), and best-effort scene outputs are stored: the dominant plane (`scene.plane`), the RANSAC shelf planes (`scene.planes`), and the voxel point cloud (`scene.point_cloud`, packed blob in `bin` media — disable with `config.build_point_cloud: false`). operationId: createAiObjectDetectionSessionAnalysis requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/AnalysisRequest" responses: "201": description: | The freshly-created analysis document with `status: pending` plus a `message`. The heavy work runs in the background — the fused `objects[]` / `concluded_labels[]` are filled in later, once the status flips to `success`. content: application/json: schema: allOf: - $ref: "#/components/schemas/SessionAnalysis" - type: object properties: message: type: string description: Human-readable note that the job started in the background. "400": description: | The session has no frames/tasks, or a live (non-stale) analysis is already running for it — wait for that run to finish or for the 15-minute stale watchdog to fail it. patch: summary: Bulk-update analyses description: | Bulk update via the `patch-action` writeQuery shape. The rows to update are selected by the SAME query filters as the list endpoint (`_id`, `session`, `model_version`, `status`, `creator._id`, `from_/to_createdAt`, `from_/to_updatedAt`, `disabled`). Returns `{ nFound, nModified }`. operationId: patchAiObjectDetectionSessionAnalysis parameters: - in: query name: _id schema: oneOf: - type: string - type: array items: { type: string } - in: query name: session schema: oneOf: - type: string - type: array items: { type: string } - in: query name: model_version schema: oneOf: - type: string - type: array items: { type: string } - in: query name: status schema: oneOf: - type: string - type: array items: { type: string } - in: query name: creator._id schema: oneOf: - type: string - type: array items: { type: string } - in: query name: from_createdAt schema: { type: number } - in: query name: to_createdAt schema: { type: number } - in: query name: from_updatedAt schema: { type: number } - in: query name: to_updatedAt schema: { type: number } - in: query name: disabled schema: { type: boolean } requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/PatchActionBody" responses: "200": description: Bulk-update result. content: application/json: schema: $ref: "#/components/schemas/PatchActionResult" /ai-object-detection-session-analysis/{id}: get: summary: Get one session analysis operationId: getAiObjectDetectionSessionAnalysis parameters: - in: path name: id required: true schema: { type: string } - in: query name: populatedKeys description: Same population keys as the list endpoint (populated in place). schema: type: array items: type: string enum: - session - model_version - objects.label_id - objects.winning_image_media - concluded_labels.label_id responses: "200": description: The analysis document. content: application/json: schema: $ref: "#/components/schemas/SessionAnalysis" put: summary: Update one session analysis description: | Raw document update (`updateOne` with the body; `editor` is server-stamped). Intended for minimal edits such as `disabled: true` (soft-delete) — overwriting run output fields would corrupt the analysis. The tenant key is derived from the caller's session. operationId: updateAiObjectDetectionSessionAnalysis parameters: - in: path name: id required: true schema: { type: string } requestBody: required: true content: application/json: schema: $ref: "#/components/schemas/AnalysisUpdateBody" responses: "200": description: The analysis document after the update is applied. content: application/json: schema: $ref: "#/components/schemas/SessionAnalysis" "404": description: No analysis with that id in the caller's namespace. delete: summary: Soft-delete one session analysis operationId: removeAiObjectDetectionSessionAnalysis parameters: - in: path name: id required: true schema: { type: string } responses: "200": description: The analysis document after `disabled` is set to true. content: application/json: schema: $ref: "#/components/schemas/SessionAnalysis" components: securitySchemes: ApiKeyAuth: type: apiKey in: header name: api-key description: | Server-issued API key. Also accepted via the `x-api-key` header or the `?apiKey=` query parameter as fallbacks. JwtAuth: type: apiKey in: header name: Authorization description: | Raw JWT in the `Authorization` header — **no `Bearer ` prefix**. Obtained from `POST /authenticate` (admin / rep / client login). schemas: AnalysisRequest: type: object required: - session properties: session: type: string description: The `ai-object-detection-session` _id to analyze. model_version: type: string description: | `ai.objectDetectionModelVersions` _id used to infer any not-yet-placed tasks. model: type: string description: Optional object-detection model `_id` override. engine: type: string enum: [auto, trained, zero_shot] default: auto description: | Detector for tasks inferred during this run. `trained` uses the Ultralytics lambda; `zero_shot` uses a hosted Qwen-VL prompted with the namespace's label names (for sessions with no trained model yet); `auto` (default) tries trained and falls back to zero-shot when nothing trained resolves. zero_shot_model: type: string description: | Zero-shot VLM override, e.g. `qwen/qwen3-vl-8b-instruct`. Must be one of the supported models (see `ai-object-detection-inference`). conf: type: number description: Optional detector confidence override. iou: type: number minimum: 0 maximum: 1 description: Optional detector NMS IoU override. agnostic_nms: type: boolean description: Optional class-agnostic NMS override. force_reinference: type: boolean default: false description: | Re-infer EVERY task even if it already has an auto annotation group for this `model_version`. Normal reuse keys on the group's stored `model_version`; if an earlier run stamped that version but the boxes were produced by a different net (stale provenance), those boxes would be reused forever. Set this once to overwrite them. inference_concurrency: type: integer minimum: 1 maximum: 10 default: 4 description: | Parallel inference calls for this run. Each call runs on its own lambda instance (~30 s/frame on CPU), so a session's inference wall-clock divides by this. `1` restores serial behavior; capped at 10 until GPU inference lands. May also ride inside `config` (the dashboard dialog and category model_settings put it there); this top-level field wins when both are present. recompute: type: string enum: [compose] description: | STAGE RECOMPUTE — "compose" re-runs ONLY the shelf composition of the existing analysis given in `analysis`, synchronously and in place, using the `shelf_*` keys from `config` merged over the stored config (non-shelf keys in `config` are IGNORED on this path — accepting them would update their stage fingerprints without recomputing those stages). Fused objects, ledger and funnel are untouched (their config slices didn't change). Milliseconds instead of a full run — the path a shelf-knob change takes. Returns the updated analysis document directly (no background job). analysis: type: string description: The existing analysis _id to recompute (required with `recompute`). config: $ref: "#/components/schemas/SceneMathConfig" company_namespace: type: array items: { type: string } description: Optional tenant namespace override for SDK callers. AnalysisUpdateBody: type: object description: | Body for `PUT /{id}` — any stored field; `editor` is server-stamped. Set `disabled: true` to soft-delete. properties: disabled: { type: boolean } config: $ref: "#/components/schemas/SceneMathConfig" status: type: string enum: [pending, in_progress, success, failed] additionalProperties: true PatchActionWrite: type: object properties: key: { type: string } command: type: string enum: [set, addToSet, pull] value: {} PatchActionBody: type: object description: Bulk-update body. `writeQuery[]` describes the writes applied to filtered rows. required: [writeQuery] properties: writeQuery: type: array items: $ref: "#/components/schemas/PatchActionWrite" PatchActionResult: type: object properties: nFound: { type: number } nModified: { type: number } AnalysisFindResult: type: object description: Standard paginated result envelope. properties: data: type: array items: $ref: "#/components/schemas/SessionAnalysis" total_result: { type: number } current_count: { type: number } total_pages: { type: number } current_page: { type: number } per_page: { type: number } first_page_url: { type: string } last_page_url: { type: string } next_page_url: { type: string, nullable: true } prev_page_url: { type: string, nullable: true } path: { type: string } SceneMathConfig: type: object description: Optional clustering / back-projection tuning. Omit for defaults. properties: inference_concurrency: type: integer minimum: 1 maximum: 10 default: 4 description: | Run-level alias of the top-level `inference_concurrency`, carried inside `config` by the dashboard's analyze dialog and by category `model_settings` (the category auto-analysis forwards only `config`). The top-level field wins when both are present. cluster_eps_m: type: number default: 0.08 description: Base world distance to merge detections into one object. class_agree_bonus_m: type: number default: 0.02 description: Extra merge radius when labels match. block_same_frame: type: boolean default: true description: | A scene object absorbs at most ONE detection per frame during the walk — each frame's detection count stays authoritative for how many objects exist there. same_frame_iou_thresh: type: number default: 0.92 description: | Same-frame quality gate — two same-label boxes with 2D IoU at/above this are a detector double-fire; the weaker one is dropped (`duplicate_box_in_frame`). same_frame_min_separation_m: type: number default: 0.02 description: | Same-frame quality gate — two same-label detections placed closer than this in world coords cannot be two physical objects; the weaker one is dropped (`same_spot_in_frame`). min_detection_confidence: type: number default: 0 description: | Same-frame quality gate — drop detections whose detector confidence is below this floor (`below_min_confidence`). 0 disables the gate. min_object_size_cm: type: number default: 0.5 description: Same-frame quality gate — drop implausibly small front faces. max_object_size_cm: type: number default: 500 description: Same-frame quality gate — drop implausibly large front faces. plane_merge: type: boolean default: true description: | Cross-frame matcher: project every placed detection onto the session's consensus shelf plane (covariance normal over detection positions, gravity-flattened; offset = median projection) and merge same-label detections whose projected rects overlap at the same plane depth. Falls back to the `cluster_eps_m` distance rule when the plane is degenerate (< 3 detections, collinear stack) or a detection has no world size. plane_merge_iou: type: number default: 0.1 description: | Minimum IoU of the two plane-projected front-face rects to treat them as the same physical object. Adjacent identical facings project side-by-side (IoU ~ 0), so they are never merged by this rule. Default 0.1 — lenient so viewpoint-skewed or border-clipped re-observations (whose projected overlap drops well below 0.3) still merge, while side-by-side facings stay separate. plane_merge_depth_delta_m: type: number default: 0.25 description: | Maximum difference of the two detections' distances to the consensus plane (metres) — the "virtual object depth". Overlapping rects farther apart than this along the shelf normal are different objects (front row vs back row), not a re-observation. reclassify_labels: type: boolean default: false description: | Dims reclassifier master switch — re-labels a placed detection to a SIBLING label (same label_group) when its MEASURED physical size fits the sibling's expected dims better. Annotations keep `original_label` + `reclassification_reason` provenance; running with the switch OFF reverts earlier auto reclassifications. reclassify_keep_dev: type: number default: 0.15 description: "Fit error at/below which the ORIGINAL label is kept outright (~ ±16% size). Raising it protects the detected label." reclassify_target_dev: type: number default: 0.15 description: "A sibling must fit within this error to steal the detection. Raising it favors re-labeling." reclassify_min_margin: type: number default: 0.06 description: "Base margin E(original) − E(best sibling) must exceed. Raising it protects the detected label." reclassify_conf_margin_scale: type: number default: 0.5 description: "Margin scales ×(1 + this × detector confidence) — confident detections are harder to overturn; 0 ignores confidence." reclassify_weight_scale: type: number default: 1.0 description: "Weight of the SIZE mismatch in the fit error. Raising it favors re-labeling (size separates variants)." reclassify_weight_aspect: type: number default: 0.5 description: "Weight of the SHAPE mismatch in the fit error. Raising it protects the detected label (siblings share shape)." reclassify_min_depth_confidence: type: number default: 0.5 description: "Detections with depth confidence below this are never reclassified." group_consensus_merge: type: boolean default: true description: | Post-walk safety net (active only with reclassify_labels) — merge clusters of DIFFERENT labels from the same label_group that overlap on the consensus plane; the members are re-labeled to the best dims fit. size_gate: type: boolean default: true description: | Physical SIZE GATE — runs AFTER the reclassifier: a detection whose measured size still exceeds (1+allowance)× its (possibly re-labeled) label's expected width/height/area is DROPPED from the walk (ledger disposition `size_rejected`; the annotation is kept and flagged `size_rejected` + `size_reject_detail`). One-sided by design — under-size is routinely a partially occluded real product. No-op for labels without both dims. size_gate_dims_allowance: type: number default: 0.35 description: "Per-axis allowance: reject when measured width or height > (1+this) × expected. Raising it keeps more (looser gate)." size_gate_area_allowance: type: number default: 0.35 description: "Area allowance: reject when measured w·h > (1+this) × expected area. Noise compounds in area, so this is the binding check. Raising it keeps more." size_gate_min_depth_confidence: type: number default: 0.5 description: "Detections with depth confidence below this are never size-gated (unmeasurable ≠ oversized)." min_depth_m: type: number default: 0.05 max_depth_m: type: number default: 6 conf_threshold: type: number default: 1 front_percentile: type: number default: 30 shelf_tolerance_m: type: number default: 0.3 shelf_analysis: type: boolean default: true description: | Master switch for SHELF COMPOSITION — after fusion, build the planogram structure (shelves → stacks → objects) from the fused objects' world geometry and persist it as `shelf_composition`. Shelf BOARDS are found first (density modes of object bottoms, accepted strongest-first at a minimum physical spacing); a STACK is then everything between two boards at one horizontal slot — two stacks can never sit on top of each other by construction, and stacking never hinges on fragile box-touch tolerances. shelf_support_min_overlap: type: number default: 0.5 description: | Min horizontal overlap ratio (of the narrower object, 0..1) for an object to belong to the same column as the one below it. Increase: demands straighter piles — offset cans become separate stacks. Decrease: diagonal neighbours chain into one stack, understating facing counts. shelf_gap_split_m: type: number default: 0.06 description: | Half-width (metres) of the bottom-density window used to find candidate shelf levels — bottoms within this of a mode belong to it. Increase: nearby levels blur into one candidate. Decrease: measurement noise splinters a level into weak candidates. shelf_min_spacing_m: type: number default: 0.12 description: | Minimum vertical distance between two shelf BOARDS (board thickness + product clearance — hard store physics). Candidate levels are accepted strongest-first, each at least this far from every accepted board, which is what keeps a jar-on-jar stacking layer (bottoms +8–11 cm above their base) from ever becoming a phantom shelf. Increase: genuinely shallow shelves read as stacking. Decrease: stacking layers start reading as shelves. shelf_stack_max_penetration_m: type: number default: 0.035 description: | Measurement slack for stacked boxes (metres). Two objects cannot co-occupy space: a joiner whose bottom sinks deeper than this into an overlapped column member is a side-by-side facing with an inflated box, never a pile member. Increase: inflated neighbours glue into false towers. Decrease: noisy true piles split apart. shelf_row_split_m: type: number default: 0.15 description: | Plane-depth gap separating front/back rows within one shelf (metres). Row 0 is the frontmost (closest to the shopper). Increase: everything collapses into a single row. Decrease: minor depth noise becomes extra rows. shelf_min_stacks: type: number default: 1 description: | A level must hold at least this many stacks to become a shelf; leaner levels are reported under `unshelved` instead. Increase: filters hook-wall/pegboard singletons and stray mis-placed bases out of the shelf list. Decrease (1): every level becomes a shelf, including one-off outliers. build_point_cloud: type: boolean default: true description: | Build + store the scene voxel point cloud (and RANSAC shelf planes) during the run. Downloads each frame's depth blob + RGB image server-side, so disable for very large sessions when the 3D scene's cloud layer is not needed. pc_stride: type: number default: 2 description: Depth-grid stride when back-projecting the cloud (every Nth pixel). pc_voxel_size_m: type: number default: 0.02 description: Voxel edge for point-cloud downsampling, metres. ransac_max_planes: type: number default: 4 description: Max shelf planes to peel off the voxel cloud. ransac_min_inlier_ratio: type: number default: 0.05 description: Stop peeling planes below this inlier share. ransac_distance_thresh_m: type: number default: 0.02 description: Point-to-plane inlier distance for RANSAC, metres. AnalysisObject: type: object description: | A concluded object on the shelf (one per cross-frame cluster). `state` is `kept` for a single facing or `merged` for several facings fused. properties: _id: type: string description: | Cluster link target — contributing task annotations carry this id in their `cluster_id` after the run succeeds. label_id: { type: string } label_name: { type: string } state: type: string enum: [kept, merged] confidence: type: number description: Fused placement confidence (cluster mean), 0..1. world: type: object description: Centroid in world coordinates, **metres**. properties: x: { type: number } y: { type: number } z: { type: number } size: type: object description: Physical front-face size, **centimetres**. properties: w: { type: number } h: { type: number } depth: type: number description: Distance from the camera, **metres**. height_above_ground_m: type: number nullable: true description: | Metres above the device-locked floor (`session.ground.y_world`) — the world frame is gravity-aligned, so this is the object's true shelf height. Compare against `session.eye_level_m` for shelf-band analytics (eye level ≈ premium band). `null` when the rep skipped the floor point at capture entry. yaw: type: number nullable: true description: | Facing rotation about world-Y, **radians** — the direction from the object centroid to the winning frame's camera. `null` when the winning task lacks a pose. cluster_size: type: number description: How many facings/frames contributed. bbox_3d: type: object properties: min: { type: array, items: { type: number } } max: { type: array, items: { type: number } } winning_task: { type: string } winning_annotation_id: { type: string } winning_box: type: object description: | Winner's 2D crop region on `winning_image_media` (YOLO-normalized cx, cy, w, h). Front-ends texture the object with this sub-region. properties: x1: { type: number } y1: { type: number } x2: { type: number } y2: { type: number } winning_image_media: type: string description: | `media.mediaStorages` id of the winning frame image (the crop source). Populate with `objects.winning_image_media` to get its URL. contributing_tasks: type: array items: { type: string } arbitration: type: object description: | Full winner-selection rationale: `strategy` (`weighted_view_score`), the factor `weights`, and per-member `scores[]` — each with the six normalized inputs (`det`, `depth`, `track`, `center`, `size`, `agree`), the weighted `score`, and a `winner` flag. Renders the dashboard's "why did this view win" inspector. properties: strategy: { type: string } weights: type: object additionalProperties: { type: number } scores: type: array items: type: object properties: task_id: { type: string } annotation_id: { type: string } frame_id: { type: number } score: { type: number } winner: { type: boolean } inputs: type: object properties: det: { type: number } depth: { type: number } track: { type: number } center: { type: number } size: { type: number } agree: { type: number } ConcludedLabel: type: object description: Per-label conclusion — detection counts by outcome. properties: label_id: { type: string } label_name: { type: string } kept: type: number description: Detections placed as a single facing. merged: type: number description: Detections fused into multi-facing objects. ignored: type: number description: Detections that could not be placed. dropped: type: number description: Detections removed by same-frame quality gates. size_rejected: type: number description: Detections dropped by the physical-size gate. object_count: type: number description: Resulting objects (kept + merged clusters). IgnoredDetection: type: object description: A detection that could not be placed in world coordinates. properties: task: { type: string } annotation_id: { type: string } label_id: { type: string } frame_id: { type: number } reason: type: string description: | Why placement was impossible: `no_pose`, `no_intrinsics`, `no_depth`, `empty_depth_region`, `insufficient_depth_pixels`, `behind_shelf`, or `not_placed` (legacy annotations without a recorded reason). DetectionLedgerEntry: type: object description: | The audit trail of ONE detection through the fusion pipeline — where it went and why. `new_object` spawned an object; `re_observation` merged into an object first seen in an earlier frame; `dropped_same_frame` was removed by a quality gate (see `reason` + `kept_by_annotation_id`); `size_rejected` was dropped by the physical-size gate (annotation kept and flagged); `unplaced` never had world data (see `reason`). properties: task: { type: string } annotation_id: { type: string } label_id: { type: string } label_name: { type: string } frame_id: { type: number } disposition: type: string enum: [ new_object, re_observation, dropped_same_frame, size_rejected, unplaced, ] reason: type: string description: | Machine cause — `first_observation`, `re_observation`, `duplicate_box_in_frame`, `same_spot_in_frame`, `below_min_confidence`, `implausible_size`, `size_gate`, or an unplaced reason (`no_pose`, `no_depth`, `behind_shelf`, ...). detail: type: string description: Human-readable explanation, ready to render. object_id: type: string description: The `objects[]._id` this detection created or merged into. kept_by_annotation_id: type: string description: For same-frame drops — the stronger sibling that was kept. matched_distance_m: type: number description: For re-observations — world distance to the object, metres. world: type: object properties: x: { type: number } y: { type: number } z: { type: number } box: type: object description: The detection's 2D box (YOLO-normalized cx, cy, w, h). properties: x1: { type: number } y1: { type: number } x2: { type: number } y2: { type: number } confidence: { type: number } placement_confidence: { type: number } AnalysisFunnel: type: object description: Where every detection went — the dashboard's fusion summary. properties: tasks: { type: number } tasks_manual: type: number description: | Frames whose boxes came from a confirmed HUMAN annotation group (manual / auto_edited) — the "computed on N human-verified frames" number business metrics report. detections_total: { type: number } unplaced: { type: number } size_rejected: type: number description: Detections dropped by the physical-size gate. frames: { type: number } detections_placed: { type: number } dropped_same_frame: { type: number } re_observations: { type: number } new_objects: { type: number } clusters_final: { type: number } SessionAnalysis: type: object properties: _id: { type: string } disabled: { type: boolean } session: { type: string } model_version: { type: string } config: $ref: "#/components/schemas/SceneMathConfig" frame_sources: type: array description: | PROVENANCE — which annotation group each frame contributed, chosen human-truth-first (a confirmed manual/auto_edited group beats every auto group regardless of the requested model version). The dashboard resolves the exact boxes this analysis consumed from here; `edit_time` lets the UI flag analyses older than a frame's latest correction. items: type: object properties: task: { type: string } group_id: { type: string } annotation_state: type: string enum: [auto, manual, auto_edited] model_version: { type: string } edit_time: { type: number } stage_fingerprints: type: object description: | Per-stage config hashes (detect / fuse / compose slices of `config`). Staleness is a fingerprint diff — a shelf-knob change flips only `compose`, unlocking the compose-only `recompute`. properties: detect: { type: string } fuse: { type: string } compose: { type: string } status: type: string enum: [pending, in_progress, success, failed] started_at: { type: number } finished_at: { type: number } heartbeat_at: type: number description: | Liveness heartbeat (epoch ms) refreshed by the background run after each task inference; the stale-run watchdog measures against it. tasks_total: { type: number } tasks_inferred: type: number description: Tasks inferred during this run (not previously placed). kept_count: type: number description: Detections kept (single facings). merged_count: type: number description: Detections merged into objects. ignored_count: type: number description: Detections that could not be placed. dropped_count: type: number description: Detections removed by same-frame quality gates. size_rejected_count: type: number description: Detections dropped by the physical-size gate. objects_count: type: number description: Resulting objects (kept + merged). objects: type: array items: $ref: "#/components/schemas/AnalysisObject" concluded_labels: type: array items: $ref: "#/components/schemas/ConcludedLabel" ignored: type: array items: $ref: "#/components/schemas/IgnoredDetection" detections: type: array description: Per-detection fusion audit trail (one entry per analyzed annotation). items: $ref: "#/components/schemas/DetectionLedgerEntry" funnel: $ref: "#/components/schemas/AnalysisFunnel" shelf_composition: type: object description: | PLANOGRAM STRUCTURE composed from the fused objects' world geometry (present unless `config.shelf_analysis: false` or nothing was placed). Ordering contract: `shelves[0]` is the LOWEST shelf, `shelves[].stacks[0]` is the SHOPPER's leftmost stack, and `stacks[].object_ids` list the pile bottom-up. Compliance rules (same-shelf exclusions, adjacency, share-of-shelf) read directly off this structure. properties: plane: type: object description: | Orthonormal world basis. `n` is the horizontal unit normal of the shelf face pointing TOWARD the shopper (the session's cameras); `u = up × n` points to the shopper's RIGHT. A plane coordinate maps back to world as `û·u + Ŷ·y + n̂·s`. properties: n: type: array items: { type: number } u: type: array items: { type: number } orientation: type: string enum: [shopper] description: Stacks are indexed in the shopper's frame (0 = leftmost). shelves: type: array items: type: object properties: index: type: number description: 0 = the lowest shelf. y_world: type: number description: | Board level (world Y, metres) — a low percentile (p15) of the member stack bases, so the board sits UNDER its products rather than through the middle of the noisy base band. height_above_ground_m: type: number nullable: true description: Shelf level above the device-locked floor; null when the floor was not locked. u_from: type: number description: Occupied horizontal extent start (metres, plane coords). u_to: type: number s: type: number description: Median plane depth of member stacks (metres). stacks: type: array items: type: object properties: index: type: number description: 0 = leftmost as the shopper sees the shelf. u_from: { type: number } u_to: { type: number } y_from: { type: number } y_to: { type: number } s: { type: number } row: type: number description: Depth row within the shelf — 0 = front row (closest to the shopper). object_ids: type: array items: { type: string } description: "Refs into `objects[]._id`, ordered bottom-up (index 0 sits on the shelf)." unshelved: type: array description: Objects that didn't land on a shelf level (hook walls, sparse levels). items: type: object properties: object_id: { type: string } reason: { type: string } scene: type: object description: | Scene-level outputs. All best-effort — each part is omitted when it can't be computed (too few points, media unavailable, or `config.build_point_cloud: false`). properties: plane: type: object description: | Dominant (shelf) plane fitted over all placed detections' world positions (covariance eigen; legacy single plane, kept for back-compat). Omitted when fewer than 3 placed detections exist or the fit fails. properties: normal: type: array items: { type: number } description: "Unit normal [x, y, z]." d: type: number description: "Plane equation term: n·p + d = 0." centroid: type: array items: { type: number } description: "Centroid [x, y, z], metres." inliers: type: number description: Points used in the fit. planes: type: array description: | Bounded shelf planes peeled off the voxel point cloud by iterative RANSAC, strongest first. Front-ends render each as a translucent rectangle sized by its extent corners. items: type: object properties: normal: type: array items: { type: number } description: "Unit normal [x, y, z]." point: type: array items: { type: number } description: "A point on the plane (inlier centroid), metres." extent_min: type: array items: { type: number } description: One corner of the fitted inlier rectangle (world coords). extent_max: type: array items: { type: number } description: The opposite corner. inlier_count: { type: number } inlier_ratio: type: number description: "Inliers / cloud points, 0..1." point_cloud: type: object description: | Voxel-downsampled scene point cloud metadata. The packed little-endian Float32 `[x, y, z, r, g, b, w]` blob is stored as `bin` media (`media`) — the `view-3d-scene` service inlines it as base64 for the dashboard. properties: media: type: string description: "`media.mediaStorages` `_id` of the packed blob." num_points: { type: number } voxel_size_m: { type: number } aabb_min: type: array items: { type: number } description: "Axis-aligned bounds min [x, y, z], metres." aabb_max: type: array items: { type: number } _errors: type: array items: { type: object } creator: $ref: "#/components/schemas/UserRef" editor: $ref: "#/components/schemas/UserRef" company_namespace: type: array items: { type: string } description: Tenant key. Server-injected — never accept from clients. createdAt: { type: string, format: date-time } updatedAt: { type: string, format: date-time } UserRef: type: object description: Compact actor reference (rep or admin). properties: _id: { type: string } type: { type: string, enum: [admin, rep] } name: { type: string } rep: { type: string } admin: { type: string }