---
name: data-research
description: "Recipe-driven structured-data extraction. Takes a named recipe and a set of sources (emails, reports, web pages, forms, PDFs) and writes typed entities to the graph per the recipe's schema. Triggers when the owner says 'extract X from these emails', 'track investor updates', 'pull donation amounts', 'build a table of Y from these documents'."
---

# Data research

Invoked from `specialists:research-assistant`.

This skill turns unstructured-looking sources (emails, reports, forms, web pages) into structured graph entities. The transformation is always recipe-driven: a named YAML recipe declares what fields to extract, what validations to apply, and what graph nodes to write. The skill runs a named recipe; it never invents a schema on the fly.

**Default parent.** Every recipe declares its hierarchy parent via `graph_target.parent` — either `Project` (the run-time recipe argument names which Project elementId to attach to) or `LocalBusiness` (the account root, for cross-project recipes like investor-updates). The skill refuses to run a recipe whose `parent` field is missing or whose named Project does not exist. Per-row writes attach to that parent via the canonical containment edge; the recipe's `relationships:` block describes cross-hierarchy edges (e.g. `ROUND_FOR → Organization`) which are written separately and do not substitute for the parent.

## When to run

Run when the owner wants the same extraction repeated across many sources. The trigger phrases are above. Do not run for one-off extractions; a one-off is what `document-ingest` does.

## Inputs

| Input | Meaning |
|---|---|
| `recipe` | The name of an existing recipe at `plugins/deep-research/recipes/<name>.yaml`. |
| `sources` | One or more source paths or URLs. Email thread paths, PDF paths, web URLs, or a folder. |
| `parent` | The Project elementId (or `LocalBusiness` for cross-project recipes) the writes anchor to. Required when the recipe's `parent` field is `Project`; ignored when the recipe's parent is `LocalBusiness` (the account root resolves automatically). |

If `recipe` does not match any file in the recipes directory, refuse. List the available recipe names. Do not invent a recipe.

If the owner asks for an extraction shape that no existing recipe covers, the answer is "no recipe exists for that yet. Want me to draft one?". The skill never extracts data without a recipe; that is the point of the recipe layer.

## Recipe shape

A recipe is a YAML file with this shape:

```yaml
name: investor-updates
description: One-line description of what this recipe extracts.
source_kinds:
  - email-thread
  - web-page
  - pdf
fields:
  - name: company_name
    type: string
    required: true
  - name: round
    type: string
    required: false
  - name: amount_gbp
    type: number
    required: false
    validation:
      min: 0
  - name: date_announced
    type: date
    required: true
graph_target:
  label: FundingRound
  parent: LocalBusiness   # or `Project` — names the hierarchy anchor for every row
  identity:
    - company_name
    - date_announced
  relationships:
    - type: ROUND_FOR
      target_label: Organization
      target_match: company_name
```

## Method

1. **Load and validate the recipe.** Read the recipe file. Check that every field declared has a `name` and `type`. Check that `graph_target.label` is a label in the live ontology (`maxy-graph-get_neo4j_schema`). Check that `graph_target.parent` is present and resolves (Project elementId supplied at call time, or `LocalBusiness` resolved from the account root). If anything fails, refuse and report the recipe's defect.
2. **Iterate sources.** For each source: classify by kind against `source_kinds`. Skip sources whose kind is not in the recipe's allowed list, list them in the report.
3. **Extract per source.** Fetch the source. Use Haiku-driven classification (`memory-classify` with a recipe-specific prompt) to extract each declared field. Apply each field's validation. Reject rows that fail required-field checks; capture them in a rejected list.
4. **Write to the graph.** For each accepted row, `memory-write` a node with the declared label and the extracted properties. Attach it to the resolved hierarchy parent via the canonical containment edge. Stamp `createdByAgent: data-research`, `createdByRecipe: <recipe name>`, `sourceDocumentId: <source>`. Write the declared cross-hierarchy relationships, resolving target nodes via `memory-search` on the named match property.
5. **Report.** Return one line per source (accepted count, rejected count) and one summary line: total rows written, total rejected, total relationships created.

## Output format

```
**Recipe.** <name>
**Sources.** <N> processed.

| Source | Accepted | Rejected | Reason |
|---|---|---|---|
| <source path or URL> | 3 | 1 | "amount_gbp negative" |
| ... | | | |

**Total.** 12 rows written as :<Label>, 9 relationships created via <RelType>.
**Rejections.** 3 rows rejected (full list below).
```

## Why recipes

A recipe is a contract. The owner reads the recipe and knows what will be written. The recipe is version-controlled and auditable. Without recipes, the skill would extract whatever the LLM thinks is interesting, which means the graph schema drifts with every run and the owner cannot trust what arrives. The recipe layer is the difference between a controlled pipeline and a junk shop.

## Failure modes

- **Recipe file is malformed.** Surface the YAML error literally and refuse.
- **Recipe declares a `graph_target.label` not in the ontology.** Refuse. Tell the owner the label is missing and recommend either adjusting the recipe or extending the ontology.
- **Recipe's `graph_target.parent` is missing or unresolvable.** Refuse. The hierarchy parent is mandatory; without it, every row would land as an orphan.
- **No sources match the recipe's allowed kinds.** Refuse. List what the recipe accepts.
- **Target-match relationship cannot resolve** (the recipe wants `ROUND_FOR -> Organization` and the named company is not in the graph). Capture in the rejected list with the reason "target organisation not found"; do not create a placeholder organisation.
- **Embedding service unavailable mid-batch.** Stop. Report which sources completed and which did not.

## Where recipes live

Recipes live at `platform/plugins/deep-research/recipes/<name>.yaml`. The directory is part of the plugin and ships with the platform. To author a new recipe, the owner names the extraction they want; the skill can scaffold the YAML and the owner reviews and saves it. Recipe authoring is a separate path; this skill consumes recipes, it does not write them inline.

## What this skill does not do

It does not run free-form extractions. It does not write to the graph without an approved recipe. It does not modify recipes during a run; if the recipe needs changing, the run aborts, the owner edits the recipe, and the run is re-issued.