{
  "skill_name": "wiki-orm-agent",
  "evals": [
    {
      "id": 1,
      "prompt": "Detect and map the ORM entities in this Spring Boot project that uses JPA with PostgreSQL",
      "expected_output": "Produces database-mapping.md with entity-table mapping, Mermaid ER diagram, and unmapped tables list",
      "files": [],
      "expectations": [
        "Both User AND Order are identified as @Entity classes — detected-entities.json lists class_name: 'User' and class_name: 'Order'",
        "@Table(name) maps to table names 'users' and 'orders' respectively; User also carries schema='public' from @Table(schema=\"public\")",
        "@OneToMany and @ManyToOne relationships are captured — detected-entities.json includes at least one one_to_many and one many_to_one relationship",
        "Every relationship in detected-entities.json has a non-empty target_entity that names another extracted class — no blank target_entity strings",
        "Mermaid erDiagram references real extracted tables at both endpoints; any external target (e.g., Role when Role.java isn't in scope) is emitted as a stub `table { _external ... }` block BEFORE the edge, so there is NEVER an edge to a node that isn't declared",
        "Mermaid erDiagram contains an edge between 'users' and 'orders' (either ||--o{ or }o--||) and uses 'users' and 'orders' as literal node names",
        "Generated markdown has frontmatter with type='entity' and orm_profile='jpa'",
        "mermaid_lint.js reports no syntax issues on the generated page"
      ]
    },
    {
      "id": 2,
      "prompt": "I have a custom ORM that uses BaseModel classes with __table__ attributes. Help me create an ORM profile for it",
      "expected_output": "Walks through Q&A and generates a custom YAML profile matching the ORM profile schema",
      "files": [],
      "expectations": [
        "Q&A covers detection markers (class inheritance + __table__), file patterns, entity extraction (class_pattern, table_pattern, column_pattern), and at least one relationship pattern",
        "Q&A covers naming conventions (table_from_class and column_from_field)",
        "Generated YAML has every required top-level field: name, language, description, detection, entity_extraction, relationship_patterns, naming_conventions",
        "entity_extraction.class_pattern has a single capture group that matches at least one class name in example.py (verified by running the pattern against the fixture)",
        "profile loader (profiles.js loadProfile) accepts the YAML without raising ProfileValueError",
        "When the profile is applied to an example.py containing THREE BaseModel classes with DIFFERENT __table__ values, extractEntities yields 3 entities with 3 DIFFERENT table_names — NOT all three collapsing to the first table (the per-class windowing fix must hold even with a straightforward table_pattern that matches every __table__ occurrence)"
      ]
    },
    {
      "id": 3,
      "prompt": "Detect and map the ORM entities in this Python project that uses SQLAlchemy with PostgreSQL. The project has a single models.py file declaring multiple entities that inherit from a shared declarative Base.",
      "expected_output": "Produces database-mapping.md with one row per entity, each mapped to its __tablename__, plus a Mermaid ER diagram showing the relationships()",
      "files": [],
      "expectations": [
        "Every class that inherits from the declarative Base is identified as an entity — detected-entities.json has an entry per SQLAlchemy class in the fixture",
        "Each entity's table_name matches its own __tablename__ assignment (the per-class windowing fix; multi-class single-file input must not collapse to one table)",
        "relationship() declarations are picked up by the relationship_patterns, with type and target_entity populated where the back-ref is unambiguous",
        "Mermaid erDiagram contains one node per extracted entity with its columns, and AT LEAST ONE edge between two of them — with the extended edge-rendering in output.ts, `relationship`, `foreign_key`, `one_to_one`, `many_to_one` types now produce edges too (not just one_to_many + many_to_many). Zero edges is a failure.",
        "Generated markdown has frontmatter with type='entity' and orm_profile='sqlalchemy'",
        "mermaid_lint.js passes on the generated page"
      ]
    },
    {
      "id": 4,
      "prompt": "Detect and map the ORM entities in this Prisma project. The schema lives at agents/wiki-orm-agent/evals/fixtures/prisma/schema.prisma — three models (User, Post, Comment) with explicit @relation declarations linking them.",
      "expected_output": "Produces database-mapping.md with each Prisma model row-mapped to its @@map table, every @relation resolved to a target_entity, Mermaid edges matching declared cardinalities",
      "files": [],
      "expectations": [
        "All three models (User, Post, Comment) are extracted into detected-entities.json as separate entries",
        "Each entity's table_name matches the @@map(...) value in the fixture (e.g., User → 'users', Post → 'posts', Comment → 'comments')",
        "Every @relation(fields: [...], references: [...]) field produces a relationship in detected-entities.json with a non-empty target_entity that names another extracted model — no blank target_entity strings",
        "One-to-many sides (e.g., posts Post[]) produce relationships of type one_to_many; their corresponding many-to-one inverse (e.g., author User @relation(...)) produces type many_to_one, both with the correct target_entity",
        "Generated database-mapping.md frontmatter has type='entity' and orm_profile='prisma'",
        "Mermaid erDiagram contains nodes named 'users', 'posts', 'comments' (the table names from @@map, not the model names) and at least one edge of cardinality ||--o{ linking users-posts AND another linking posts-comments",
        "mermaid_lint.js reports no issues on the generated page"
      ]
    },
    {
      "id": 5,
      "prompt": "Detect and map the TypeORM entities in agents/wiki-orm-agent/evals/fixtures/typeorm/ — three .ts files (Author.ts, Book.ts, Publisher.ts) using @Entity, @Column, @OneToMany, @ManyToOne.",
      "expected_output": "Produces database-mapping.md with each @Entity row-mapped to its annotation table name, every relationship decorator resolved to a target_entity from the call argument, Mermaid edges matching cardinalities",
      "files": [],
      "expectations": [
        "All three entities (Author, Book, Publisher) are extracted into detected-entities.json as separate entries with table_names matching the @Entity('...') first arg (e.g., 'authors', 'books', 'publishers')",
        "Each @OneToMany(() => Book, ...) and @ManyToOne(() => Author, ...) produces a relationship with target_entity resolved from the arrow-function call argument — never blank, never the literal string 'Book'/'Author' (the resolver must dereference to the imported class)",
        "Relationship type matches the decorator name (one_to_many for @OneToMany, many_to_one for @ManyToOne, one_to_one for @OneToOne)",
        "Generated database-mapping.md frontmatter has type='entity' and orm_profile='typeorm'",
        "Mermaid erDiagram contains nodes for all three table names and at least two edges (authors-books and books-publishers, or whatever the fixture declares) using cardinality glyphs that match the decorator (||--o{ for one_to_many, }o--|| for many_to_one)",
        "Any external target referenced via @ManyToOne whose class file is NOT in the fixture set is emitted as a _external stub before the edge — never an edge to an undeclared node",
        "mermaid_lint.js passes on the generated page"
      ]
    }
  ]
}