# bsdata40k-to-json

Convert [BSData](https://github.com/BSData) BattleScribe XML for **Warhammer 40,000** into JSON. Turn army list data (factions, units, weapons, abilities, points) from the community-maintained BSData repos into two output formats:

- **Raw** — Faithful 1:1 XML-to-JSON conversion preserving all BattleScribe elements, attributes, and list-builder logic
- **Curated** — Clean, game-ready JSON: datasheet stats (M, T, SV, W, LD, OC), ranged/melee weapon profiles, abilities, keywords, transport capacity, leader attachments, enhancements, and points per faction

Built for **Warhammer 40,000** (40k) 10th edition data. Use the output for list-building apps, points calculators, roster viewers, or any tool that needs structured 40k game data.

## Install

```bash
npm install bsdata40k-to-json
```

Or install globally for CLI access:

```bash
npm install -g bsdata40k-to-json
```

## CLI Usage

Pass the input directory (BSData repo) and output directory as two arguments:

```bash
bsdata40k-to-json ./wh40k-10e ./output
```

### Options

| Argument / Flag | Description |
|---|---|
| `<input-dir>` | Directory containing `.cat` and `.gst` files (required) |
| `<output-dir>` | Directory to write JSON output (required) |
| `--raw-only` | Only produce raw (1:1 XML-to-JSON) output |
| `--curated-only` | Only produce curated game-data output |
| `--help`, `-h` | Show help message |

### Development scripts

| Command | Description |
|---|---|
| `npm run convert -- <input-dir> <output-dir>` | Run both pipelines via `tsx` |
| `npm run convert:raw -- <input-dir> <output-dir>` | Raw XML-to-JSON only |
| `npm run convert:curated -- <input-dir> <output-dir>` | Curated extraction only |
| `npm run build` | Compile TypeScript to `dist/` |

## Library Usage

```typescript
import { convert } from "bsdata40k-to-json";

const result = await convert({
  inputDir: "./wh40k-10e",
  outputDir: "./output",
  curatedOnly: true,
});

console.log(`Converted ${result.factionCount} factions`);
```

Individual extractors are also available:

```typescript
import {
  convertFileToRawJson,
  extractGameSystem,
  extractCatalogue,
  discoverFiles,
} from "bsdata40k-to-json";

const { gstFiles, catFiles } = await discoverFiles("./wh40k-10e");
const rawJson = await convertFileToRawJson(gstFiles[0]);
const gameSystem = extractGameSystem(rawJson);
```

All TypeScript types are exported:

```typescript
import type {
  CuratedFaction,
  CuratedUnit,
  WeaponProfile,
  Enhancement,
  CuratedGameSystem,
  ConvertOptions,
  ConvertResult,
} from "bsdata40k-to-json";
```

## Output Structure

```
<output-dir>/
  raw/                          # 1:1 XML-to-JSON
    game-system.json            # Warhammer 40,000.gst (core rules, profile types)
    necrons.json                # One file per faction catalogue (.cat)
    imperium-space-marines.json
    ...

  curated/                      # Clean game data
    game-system.json            # Shared rules (e.g. Blast, Devastating Wounds), profile types, categories
    factions/
      necrons.json
      imperium-space-marines.json
      ...
    index.json                  # Faction manifest with unit counts
```

### What’s in the curated output (40k terms)

- **Factions** — One JSON file per 40k faction (e.g. Necrons, Space Marines, Tyranids), with `name`, `catalogueName`, and all units/enhancements for that army.
- **Units** — Full datasheet-style data: Movement (M), Toughness (T), Save (SV), Wounds (W), Leadership (LD), Objective Control (OC), plus keywords (Infantry, Character, etc.) and faction keywords.
- **Weapons** — Ranged and melee weapon profiles with Range, Attacks (A), Ballistic Skill (BS) or Weapon Skill (WS), Strength (S), Armour Penetration (AP), Damage (D), and weapon keywords (e.g. Devastating Wounds, Blast).
- **Abilities** — Named abilities and their rules text (e.g. *Master Chronomancer*, *Oath of Moment*).
- **Enhancements** — Detachment enhancements with name, points cost, and description; optional `detachment` when applicable.
- **Leader attachments** — Which units a character can lead (e.g. Orikan the Diviner → Immortals, Necron Warriors).

## Curated JSON Format (example: Necrons)

Each faction file contains units with full 40k-style datasheets:

```json
{
  "id": "b654-a18a-ea1-3bf2",
  "name": "Necrons",
  "catalogueName": "Xenos - Necrons",
  "units": [
    {
      "id": "ba86-eaad-5396-fc07",
      "name": "Orikan the Diviner",
      "points": 80,
      "keywords": ["Infantry", "Character", "Epic Hero", "Cryptek"],
      "factionKeywords": ["Necrons"],
      "stats": { "M": "5\"", "T": "4", "SV": "4+", "W": "4", "LD": "6+", "OC": "1" },
      "rangedWeapons": [],
      "meleeWeapons": [
        {
          "name": "Staff of Tomorrow",
          "range": "Melee",
          "A": "2",
          "WS": "3+",
          "S": "4",
          "AP": "-3",
          "D": "D3",
          "keywords": ["Devastating Wounds"]
        }
      ],
      "abilities": [
        { "name": "Master Chronomancer", "description": "While this model is leading a unit, models in that unit have a 4+ invulnerable save." }
      ],
      "leader": ["Immortals", "Necron Warriors"]
    }
  ],
  "enhancements": [
    {
      "name": "Veil of Darkness",
      "points": 20,
      "description": "NECRONS model only. Once per battle...",
      "detachment": "Awakened Dynasty"
    }
  ]
}
```

## Project Structure

```
src/
  cli.ts                         # CLI entry point (arg parsing)
  index.ts                       # Library entry point (programmatic API + re-exports)
  raw-converter.ts               # Faithful XML-to-JSON using fast-xml-parser
  utils.ts                       # File discovery, slugification, helpers
  curated/
    types.ts                     # TypeScript interfaces for output shapes
    extract-game-system.ts       # .gst extractor (rules, profile types, categories)
    extract-catalogue.ts         # .cat extractor (units, weapons, abilities, enhancements)
```

## How It Works

1. **Discovery** — Scans the input directory for the 40k game system (`.gst`) and faction catalogues (`.cat`).
2. **Raw conversion** — Parses each XML file with `fast-xml-parser`, preserving attributes (prefixed with `@_`) and forcing arrays for repeatable elements.
3. **Curated extraction** — Walks the parsed XML tree to extract 40k game data:
   - Resolves `entryLink` cross-references (shared units, wargear, weapons) by building an ID lookup map
   - Recursively collects profiles from nested `selectionEntry`, `selectionEntryGroup`, and `entryLink` elements
   - Deduplicates weapons by name
   - Parses leader attachment targets from ability descriptions (e.g. “This model can be attached to…”)
   - Separates faction keywords (e.g. `Faction: Necrons`) from unit keywords (Infantry, Character, etc.)

## Data Source (Warhammer 40,000)

Input files come from the [BSData](https://github.com/BSData) community-maintained **Warhammer 40,000** data. For 10th edition use the [BSData/wh40k-10e](https://github.com/BSData/wh40k-10e) repository:

```bash
git clone https://github.com/BSData/wh40k-10e.git
bsdata40k-to-json ./wh40k-10e ./output
```

The repo contains the core game system (`Warhammer 40,000.gst`) and one catalogue (`.cat`) per faction (Space Marines, Necrons, Tyranids, etc.). Download or clone the repo and pass its path as the first argument.

## License

MIT