# Data Module data flow

## What It Does
- Summary: Provides the workbook data-access and normalization utilities. Exposes `getData` to build and return cell maps or column-wise payloads for renderers and external consumers, formats cell values (`getValueFromFormat`), and normalizes sparse models via `getModel` and `processIdx`. Handles formula calculation triggers, data-source synchronization, merges, hidden rows/cols, and frozen-index considerations. See `src/workbook/base/data.ts`.

## Entry Points (UI / Core)
- UI (Consumers):
  - `getData(context, address, ...)` — primary async API used by render layer (`Render.refreshUI`) to request cells for a given address or viewport. Returns `Promise<Map<string, CellModel>>` or column-wise arrays depending on options.
  - `getValueFromFormat(context, cell, rowIdx, colIdx, sheetIdx, ...)` — formatting helper used when building export payloads or value-only responses.
- Core (Model / Utilities):
  - `getModel(model, idx)` — ensures sparse arrays (sheets/rows/cells) contain proper slot entries and populates nulls for missing indices.
  - `processIdx(model, isSheet?, context?, isSort?, rowIndex?, processComment?, processNote?)` — normalizes index gaps, creates sheets when needed, processes comments/notes and adjusts usedRange for sheets.

## Core Logic Flow (ASCII)
Caller requests range → `getData()`
         ↓
Resolve sheet index and trigger `updateSheetFromDataSource` (if datasources present)
         ↓
If column-wise: iterate address ranges → collect per-row objects or value-only cells (apply formatters/parsers)
Else: iterate rows/cols → for each cell: trigger `queryCellInfo`, fetch `getCell`, detect spans (rowSpan/colSpan) and repair via `setCell` as needed
         ↓
If formula present and value missing → trigger `calculateFormula`
         ↓
Return assembled `Map<string, CellModel>` or column-wise array to caller

## Operations Handled (functions & implementation notes)
- `getData(...)`:
  - Accepts flags: `columnWiseData`, `valueOnly`, `frozenIndexes`, `skipHiddenRows`, `dateColData`.
  - Calls `context.notify(updateSheetFromDataSource, args)` and waits for returned promise to ensure data-bound ranges are up-to-date.
  - For column-wise requests builds array of row objects keyed by column header (for exports/CSV) and supports value-only payloads with numeric parsing and locale decimal handling.
  - For standard cell-map requests iterates rows and columns, triggers `queryCellInfo` events, collects `getCell(..., false, true)` results and performs span-fixup by calling `setCell` to mark filler cells for merged regions.
  - Guards frozen region reads with `frozenIndexes` and skips hidden rows/cols when appropriate.
- `getValueFromFormat(...)`:
  - If a formula exists and value missing, notifies `calculateFormula` to resolve computed value.
  - If `cell.format` present, notifies `getFormattedCellObject` and returns formatted text or date object depending on flags.
  - For raw values applies number-to-string conversion, DPR-aware decimal parsing via `parseDecimalNumber`, and returns empty string for blank cells.
- `getModel(...)`:
  - Ensures array slots exist up to requested index, injects nulls when missing, and assigns `index` values to models that include them.
- `processIdx(...)`:
  - Normalizes index gaps across model arrays, optionally creates missing sheets, sets `usedRange` for sheets, and calls `processComments`/`processNotes` for cell-level metadata during imports.

## Validation & Safety
- Data sync hook: `getData` notifies `updateSheetFromDataSource` and waits on `args.promise` to ensure external range data is loaded before rasterization.
- Span repair: detects `rowSpan`/`colSpan` > 1 and writes filler cells via `setCell` to prevent renderer misreads.
- Hidden/frozen awareness: honors `isHiddenRow`/`isHiddenCol`, uses `frozenIndexes` to skip over frozen regions when serving virtualized ranges.
- Formula safety: defers to `calculateFormula` when needed so dependent evaluations are consistent and not stale.
- Parsing robustness: numeric parsing respects locale decimals and falls back safely on parse errors.

## Desired Outputs
- User-Facing:
  - Correct rendering data for visible viewport and exports: formatted strings, dates as objects (when requested), and consistent merged-cell behavior.
  - Stable results even when data-sources or dynamic ranges are present (print/export/render waits for data updates).
- System-Level:
  - Return types: `Promise<Map<string, CellModel>>` for normal reads; column-wise arrays for export-like requests.
  - Notifications triggered: `updateSheetFromDataSource`, `queryCellInfo`, `calculateFormula`, `getFormattedCellObject`, `parseDecimalNumber`.
  - Side-effects: `processIdx` may call `context.createSheet(...)` when sheet indexes include gaps; `getData` may call `setCell` to insert merge placeholders.
  - Performance: supports skipHiddenRows and frozen-index-aware iteration to minimize work for virtualized renderers.