# @pi-stef/finance-api

Always-on local service for financial data ingestion, storage, and deterministic quant analysis. Backed by SQLite; serves a bearer-token-authenticated HTTP API to the `@pi-stef/finance` extension and any other client.

---

## Quick start

### Docker (recommended)

```bash
cd packages/finance-api/docker
docker compose up -d
```

Pulls `ghcr.io/sfiorini/pi-stef/finance-api:latest` and starts the service. By default it binds to `127.0.0.1:7780` (localhost only) — if the pi client runs on a **different machine**, change the port mapping to `"7780:7780"` in `docker-compose.yml`. See the [Docker guide](docker/README.md#port-binding-same-machine-vs-remote-server) for details, image tags, volumes, and retrieving the token.

### Native

```bash
pnpm install
pnpm serve
```

See [docs/native-run.md](docs/native-run.md) for launchd/systemd setup.

### Verify

```bash
curl http://127.0.0.1:7780/v1/health
# {"ok":true,"data":{"status":"ok","uptimeS":0}}
```

---

## Architecture: server vs client

finance-api is a **server** that typically runs in Docker on a machine you control. The **client** is the `finance` extension, which runs wherever you use pi. Their files live on **different machines**:

| Component | Runs where | Files |
|-----------|-----------|-------|
| **finance-api** (server) | Docker container or native on a server | Token, SQLite DB, `secrets.json` — inside the container (`/root/.pi/sf/finance/` or `/data/`) or on the server (`~/.pi/sf/finance/`) |
| **finance** (client) | Inside pi, on your workstation | `config.json` — at `~/.pi/sf/finance/config.json` on **your** machine |

When the docs say `~/.pi/sf/finance/`, check whether they're referring to the **server** (inside the Docker container) or the **client** (your workstation). The sections below make this explicit.

## Authentication

All endpoints except `/v1/health` require a bearer token via the `Authorization` header:

```
Authorization: Bearer <token>
```

**Token lifecycle:**

- On first start, the service generates a random UUID token and writes it to `~/.pi/sf/finance/token` (`chmod 600`), created atomically and race-safe via `O_EXCL`.
- The token is stable across restarts as long as the token file persists.
- In Docker, the token is stored inside the container at `/root/.pi/sf/finance/token` and persists via the `finance-config` volume. Retrieve it with:
  ```bash
  docker compose exec finance-api cat /root/.pi/sf/finance/token
  ```

**Override:** Set `SF_FINANCE_TOKEN` to pin a specific token (useful for CI or sharing across hosts).

The `@pi-stef/finance` extension reads this token automatically when co-located on the same host.

---

## Configuration

All configuration is via environment variables (prefix `SF_FINANCE_`):

| Variable | Default | Description |
|----------|---------|-------------|
| `SF_FINANCE_HOST` | `127.0.0.1` (`0.0.0.0` in Docker) | Server bind host |
| `SF_FINANCE_PORT` | `7780` | Server port |
| `SF_FINANCE_DB` | `~/.pi/sf/finance/finance.db` (`/data/finance.db` in Docker) | SQLite database path |
| `SF_FINANCE_TOKEN` | (auto-generated) | Bearer token (overrides the token file) |
| `SF_FINANCE_DATA_FEED` | `stooq` | Price data feed (`stooq`) |

### Provider credentials

Working providers in this release do **not** use the server's `secrets.json`:

- **SnapTrade** — credentials live in the **client's** `config.json` on your workstation and are sent per-request. See [SnapTrade setup](#snaptrade-setup) below.
- **SimpleFIN** — credentials live in the **client's** `config.json` on your workstation and are sent per-request. See [SimpleFIN setup](#simplefin-setup) below.
- **File Import** — no stored credentials; the file path is provided per-request.

The `secrets.json` file (at `~/.pi/sf/finance/secrets.json` on the **server**) is for server-side providers. Coinbase uses server-side CDP API keys (see the [Coinbase guide](https://sfiorini.github.io/pi-stef/packages/finance-api-coinbase.html)):

```json
{
  "coinbase": {
    "keyName": "your-api-key",
    "privateKey": "your-private-key"
  }
}
```

Each provider's required credentials are documented under [Providers](#providers).

---

## Providers

| Provider | Kind | Auth | Status |
|----------|------|------|--------|
| File Import (CSV/OFX) | brokerage/banking | `filePath` or `content` (per-request) | ✅ Working |
| Coinbase | crypto | `keyName` + `privateKey` (in `secrets.json`) | ✅ Working (CDP ES256 JWT) |
| SnapTrade | brokerage | `clientId` + `consumerKey` (in client `config.json`, passed per-request) | ✅ Working |
| SimpleFIN | banking | `setupToken` → `accessUrl` (in client `config.json`, auto-persisted after first sync) | ✅ Working |

**Provider setup:**

Providers are **co-equal** — you can enable any combination, and multiple providers can run side by side (e.g. SnapTrade for live brokerage sync *and* File Import for a bank OFX export). Each provider is documented on its own page:

- [File Import](#file-import-csvofx) — manual CSV/OFX uploads via `/v1/import`
- [SnapTrade](#snaptrade-setup) — live brokerage aggregation (30+ brokers)
- [SimpleFIN](#simplefin-setup) — live banking data (balances + transactions)

> **⚠️ Cross-provider deduplication is not supported yet.** If the same real-world account surfaces through two providers (e.g. imported via CSV *and* synced via SnapTrade), it appears as **two separate accounts** — there is no mechanism today to recognize and merge them. This is tracked for a future release. For now, use one provider per account to avoid double-counting.

**SimpleFIN** — SimpleFIN is a working provider — see [SimpleFIN setup](#simplefin-setup) below.

### SnapTrade setup

SnapTrade aggregates brokerage accounts (Fidelity, Vanguard, Schwab, Robinhood, and 30+ others) behind a unified API. This integration uses a **Personal API key** — your brokerage connections live under your own SnapTrade Personal account, and identity is resolved from the signed `consumerKey` on every request.

> **v1 supports Personal accounts only.** The Commercial model (`userId`/`userSecret` via `registerSnapTradeUser`) is intentionally not supported.

**Credentials live in the client config** on the machine where you run pi (`~/.pi/sf/finance/config.json` on **your workstation**, not the finance-api server), **not** in the server's `secrets.json`. This keeps one finance-api deployment able to serve different SnapTrade users — each caller passes its own key per request.

```json
{
  "apiUrl": "http://127.0.0.1:7780",
  "token": "<service bearer token>",
  "providers": {
    "snaptrade": {
      "clientId": "PERS-...",
      "consumerKey": "<your personal consumer key>"
    }
  }
}
```

**Self-provision once** at [snaptrade.com](https://snaptrade.com):
1. Create a **Personal** account.
2. Open the Connection Portal and connect each of your brokerage accounts (Fidelity, Vanguard, Schwab, …). Connections are managed out-of-band on the SnapTrade dashboard — there are no in-service endpoints for it.
3. Copy your Personal `clientId` + `consumerKey` from the dashboard.
4. Add them to the client `config.json` under `providers.snaptrade` as above.

**How a sync works:** SnapTrade is **on-demand only** — the always-on server daemon does not poll SnapTrade on its own (it has no server-side key). Trigger a sync from the `finance` extension:
- `sf_fin_sync_now` (no args) → syncs **all** providers, attaching your Personal SnapTrade key.
- `sf_fin_sync_now({ provider: "snaptrade" })` → syncs **only** SnapTrade.

The client sends `clientId` + `consumerKey` in the request body of `/v1/sync`; the server uses them for that single tick and stores nothing.

**What is synced:** positions (equities/ETFs/mutual funds/crypto — options/futures are out of scope), transactions (incremental, id-keyed — only new activity since the last sync is fetched), and cash balance.

**Polling & rate limits:** each sync serializes accounts; SnapTrade's customer-level limit is 250 requests/minute. A `429` surfaces as a normal ingest error and is retried on the next sync — no special throttling is required for v1.

**Limitations (v1):** short positions are skipped (the data model cannot represent negative quantity); the SnapTrade payee/description field is not captured on transactions; connection management (connect/revoke) happens out-of-band at snaptrade.com. The imported `balance.marketValue` is the SnapTrade-reported **total account value** (cash + positions), not the position-only market value. Only Personal accounts are supported.

---

### SimpleFIN setup

SimpleFIN aggregates bank account data (checking, savings, credit cards) via the SimpleFIN Bridge. Unlike SnapTrade (investment positions), SimpleFIN provides **balances and transactions** — no holdings/positions (banking accounts have no equity).

**Credentials live in the client config** on the machine where you run pi (`~/.pi/sf/finance/config.json` on **your workstation**, not the finance-api server), **not** in the server's `secrets.json`.

#### Auth flow (setup token → access URL)

SimpleFIN uses a one-time token exchange:

1. You obtain a **setup token** from the SimpleFIN Bridge (a base64-encoded URL, one-time use).
2. On the first sync, the finance-api server POSTs to the decoded URL and receives an **access URL** (persistent, embeds Basic Auth credentials).
3. The server returns the access URL in the sync response. The finance extension automatically writes it to your `config.json`, replacing the setup token.
4. Future syncs use the access URL directly — no re-exchange.

```json
{
  "apiUrl": "http://127.0.0.1:7780",
  "token": "<service bearer token>",
  "providers": {
    "simplefin": {
      "setupToken": "aHR0cHM6Ly9iZXRhLWJyaWRnZS5zaW1wbGVmaW4ub3JnL3NpbXBsZWZpbi9jbGFpbS8uLi4="
    }
  }
}
```

After the first sync, config is automatically updated:

```json
{
  "providers": {
    "simplefin": {
      "accessUrl": "https://user:pass@bridge.simplefin.org/simplefin"
    }
  }
}
```

> **Note:** Setup tokens are one-time use. Once exchanged, the token is dead. The finance extension handles persistence automatically — you never need to manually update the config.

#### Self-provision once

1. Visit [beta-bridge.simplefin.org](https://beta-bridge.simplefin.org) and create a SimpleFIN account.
2. Generate a setup token from the SimpleFIN Bridge dashboard.
3. Add it to your client `config.json` under `providers.simplefin.setupToken`.
4. Run `sf_fin_sync_now` — the adapter exchanges the token and persists the access URL.

#### What is synced

- **Accounts** — bank accounts (checking, savings, credit cards) with balances.
- **Balances** — current balance per account, persisted as cash.
- **Transactions** — deposits, withdrawals, payments (credit/debit classification). Pending transactions are excluded.
- **No holdings** — banking accounts have no equity positions.

#### Rate limits

SimpleFIN limits to **24 requests per day**. Each sync uses ~1 request (one `/accounts` call that returns all accounts with balances and transactions). This is well within the limit for daily or even hourly syncs.

The transaction date range is limited to **90 days** by the SimpleFIN server. Historical transactions beyond 90 days are not available.

#### curl examples

```bash
# Sync SimpleFIN (first time — exchanges setup token)
curl -X POST http://127.0.0.1:7780/v1/sync \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"providers":["simplefin"],"credentials":{"simplefin":{"setupToken":"your-setup-token"}}}'

# Sync SimpleFIN (subsequent — uses persisted access URL)
curl -X POST http://127.0.0.1:7780/v1/sync \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"providers":["simplefin"],"credentials":{"simplefin":{"accessUrl":"https://user:pass@bridge.simplefin.org/simplefin"}}}'

# Check banking holdings (balances)
curl -X GET http://127.0.0.1:7780/v1/holdings \
  -H "Authorization: Bearer $TOKEN"

# Check banking transactions
curl -X GET http://127.0.0.1:7780/v1/history?symbol=USD \
  -H "Authorization: Bearer $TOKEN"
```

---

## File Import (CSV/OFX)

File Import is a co-equal provider that ingests holdings and transactions from manual file uploads via `POST /v1/import`. It covers two formats: CSV (for holdings/positions from any brokerage export) and OFX (for transactions + cash balance from bank exports). This is a good fit for institutions not covered by SnapTrade, or when you prefer not to grant API access.

This section covers everything you need to know — the exact formats accepted, how to export from common brokerages and banks, and what to do when your export isn't directly supported.

### Supported formats

| Format | Extension | Data imported | Use case |
|--------|-----------|---------------|----------|
| CSV | `.csv` | Holdings (positions) | Brokerage exports (any broker that exports a positions CSV) |
| OFX | `.ofx`, `.qfx` | Transactions + balances | Bank exports (checking, savings) |

The service detects the format automatically: `.csv` → positions, OFX → transactions + cash balance.

### CSV format specification

The CSV parser (`src/ingest/file/csv.ts`) expects a **positions-style CSV** — one row per holding, with a header row. The parser is flexible on column names but strict about what data it extracts.

#### Required columns

| Column | Accepted header names (case-insensitive, trimmed) | Example value | Notes |
|--------|---------------------------------------------------|---------------|-------|
| **Symbol** | `symbol` | `AAPL`, `FXAIX`, `VTI` | Must be non-empty. Any string works (no ticker validation). |
| **Quantity** | `quantity`, `shares`, `qty` | `10`, `5.123`, `100` | Must be a non-zero number. Whitespace/symbols stripped (see below). |

#### Optional columns

| Column | Accepted header names | Example value | Notes |
|--------|----------------------|---------------|-------|
| **Price** | `last price`, `price` | `190.50`, `$3,450.00` | Stored as the holding's `price` (a sync-time snapshot), separate from `avg_cost`. The valuation layer prefers the latest price from the prices table, falling back to this `price`, then `avg_cost`. |
| **Cost basis** | `average cost basis`, `cost basis` | `$4,164.66`, `180.00` | Stored as `avgCost`. When absent, equities fall back to the `Price` column; crypto leaves `avgCost` unset. |

> **BOM handling:** a leading UTF-8 BOM (`\uFEFF`) is automatically stripped before parsing, so exports from Excel/Windows that include a BOM import correctly.

#### How numeric values are parsed

Numeric parsing differs by field:

- **Quantity** — stripped to digits/dot/hyphen and forced positive:
  ```
  cols[idx].replace(/[^0-9.\-]/g, "")  →  Number(...)  →  Math.abs()
  ```
  A value of `-10` becomes `10`; short positions cannot be imported.
- **Price and Cost basis** — parsed by `parseCurrency()`:
  ```
  val.replace(/[$,\s]/g, "").replace(/^\((.+)\)$/, "-$1")  →  Number(...)
  ```
  This strips `$`, commas, and whitespace, and converts accounting-style parenthesized negatives to a leading minus: `($4,164.66)` → `-4164.66`. So `($190.50)` is preserved as a negative price/cost.

| Input | Quantity (strip + `abs`) | Price / Cost basis (`parseCurrency`) |
|--------|--------------------------|--------------------------------------|
| `$1,234.56` | `1234.56` | `1234.56` |
| `10` | `10` | `10` |
| `5.123` | `5.123` | `5.123` |
| `($4,164.66)` | `4164.66` | `-4164.66` |
| `(190.50)` | `190.50` | `-190.50` |
| `-10` | `10` | `-10` |

> ⚠️ **Negative quantities become positive.** `Math.abs()` forces quantity positive, so `-10` (a short) imports as `10`. Short positions are not currently supported. This applies to quantity only — parenthesized prices and cost bases keep their sign.

#### Notes & limitations

1. **Quoted fields are supported.** The parser uses a proper state machine (`parseCsvLine`) that handles values wrapped in double quotes, including embedded commas (e.g. `"Apple, Inc."`) and escaped doubled quotes (`""`). You no longer need to strip commas from text fields.

2. **Crypto auto-detection.** Symbols matching `XXX/USD`, `XXX/USDT`, `XXX/EUR`, or `XXX/GBP` (e.g. `BTC/USD`, `ETH/USD`) are classified as `assetClass: "crypto"` with `securityType: "crypto"`; the stored symbol is the base currency (e.g. `BTC`). All other symbols import as `equity` with `subclass: "us"`. Bonds and cash are not auto-detected from CSV.

3. **Any column beyond the 4 recognized ones is ignored.** Extra columns like `Description`, `Account`, and `Last Price Change` are silently discarded.

4. **Empty/zero-quantity rows are silently skipped.** If a row has an empty symbol or a zero/NaN quantity, it's dropped with no warning. A CSV with only a header and no data rows returns `[]`.

#### Minimum valid CSV

```csv
Symbol,Quantity
AAPL,10
FXAIX,5.123
```

#### Fully specified CSV (typical brokerage export)

```csv
Account,Symbol,Description,Quantity,Last Price,Cost Basis
Brokerage,AAPL,"Apple, Inc.",10,190.50,150.25
Brokerage,VTI,Total Stock Market,5.123,180.00,170.00
Brokerage,BTC/USD,Bitcoin,0.05,64000.00,45000.00
```

Most brokerages export positions with a header row like `Account,Symbol,Description,Quantity,Last Price` (sometimes `Shares` instead of `Quantity`, or `Price` instead of `Last Price`). The parser accepts all of these. `Cost Basis` / `Average Cost Basis` is optional and is stored as `avgCost`. Symbols of the form `BTC/USD` are auto-classified as crypto.

### OFX format specification

The OFX parser (`src/ingest/file/ofx.ts`) supports standard OFX 1.x / QFX files (the format used by most banks for transaction downloads).

#### What the parser extracts

| XML element | Mapped to | Default if absent |
|-------------|-----------|-------------------|
| `<ACCTID>` | Account ID | `"unknown"` |
| `<BALAMT>` | Cash balance | `0` |
| `<STMTTRN>` → `<TRNAMT>` | Transaction amount | `0` |
| `<STMTTRN>` → `<DTPOSTED>` | Transaction date (YYYYMMDD or YYYYMMDDHHMMSS) | unix epoch `0` (1970-01-01) |
| `<STMTTRN>` → `<NAME>` | Payee name (parsed by `parseOfx`) | `""` (empty) |

#### What reaches the API response

The file adapter (`src/ingest/file/index.ts`) transforms the parsed OFX data before it reaches the API:

| Field | Parser layer | Adapter layer (API response) |
|-------|-------------|------------------------------|
| Balance → `RawBalance` | `parseOfx().balance` | `{ cash: balance, marketValue: 0, asOf: timestamp }` — treated as cash |
| Transactions → `RawTxn[]` | `parseOfx().transactions` | `{ id: "${arrayIndex}", date: unixMs, type: "credit"\|"debit", fees: 0 }` |
| Payee (`<NAME>`) | ✅ Parsed by `parseOfx` | ❌ **Discarded** — not present in API response |
| Fees | Not parsed | Hardcoded to `0` |
| Symbol/quantity | N/A | N/A — OFX imports transactions, not holdings |

> ⚠️ **`.qfx` detection is by content (OFXHEADER literal), not by extension.** The parser detects OFX when the file has a `.ofx` extension OR when the file content contains the literal string `OFXHEADER` (which standard `.qfx` exports always include). A `.qfx` file without `OFXHEADER` would not import.

> ⚠️ **OFX holdings are always empty.** OFX is for banking transactions. To import positions (stocks/ETFs), use CSV.

#### Date parsing

OFX dates are parsed as `YYYYMMDD` or `YYYYMMDDHHMMSS`. If the date string is empty or shorter than 8 characters, the date defaults to unix epoch `0` (displayed as `1970-01-01`). In practice this never happens with real OFX exports.

#### Detection

- The file adapter checks for `.ofx` extension for the initial routing (in `getHoldings`).
- Files containing the literal string `OFXHEADER` anywhere in the content are treated as OFX (in `getTransactions` and `getBalances`; this is how `.qfx` files work — they always contain `OFXHEADER`).
- A file without `.ofx` extension AND without `OFXHEADER` content will not be parsed as OFX.

> ⚠️ **The payee name (`<NAME>`) is parsed from the OFX file but discarded by the adapter.** Imported transactions have no payee/description field. This is a known gap.

### How to export from your brokerage

The CSV parser accepts any positions export with a `Symbol` column and a `Quantity`/`Shares`/`Qty` column (an optional `Last Price`/`Price` column is read as average cost). The exact menu path varies by brokerage, but the steps are the same everywhere:

1. **Log into your brokerage's website** and navigate to your Portfolio / Positions / Holdings page.
2. **Select the account** you want to export.
3. **Look for Download / Export** — typically a down-arrow icon or a link near the positions table header.
4. **Choose CSV format** (not PDF, not Excel).
5. **Save the file** — it typically downloads as `Positions_<date>.csv` or similar.
6. **Check the headers** — you need `Symbol` and one of `Quantity`/`Shares`/`Qty`. `Last Price`/`Price` and `Cost Basis`/`Average Cost Basis` are optional. Extra columns (Description, Account, …) are ignored.
7. **Import the file:**
   ```bash
   curl -X POST http://127.0.0.1:7780/v1/import \
     -H "Authorization: Bearer $(cat ~/.pi/sf/finance/token)" \
     -H "Content-Type: application/json" \
     -d '{"filePath":"/Users/me/Downloads/positions.csv"}'
   ```
8. **Verify the import worked:**
   ```bash
   curl -X GET http://127.0.0.1:7780/v1/holdings \
     -H "Authorization: Bearer $(cat ~/.pi/sf/finance/token)"
   ```

> **Tip:** `Last Price`/`Price` is stored as the holding's `price`, and `Cost Basis`/`Average Cost Basis` as `avgCost` (equities fall back to the price column when no cost basis is present). The `/v1/net-worth` endpoint prefers the latest price from the price feed, falling back to `price` then `avg_cost`. So a missing or stale price column is largely harmless.
>
> **Prefer live sync?** If your brokerage is one of the 30+ supported by SnapTrade (Fidelity, Vanguard, Schwab, Robinhood, …), [SnapTrade](#snaptrade-setup) gives you automatic live sync with no manual exports.

### Exports that need adjustment

Some institutions export data in a shape the CSV parser can't read directly. This is not an exhaustive list — the same patterns apply to any similar export.

#### Crypto exchanges (e.g. Coinbase)

**Reason:** Crypto exchanges export **transaction history**, not portfolio positions. A typical CSV has columns like:

```
Timestamp,Transaction Type,Asset,Quantity Transacted,Spot Price,Subtotal,Total,Notes
```

There is no `Symbol` column (the parser's required key). Even if renamed, the data represents *transactions* (buys/sells/transfers), not *current holdings*. The parser would need significant changes to aggregate transactions into a portfolio snapshot.

**What you can do today:**
1. Manually create a CSV with `Symbol,Quantity` columns from your exchange balances.
2. Get your current balances from the exchange UI (Dashboard → each asset → balance).
3. Write a CSV like:
   ```csv
   Symbol,Quantity
   BTC/USD,0.05
   ETH/USD,2.0
   ```
   Use the `BASE/USD` form so the parser classifies the row as crypto (`assetClass: "crypto"`, `securityType: "crypto"`). A bare `BTC` would import as equity.
4. Import with `POST /v1/import`.

**Note:** A direct Coinbase API provider (`src/ingest/direct/coinbase.ts`) is implemented and pulls positions via CDP ES256-JWT API keys. See the [Coinbase guide](https://sfiorini.github.io/pi-stef/packages/finance-api-coinbase.html).

#### Banks (e.g. Bank of America)

**Reason:** Banks export **account activity** (transactions), not portfolio positions. A typical bank CSV has columns like:

```
Date,Description,Amount,Running Bal.
```

No `Symbol` column. No `Quantity` column. The data is a transaction ledger, not a position list.

**What you can do today:**
- **OFX/QFX:** Most banks support OFX download (via "Download Transactions" → choose "Microsoft Money" or "Quicken" format). This is the **recommended path** — the OFX parser handles these files correctly for transaction history and cash balance.
  ```bash
  curl -X POST http://127.0.0.1:7780/v1/import \
    -H "Authorization: Bearer $(cat ~/.pi/sf/finance/token)" \
    -H "Content-Type: application/json" \
    -d '{"filePath":"/Users/me/Downloads/bank-activity.ofx"}'
  ```
- **For positions** held in a bank's investing arm (e.g. Merrill Edge for BoA), download a positions export from the investment section — these typically follow the `Symbol,Quantity,Last Price` format the parser accepts.
- **Live sync:** If the brokerage side is supported by SnapTrade, [SnapTrade](#snaptrade-setup) gives you automatic sync.

#### If your CSV doesn't match

Most brokerages export positions with `Symbol`, `Quantity`/`Shares`, and optionally a `Last Price`/`Price` column. If your export has these columns under any accepted header name, it will import. If the import returns empty holdings, check that your CSV has `Symbol` and `Quantity`/`Shares`/`Qty` in the header row (`head -1 your-file.csv`).

### curl examples

**Import CSV (holdings):**
```bash
curl -X POST http://127.0.0.1:7780/v1/import \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"filePath":"/absolute/path/to/positions.csv"}'
```

**Import OFX (transactions):**
```bash
curl -X POST http://127.0.0.1:7780/v1/import \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"filePath":"/absolute/path/to/transactions.ofx"}'
```

**Import via `content` (remote deployments):** When the file is not reachable from the server's filesystem — e.g. the `finance` extension runs on a different machine than finance-api — send the file contents directly instead of a path. `content` is the raw (UTF-8) file text; `filename` is optional and used only to detect the format (`.csv` vs `.ofx`):

```bash
# Use jq to safely embed the file as a JSON string
jq -n --rawfile content /absolute/path/to/positions.csv \
      '{filename:"positions.csv", content:$content}' |
  curl -X POST http://127.0.0.1:7780/v1/import \
    -H "Authorization: Bearer YOUR_TOKEN" \
    -H "Content-Type: application/json" \
    -d @-
```

> **`finance` extension:** the extension always reads the file locally on the pi machine and sends `{ content, filename }` to the server — the file never needs to exist on the server. Use `filePath` only for direct API calls when the file lives on the server's filesystem.

**Verify after import:**
```bash
# Check holdings
curl -X GET http://127.0.0.1:7780/v1/holdings \
  -H "Authorization: Bearer YOUR_TOKEN"

# Check net worth (uses latest prices, not CSV avg costs)
curl -X GET http://127.0.0.1:7780/v1/net-worth \
  -H "Authorization: Bearer YOUR_TOKEN"

# Trigger a full sync (recompute suggestions from new data)
curl -X POST http://127.0.0.1:7780/v1/sync \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Troubleshooting imports

| Symptom | Likely cause | Fix |
|---------|-------------|-----|
| `{"ok":false,"error":{"code":"bad_request","message":"Either filePath or content is required"}}` | Neither `filePath` nor `content` in the JSON body | Add `"filePath": "/path/to/file.csv"` (file on the server) **or** `"content": "..."` with `"filename": "..."` (file contents) to your request body |
| `{"ok":false,"error":{"code":"bad_request","message":"Directory traversal is not allowed"}}` | Relative path contains `..` (e.g., `../../data.csv`) | Use an **absolute** path (`/Users/me/...`) or a relative path without `..` (`./data.csv`) — absolute paths are always allowed |
| Import succeeds but holdings are empty (`[]`) | CSV is missing `Symbol` or `Quantity`/`Shares`/`Qty` column in the header, or all rows have zero/empty quantities | Check your CSV header row — it must contain one of the accepted column names. Run `head -1 your-file.csv` to inspect |
| Imported quantities are wrong | CSV uses `(value)` for negatives (accounting convention) or `-value` for short positions | Short/negative quantities cannot be imported — `Math.abs()` forces all quantities positive. No workaround in the current parser. Remove negative rows or accept them as positive.
| Import succeeds but prices are wrong or missing | CSV has no `Last Price`/`Price` column, or the column has non-numeric characters `parseCurrency` can't parse | Add a price column or accept that holdings import without a stored `price`; valuation falls back to `avg_cost`, then the price feed |
| Imported OFX transactions have date `1970-01-01` | OFX date field is missing or malformed (`<DTPOSTED>` empty or `< 8 chars`) | Verify the OFX file is valid — real bank exports always include dates. If this happens, the file may be corrupted |
| Imported OFX transactions have no merchant name | Payee `<NAME>` is parsed by `parseOfx` but discarded by the adapter | This is a known limitation — transaction descriptions are not persisted. Tracked for a future release |
| `401 Unauthorized` on import | Token is missing or wrong | Retrieve the token: `cat ~/.pi/sf/finance/token` (native) or `docker compose exec finance-api cat /root/.pi/sf/finance/token` (Docker). Include `Authorization: Bearer <token>` in your request |

---

## HTTP API reference

Base URL: `http://127.0.0.1:7780`. All endpoints return `{ "ok": true, "data": {...} }` on success or `{ "ok": false, "error": { "code": "...", "message": "..." } }` on failure.

### Interactive docs

The API ships with auto-generated OpenAPI 3.1 documentation:

| Resource | URL | Description |
|----------|-----|-------------|
| **Swagger UI** | `http://127.0.0.1:7780/docs` | Interactive API explorer — try requests live |
| **OpenAPI JSON** | `http://127.0.0.1:7780/openapi.json` | Raw OpenAPI 3.1 spec (import into Postman, Insomnia, etc.) |

Both endpoints are public (no auth required) and are generated from the Zod route schemas, so they're always in sync with the code.

### Postman collection

A ready-to-import Postman collection and environment template are in the [`postman/`](postman/) directory:

1. Open Postman → Import
2. Select `postman/finance-api.postman_collection.json`
3. Import `postman/finance-api.postman_environment.json` as an environment
4. Set the `token` variable to your bearer token
5. All requests are pre-configured with `{{base_url}}` and `{{token}}` variables

To regenerate the collection after adding/changing routes:

```bash
npx tsx packages/finance-api/scripts/gen-postman.mjs
```

### `GET /v1/health` *(public)*

Health check; no auth required.

```json
{ "ok": true, "data": { "status": "ok", "uptimeS": 123 } }
```

### `GET /v1/market-status`

Returns the current US market session classification.

```json
{ "ok": true, "data": { "session": "regular", "timestamp": 1782000000000 } }
```

`session` is one of `pre`, `regular`, `post`, `closed`. Holiday list currently covers 2026.

### `GET /v1/holdings`

Accounts and their holdings, valued at the latest known price.

```json
{ "ok": true, "data": { "accounts": [
  { "id": "snaptrade:acct-1", "provider_id": "snaptrade", "kind": "brokerage", "name": "Brokerage",
    "total_value": 5105.00,
    "holdings": [
      { "account_id": "snaptrade:acct-1", "symbol": "AAPL", "quantity": 10, "avg_cost": 150.25, "asset_class": "equity", "subclass": "us", "price": 190.50, "security_type": null, "market_value": 1905.00, "gain_loss": 402.50, "as_of": 1782000000000 },
      { "account_id": "snaptrade:acct-1", "symbol": "BTC", "quantity": 0.05, "avg_cost": 45000.00, "asset_class": "crypto", "subclass": null, "price": 64000.00, "security_type": "crypto", "market_value": 3200.00, "gain_loss": 950.00, "as_of": 1782000000000 }
    ] }
] } }
```

Per-holding fields: `price` (latest price used for valuation), `market_value` (`quantity` × `price`), `gain_loss` (`market_value` − `quantity` × `avg_cost`, or `null` when `avg_cost` is unknown), `avg_cost`, and `security_type` (e.g. `"crypto"`, else `null`). Each account includes `total_value` = Σ holding `market_value` + unbilled cash.

### `GET /v1/net-worth`

Total portfolio value using latest prices (falls back to average cost).

```json
{ "ok": true, "data": { "netWorth": 123456.78, "accountCount": 3 } }
```

### `GET /v1/allocation`

Current asset allocation as flat weights by asset class.

```json
{ "ok": true, "data": { "allocation": { "equity": 0.72, "bonds": 0.18, "cash": 0.10 }, "totalValue": 123456.78 } }
```

### `GET /v1/drift`

Allocation drift vs the configured goal's target allocation.

```json
{ "ok": true, "data": { "drift": [
  { "class": "equity", "currentPct": 0.72, "targetPct": 0.80, "deltaPct": -0.08, "value": 88888.0 }
] } }
```

### `GET /v1/goals`

List investment goals (target allocation is parsed from stored JSON).

```json
{ "ok": true, "data": { "goals": [
  { "id": "g1", "name": "Growth", "targetAllocation": { "equity": 0.8, "bonds": 0.2 }, "riskLimits": {}, "horizon_years": 10 }
] } }
```

> Note: `target_allocation` and `risk_limits` are camelCased in the response (`targetAllocation`/`riskLimits`, parsed from JSON); `horizon_years` keeps its snake_case DB form.

### `POST /v1/goals`

Create or update (UPSERT) an investment goal. Validates that the target allocation sums to ~1.0.

**Request body:**

```json
{
  "id": "g1",
  "name": "Growth",
  "targetAllocation": { "equity": 0.8, "bonds": 0.2 },
  "riskLimits": { "maxConcentration": 0.25 },
  "horizonYears": 10
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | string | yes | Goal identifier |
| `name` | string | yes | Display name |
| `targetAllocation` | object | yes | Asset-class weights (must sum to ~1.0) |
| `riskLimits` | object | no | Risk limits (e.g. `maxConcentration`) |
| `horizonYears` | number | no | Investment horizon |

```json
{ "ok": true, "data": { "id": "g1" } }
```

### `GET /v1/suggestions`

Pending rebalance/risk/drift suggestions computed by the quant engine. Each suggestion's `payload` is parsed from stored JSON.

```json
{ "ok": true, "data": { "suggestions": [
  { "id": "s-...-0", "kind": "rebalance", "status": "pending", "payload": { "symbol": "AAPL", "action": "buy", "amount": 500 } }
] } }
```

### `POST /v1/suggestions/dismiss`

Dismiss a suggestion by id.

**Request body:** `{ "id": "s-...-0" }`

```json
{ "ok": true, "data": { "dismissed": "s-...-0" } }
```

### `POST /v1/sync`

Trigger a scheduler tick: ingest from providers, refresh prices, recompute suggestions.

**Optional request body:**

| Field | Type | Description |
|-------|------|-------------|
| `providers` | `string[]` | Scope ingest to a subset of providers (e.g. `["snaptrade"]`). Omit to ingest from all configured providers. |
| `credentials` | `object` | Per-provider credentials supplied per-call (request creds override server-side `secrets.json` creds for this tick; nothing is persisted). Used for Personal SnapTrade keys. |

Example — sync SnapTrade with a per-call Personal key:

```bash
curl -X POST http://127.0.0.1:7780/v1/sync \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"providers":["snaptrade"],"credentials":{"snaptrade":{"clientId":"PERS-...","consumerKey":"..."}}}'
```

```json
{ "ok": true, "data": {
  "message": "Sync complete",
  "session": "regular",
  "accountsIngested": 3,
  "holdingsIngested": 12,
  "pricesUpdated": 12,
  "suggestionsCreated": 2,
  "errors": []
} }
```

### `POST /v1/import`

Import holdings from a local file (CSV or OFX). Accepts **either** mode in the request body:

- **`filePath`** — the server reads the file from its own filesystem (local deployments). Absolute paths are allowed (single-user local service); relative paths containing `..` are rejected.
- **`content`** (+ optional `filename`) — the file contents are sent directly as a UTF-8 string (remote deployments). The `finance` extension always uses this mode: it reads the file locally and sends `content`, so the file never needs to exist on the server.

**Request body:** `{ "filePath": "/Users/me/Downloads/positions.csv" }` or `{ "filename": "positions.csv", "content": "Symbol,Quantity\nAAPL,10\n" }`

```json
{ "ok": true, "data": { "message": "Import complete", "filePath": "...", "accounts": 1 } }
```

### `GET /v1/history?symbol=AAPL[&accountId=...]`

Price history for a symbol, newest first. `accountId` optionally filters to prices relevant to a holding in that account.

```json
{ "ok": true, "data": { "history": [
  { "symbol": "AAPL", "date": 1782000000000, "close": 210.5, "source": "stooq" }
] } }
```

### `POST /v1/export`

Export data. `format: json` returns all tables inline; `format: sqlite` writes a backup copy of the database (restricted to the finance backup directory).

**Request body:** `{ "format": "json" }` or `{ "format": "sqlite", "path": "backup.db" }`

```json
// json
{ "ok": true, "data": { "holdings": [...], "prices": [...], ... } }
// sqlite
{ "ok": true, "data": { "backupPath": "/home/.pi/sf/finance/backups/backup.db" } }
```

---

## Data model

SQLite, stored at `SF_FINANCE_DB`. Versioned migrations (see `src/store/schema.ts`); future changes add migration entries rather than mutating existing tables.

| Table | Purpose |
|-------|---------|
| `accounts` | Linked accounts (provider, kind, name, mask, currency, staleness) |
| `holdings` | Current holdings per account/symbol (quantity, avg cost, price, asset class, security type, as-of) |
| `transactions` | Transactions (date, symbol, qty, price, type, fees) |
| `prices` | Price history per symbol/date (close, source) |
| `lots` | Tax lots per holding (open date, qty, cost basis) |
| `goals` | Investment goals (target allocation, risk limits, horizon) |
| `suggestion_records` | Persisted suggestions (kind, payload, status) |
| `market_sessions` | Cached market-session snapshots per date |

---

## Scheduler & quant engine

The built-in scheduler (`src/scheduler/`) runs a periodic tick whose cadence depends on the market session: more frequent intraday, hourly after hours, and every few hours when closed. Each tick:

1. **Ingests** fresh data from configured providers via the provider registry.
2. **Refreshes prices** from the configured data feed (default `stooq`).
3. **Recomputes suggestions** deterministically through the quant engine:
   - **Drift** — current vs target allocation deltas
   - **Rebalance** — buy/sell amounts to return to target
   - **Risk** — concentration and cash-drag checks against `riskLimits`
   - **DCA** — dollar-cost-averaging recommendations (where configured)

**Determinism:** all numbers are computed by pure functions in `src/quant/`. The LLM client applies judgment but never recomputes the figures. This keeps suggestions reproducible and auditable.

`POST /v1/sync` triggers a tick on demand.

---

## Backup & restore

- **JSON export:** `POST /v1/export {"format":"json"}` returns all data inline.
- **SQLite backup:** `POST /v1/export {"format":"sqlite"}` writes a timestamped `.db` copy to `~/.pi/sf/finance/backups/` (path is sandboxed to that directory).
- **Restore:** stop the service, replace `SF_FINANCE_DB` with the backup file, restart.

---

## Observability

Structured logs are emitted to stdout (JSON) with `level`, `msg`, and contextual fields. Key events: server start, ingest results, staleness warnings, tick summaries. Increase verbosity via your process supervisor's log level (the service logs at `info` by default).

---

## Security model

- **Local-first:** bind to `127.0.0.1` by default. Docker maps `127.0.0.1:7780:7780` (localhost only) so the service is not exposed to the LAN. For remote-server deployments, change to `"7780:7780"` — see the [Docker guide](docker/README.md#port-binding-same-machine-vs-remote-server).
- **Bearer auth:** every non-health endpoint requires a token; compared with `timingSafeEqual`.
- **Secrets:** `secrets.json` is `chmod 600`; provider credentials never leave the host.
- **File imports:** absolute paths are allowed (local file access by design); relative `..` traversal is rejected.
- **Backups:** the export route sandboxes SQLite backups to the finance backup directory.

---

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| `401 Unauthorized` | Retrieve/regenerate the token (see [Authentication](#authentication)); check `SF_FINANCE_TOKEN` |
| Port already in use | Change `SF_FINANCE_PORT` and the compose port mapping |
| Stale holdings | Run `POST /v1/sync`; check provider credentials (SnapTrade → client `config.json`, others → `secrets.json`) |
| `better-sqlite3` build fails (native) | Use the Docker image, or ensure `python3 make g++` are installed |
| No suggestions after sync | Set a goal via `POST /v1/goals` — drift/rebalance need a target |

---

## Cost

- **Free tier:** File imports (CSV/OFX) and `stooq` prices — no API costs.
- **Optional:** Coinbase API (free, view-only scope) — currently a stub.
- **SnapTrade:** live brokerage aggregation (Fidelity, Vanguard, Schwab, Robinhood, 30+ others — see [SnapTrade setup](#snaptrade-setup)).
- **SimpleFIN:** live banking data (balances + transactions — see [SimpleFIN setup](#simplefin-setup)). Free tier available; premium features may have fees.

---

## Disclaimer

**This is not financial advice.** The service provides deterministic calculations based on your data and configured goals. Suggestions are informational only — no trades are executed automatically. Always consult a qualified financial advisor before making investment decisions.

## License

[MIT](../../LICENSE)