# JOE storage

Agent reference for how objects are saved and read. The code in `server/modules/Storage.js` and `server/modules/Mongo.js` wins if this file drifts.

## Path

Callers use `JOE.Storage`, not the driver.

- `JOE.Storage.load(collection, query, callback)` reads. If the schema sets `storage.type`, that backend is used (`mongo`, `mysql`, `file`, `api`). Otherwise Mongo is used when `JOE.Mongo` is connected.
- `JOE.Storage.save(data, collection, callback, specs)` writes. `specs.history` defaults to true. `specs.user` is stored on the history row.
- `JOE.Mongo.get`, `save`, `saveHistory`, and `delete` are the Mongo adapter. `save` calls back with `(err, document)`, the object that was written, not the driver write result.
- MCP `saveObject` / `saveObjects` and `/API/save` both end in `Storage.save`. A save replaces the whole document. A field subset wipes everything omitted. Get the full object, merge, then save the full object.

After a successful save, Storage emits `item_updated`, runs schema events (`create`, `save`, and `status` or `delete` when those changed), and writes one `_history` row.

## Ids

New JOE objects use a `cuid()` string as `_id`. That string is a 36-character UUID. The editor sets it on create. If a save reaches `Mongo.save` without an `_id`, Mongo assigns `cuid()` there.

Two collections do not get a cuid:

- `_history` is written by `saveHistory` with no `_id`. Mongo assigns an ObjectId. Other records point at the subject with `itemid`, not the history row's `_id`.
- `comments` are saved without a cuid. Mongo assigns an ObjectId. Lookups use `itemid`.

Older documents may already have a 24-character ObjectId. `Mongo.get` and `Mongo.save` cast those to `ObjectId` so the query matches the stored type. A cuid is left as a string. Do not rewrite existing `_id` values. Anything that stored the old id would break.

## Connection strings

`joedb` on the instance config is the database URI.

- `mongodb://user:pass@host:27017/dbname` for a classic host.
- `host/db` or `host:port/db` still works. It is expanded to `mongodb://host:27017/db`.
- `mongodb+srv://...` for Atlas.

Standalone servers (no `replicaSet` and not `mongodb+srv`) connect with `retryWrites: false`. A current Atlas cluster and a MongoDB 4.4 standalone both use this module.

`mongojs` is no longer the runtime driver. It remains in the repo for `node_scripts/migrate-complete-to-archived.js`. Do not add new calls to it.

## Reads and the deleted flag

`Mongo.get` adds `deleted: { $in: [false, 'false', 'False', null] }`. A missing `deleted` field matches `null` in that list, so normal documents are returned. `Mongo.delete` sets `deleted` to an ISO timestamp and saves the full document. It does not remove the row.

Cache is a copy for fast reads. `JOE.Cache.update` reloads collections into `JOE.Data`. After a write, read storage when the saved document is what must be confirmed. Use cache for search and display.

## History

`Storage.save` builds the history payload. It stores `itemid`, `collection`, `user`, `timestamp`, and a sanitized diff on updates. The diff is for the audit row only. The document written to the collection is the object passed to `save`.

Do not insert into `_history` from a new feature. Go through `Storage.save` so events and history stay on one path.

## Admin page

`/mongo.html` is for an admin or super user. The same gate covers:

- `GET /API/mongo/status` — connected host and database name, server version, data size, and per-collection counts. The password is not included.
- `GET /API/mongo/dump` — streams a gzip `mongodump` archive. The machine must have MongoDB Database Tools installed (`mongodump` on the PATH). This is not an npm package.
- `POST /API/mongo/copy` with `{ uri }` — copies each collection into that database with an upsert on `_id`, then copies indexes. It does not change this JOE's `joedb`. The URI is not stored and is not written to the log. `GET /API/mongo/copy/:id` returns progress only.

A copy onto the same host and database is rejected.