# JOE Universal AI Connectivity — MCP Modernization Plan

**Status:** 0.11.0 ships parallel MCP v2. Full per-user OAuth and skill generation remain later.  
**Owner:** JOE Core  
**Initiative:** JOE Universal AI Connectivity  
**First pilot:** Haddens JOE  
**Second validation:** Project 60 JOE  
**Primary implementation environment:** local JOE repository in Cursor/Codex

## 0.11.0 implementation
Legacy `POST /mcp` and `GET /.well-known/mcp/manifest.json` stay on JOE cookie or HTTP Basic auth. Standard stateless Streamable HTTP is `POST /mcp/v2` and accepts only `Authorization: Bearer` with an agent token stored hashed on a JOE user. Mint and revoke tokens, then try `initialize`, `tools/list`, and `tools/call`, at `/mcp-modernization.html`. Skills autogeneration and the ChatGPT OAuth handoff are not in this release. Do not point new agents at `/mcp`.

## 1. Objective and non-goals
Enable any suitably authorized MCP-compatible AI assistant to discover, understand and operate on JOE objects in any JOE instance, using JOE as the persistent system of record. One reusable implementation should work for Haddens, P60, Well Time and future installations without instance-specific agent APIs.

Success means interoperable tools, dynamic schema awareness, authenticated and authorized operations, preserved history, and repeatable onboarding.

**Out of scope:** rewriting JOE Storage/Cache/Schemas; rebuilding the JOE UI; creating a second database; dumping all instance content into model prompts; implementing P60 domain schemas, SharePoint import or the P60 dashboard; building distinct APIs per assistant; retaining obsolete OpenAPI/CustomGPT integration by default.

## 2. Current baseline (verify against working tree)
Prior inspection of `server/modules/MCP.js` found:
- `GET /.well-known/mcp/manifest.json` returns a custom manifest of JOE tools, descriptions, parameters, return shapes and toolsets.
- `POST /mcp` uses custom JSON-RPC calls with the tool name as the RPC `method` (for example, `"method":"getSchema"`), not standard MCP `tools/call`.
- `MCP.tools`, `MCP.params`, `MCP.returns`, `MCP.descriptions` and `MCP.toolsets` constitute useful reusable metadata/handler definitions.
- Route setup uses `JOE.auth` middleware if present, otherwise registers an unauthenticated POST route; the route handler has no independent per-tool role enforcement. This must not be carried into production MCP.

Currently exposed tool surface (from the previously reviewed live manifest; check current code):

| Domain | Tools |
| --- | --- |
| Schemas | `listSchemas`, `getSchema`, `getSchemas` |
| Objects / context | `getObject`, `understandObject`, `hydrate` |
| Search | `search`, `fuzzySearch`, `findObjectsByTag`, `findObjectsByStatus` |
| Persistence | `saveObject`, `saveObjects` |
| Applications | `listApps` |
| Agents / workflows | `compilePipeline`, `runThoughtAgent` |

Related modules: `server/modules/Schemas.js`, `Storage.js`, `Mongo.js`, `server/init.js`. Master Knowledge describes dynamic `_schemas`, `_apps` and `_plugins`, cache-oriented search and canonical storage writes with history/events. Treat older file documentation as context, not a guarantee of current behavior.

## 3. Three deliverables

### A. JOE MCP v2 (JOE Core)
- Use an official maintained MCP TypeScript/JavaScript SDK compatible with JOE's Node runtime.
- Expose standards-based **Streamable HTTP** at an instance endpoint such as `POST /mcp`, with SDK-managed request/session lifecycle and transport requirements. Verify required HTTP methods, headers, lifecycle and protocol-version handling against the selected SDK.
- Support MCP `initialize`, `tools/list`, and `tools/call`. Make the existing JOE tools available with **machine-valid JSON Schema input definitions** and useful descriptions, output schemas where supported. Do not assume current `MCP.params` metadata already conforms to JSON Schema; inspect and adapt.
- Reuse JOE tool handlers and schema summaries; preserve `understandObject`, selective `hydrate`, search, and canonical save behavior.
- Optional MCP resources/prompts only if an actual use case needs them. No separate REST endpoint per tool.
- Ensure tool discovery and invocations are scoped by authenticated identity and permissions.

**Conceptual translation:**

Old custom JSON-RPC:
```json
{"jsonrpc":"2.0","id":1,"method":"getSchema","params":{"name":"project"}}
```

Standard MCP tool invocation (inside the SDK-managed MCP transport):
```json
{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"getSchema","arguments":{"name":"project"}}}
```

### B. JOE Agent Skill (generated by JOE)
- Generate a compact `SKILL.md` containing stable operating guidance: object/itemtype terminology, schema discovery, references, read strategies, safe CRUD, multi-object workflow conventions and error handling.
- Build it from maintained core templates/metadata, using the October 2025 Master Knowledge Export and v2/v3 CustomGPT instructions as **source material**, not directly as final skill content.
- Read live instance schemas, objects and relationships through MCP tools. Do not embed per-instance object dumps or automatically regrow a skill for each new object/schema.
- Regenerate/release skill when behavior or material workflow instructions change, **not** whenever object data changes. Tool schemas and descriptions remain independently discoverable through `tools/list`.
- Skill is helpful guidance, **not required by MCP** and **not an access-control mechanism**. Whether a particular AI host supports attaching or automatically updating skills is host-specific; do not assume automatic runtime sync.

### C. ChatGPT integration (registered app/plugin)
- Register/configure an integration that points at JOE's remote MCP server and authenticates users. Use the platform's supported MCP/app/plugin workflow at implementation time.
- Enable account linking using OAuth authorization flows compatible with ChatGPT's then-current requirements, preferably an authorization server supporting authorization-code + PKCE, bearer tokens, appropriate issuer/audience/scope validation, token expiry and revocation.
- Authenticate as the **individual JOE user**, not a permanent shared administrator key. Enforce scope and object/field permissions in JOE Core regardless of assistant behavior.
- Where the platform supports importing skills, attach the generated skill. Re-import/release changes when needed; do not assume the plugin automatically reads `SKILL.md` from JOE.
- Tools can change independently of a skill, but tool-discovery refresh, approval and publication behavior are client/platform dependent. Validate the live product behavior instead of depending on presumed fixed scanning schedules.
- Additional MCP-compatible clients should consume the same JOE server without custom JOE-specific protocol wrappers.

## 4. Security and correctness decisions
1. **Fail closed.** A missing or misconfigured authentication middleware must not expose protected data or write tools. Anonymous health or metadata endpoints are explicitly limited and separately reviewed.
2. **Per-user server authorization.** Check identity, instance/tenant, role, tool and object/field permissions for each call. A toolset such as `read-only` is configuration metadata, not permission enforcement.
3. **Secrets and data minimization.** No passwords or long-lived tokens in generated skills, MCP responses or logs. Sanitize errors and exclude sensitive fields based on authorization, not a broad assumption that all schemas are public.
4. **Validation.** Check tool input structure, schema field types, required fields, enum values, reference existence and correct target `itemtype`; treat IDs as opaque rather than insisting on UUID format.
5. **Mutation semantics.** JOE sets `joeUpdated` automatically; agents must not forge it. Existing clients may read/merge/save full objects, but this is not concurrency safe unless the server checks a revision/version or performs atomic checked updates. Design optimistic concurrency / field patching in a dedicated subsequent story. Never silently overwrite another user's changes.
6. **Audit and events.** Keep writes through `JOE.Storage.save` or the canonical equivalent, preserving history and events. Verify write + history consistency and recoverability; don't assume atomicity where there is none.
7. **Approval.** AI-side confirmation for consequential changes is useful, but not sufficient security. Protect delete, mass writes and higher-risk operations with server-side rules and appropriate confirmation/approval semantics.
8. **Deletion.** Evaluate soft-delete behavior and define explicit, permissioned delete/archive semantics; do not equate `saveObject` with complete CRUD until tested.
9. **Resource bounds.** Pagination, payload limits, timeouts, rate limits, bulk-operation limits and recursion limits must protect the instance.
10. **OAuth is a separate workstream.** Story 1.1 must preserve authentication boundaries and work in a restricted local test harness; it must not ship as a production-open endpoint.

## 5. Protocol, tool and skill maintenance

| JOE change | Required action |
| --- | --- |
| New JOE object or data edit | No MCP protocol or skill release; current queries retrieve current data |
| New instance schema or schema field | Update JOE schema; tools discover current schema dynamically |
| New MCP tool or changed input shape | Register/validate tool in MCP registry, test, deploy; refresh/review client tool discovery as supported |
| Tool implementation-only change | Deploy JOE and run regression checks |
| Change to agent operating workflow | Regenerate/version/release `SKILL.md` and update the host's installed skill where applicable |
| Changed auth/client metadata | Update JOE and client/plugin registration as required |

Keep the core skill stable; publish detailed tool-specific guidance through tool descriptions and schema metadata. Do not regenerate a gigantic tool inventory inside the skill.

## 6. Initiative backlog

### Epic 1 — Standardize JOE MCP Core
- **1.1** Implement standard MCP Streamable HTTP transport and tool discovery (first story, see `stories/1.1-standard-mcp-transport.md`).
- **1.2** Adapt and validate existing JOE tool definitions/handlers as standard MCP tools, with appropriate tool semantics and errors.
- **1.3** Integrate OAuth account linking and per-user, per-tool and per-object server-side authorization.
- **1.4** Harden mutation validation, optimistic concurrency, references, soft deletion and history integrity.
- **1.5** Implement automated protocol, authorization, CRUD and regression tests.

### Epic 2 — Generate JOE Agent Skill
- **2.1** Reconcile platform Master Knowledge and v2/v3 guidance into maintained core operating rules.
- **2.2** Implement deterministic, versioned `SKILL.md` generation.
- **2.3** Document discover → understand → authoritative read → validate references → confirm → write → verify workflows.
- **2.4** Validate end-to-end multi-object workflows and dynamic schema changes without republishing the skill.

### Epic 3 — ChatGPT Integration and Reuse
- **3.1** Register JOE MCP app/plugin and configure account linking.
- **3.2** Import/release generated skill where supported; define update process.
- **3.3** Pilot against Haddens with test records and separate non-production credentials/data.
- **3.4** Validate the same implementation against a P60 JOE instance.
- **3.5** Publish generic onboarding for new instances and a second MCP-compatible AI client.

P60 schema design, data migration, and SharePoint integration belong to a separate P60 initiative.

## 7. Testing and release gates

### Local test progression
1. Run JOE locally with isolated/dev MongoDB and fixtures.
2. Execute unit/contract tests via the selected MCP SDK client; use MCP Inspector against `http://localhost:<port>/mcp`.
3. Assert handshake, tool enumeration, tool arguments, structured errors, malformed input, unknown tools, concurrent sessions and graceful shutdown.
4. Test authorization and audit constraints using restricted users once auth stories are implemented; no production release before this gate.
5. For a ChatGPT remote test, expose a **restricted temporary staging endpoint** through a secure tunnel or staging deployment with test credentials. Never connect an open tunnel to production data. Availability of a private tunnel, account linking, skills and write actions varies by client/account; validate during integration.
6. Deploy to Haddens staging, then production after approval; repeat second-instance test with P60.

### Initiative completion criteria
- Standard MCP client can initialize and discover/call JOE tools.
- At least two JOE instances use the same core build with no instance-specific MCP adapter.
- ChatGPT can authenticate a real scoped JOE user and perform allowed read/write operations where supported.
- Unauthorized tool, schema, object and field access is rejected and audited appropriately.
- Schema changes appear via live discovery without modifying a static skill.
- Changes preserve reference integrity, revision safety, history and existing JOE UI/API behavior.
- Legacy routes/artifacts are removed after dependency review, not simply left forever.

## 8. Legacy retirement policy
The legacy custom `/.well-known/mcp/manifest.json`, OpenAPI/CustomGPT Actions generator, `/mcp-prompt.html`, static onboarding artifacts, and direct tool-name JSON-RPC dispatch should not survive **solely** for backward compatibility. Check actual usage first, document any active consumer, plan a short migration window if needed, then remove. Keep a `hydrate` *tool* only if useful; retire the old UI pages independently. Do not replace the live `/mcp` transport until cutover and compatibility tests pass.

## 9. Decisions requiring implementation verification
- Which current Node runtime and official SDK versions are compatible.
- Whether the existing JOE OAuth plugin can safely act as an authorization server or needs an established identity provider.
- Correct object-level and field-level permission model in current JOE code.
- Whether `Storage.save` has atomic/revision protections and reliable history behavior.
- Actual ChatGPT app/plugin availability, account linking and imported-skill publication/update behavior for the intended account/workspace.
- Active consumers of custom JSON-RPC or CustomGPT Actions before removal.

## 10. Source inputs
- `JOE_Master_Knowledge_Export.md` — architectural baseline and historical operations, last updated October 2025.
- `joe_agent_custom_gpt_instructions_v_2.md` — schema summaries, reads, safe edits.
- `joe_agent_custom_gpt_instructions_v_3.md` — `understandObject`, tag/status queries, bulk save and the server-managed `joeUpdated` policy.
- Prior code inspection of `server/modules/MCP.js`, `Schemas.js`, `Storage.js` and `Mongo.js`; all findings must be confirmed in the current checkout.

**Reconciled differences:** v2/Master Knowledge advise agents to set `joeUpdated`, whereas v3 advises server ownership; choose server ownership. V3 says reference arrays must contain UUIDs; choose schema-valid, opaque IDs. V3 recommends `hydrate` every session; make it selective to avoid heavy context cost. Keep server-enforced rules authoritative over prose instructions.