# Matrix Hub [](https://hub.docker.com/r/ruslanmv/matrix-hub) [](https://www.python.org/downloads/release/python-3100/) [](https://github.com/agent-matrix/matrix-hub/actions/workflows/ci.yml) [](https://agent-matrix.github.io/matrix-hub/) [](LICENSE) Welcome to Matrix Hub, your all‑in‑one AI agent marketplace and installer! **Matrix Hub** is your production‑ready catalog & installer for AI agents, custom tools, and MCP servers. It ingests well‑structured manifests from remote catalogs (e.g., GitHub), offers lightning‑fast **search** (lexical + semantic hybrid), safely computes and executes **install plans** (pip/uv, Docker, Git, ZIP), and can automatically **register** everything with your MCP Gateway.  ## What is Matrix Hub? Imagine PyPI for intelligent agents and custom tools—Matrix Hub ingests curated manifests from GitHub (or any remote catalog), lets you find the perfect agent in seconds using hybrid text + semantic search, and then safely installs everything into your project (pip, Docker, Git, or ZIP) with a single command. It even auto‑registers new services with your MCP Gateway, writes scaffolding adapters, and produces a lockfile so your builds are rock‑solid. ```mermaid flowchart TD A[Agent Authors
manifests] --> B[GitHub Catalog] B --> C[Matrix Hub
catalog & install] C --> D[CLI & SDK
agent‑creator] C --> E[Agent Generator
planning_agent] D & E --> F[Install Agents/Tools] F --> G[MCP Gateway
runtime registry] G --> H[Your App
invokes agents] ``` Ready to discover, install, and manage enterprise‑grade AI skills? Explore the [full documentation](https://agent-matrix.github.io/matrix-hub/) to get started! --- ## Official Search API Matrix Hub exposes a stable, public endpoint that returns the **Top‑5** best matches for a query across agents, tools, and MCP servers. This contract is additive and safe for production. ### Endpoint ``` GET /catalog/search ``` **Common parameters** * `q` *(string, required)* – user intent, e.g. `summarize pdfs` * `type` *(enum: agent|tool|mcp\_server|any, default any)* – filter by type; `any` searches all. * `limit` *(int, default 5)* – maximum items returned. Public API caps to **5**. * `with_snippets` *(bool, default false)* – include a short snippet (first \~200 chars of summary/description) when available. Advanced parameters (optional): `mode` (`keyword|semantic|hybrid`), `with_rag` (bool), `rerank` (`none|llm`), and CSV filters `capabilities`, `frameworks`, `providers`. ### Response Each item includes ranking scores and convenience links for import/install: ```json { "items": [ { "id": "tool:hello@0.1.0", "type": "tool", "name": "hello", "version": "0.1.0", "summary": "Return greeting", "capabilities": ["hello"], "frameworks": ["example"], "providers": ["self"], "score_lexical": 0.81, "score_semantic": 0.74, "score_quality": 0.90, "score_recency": 0.88, "score_final": 0.82, "fit_reason": null, "manifest_url": "https://…/hello.manifest.json", "install_url": "https://api.example.com/catalog/install?id=tool:hello@0.1.0", "snippet": "Short summary snippet if with_snippets=true" } ], "total": 1 } ``` ### Quick start (curl) ```bash curl -s 'https://api.example.com/catalog/search?q=extract%20pdf%20tables&type=any&limit=5&with_snippets=true' | jq ``` To install a selected item: ```bash curl -s -X POST 'https://api.example.com/catalog/install' \ -H 'Content-Type: application/json' \ -d '{ "id": "tool:pdf_table_extractor@1.1.0", "target": "./" }' ``` > See the full API doc: [Top‑5 Search API](docs/api/search-api.md). **Deployment notes** * Set `PUBLIC_BASE_URL` in your environment (e.g., `https://api.example.com`). * The optional helper route `GET /catalog/manifest/{id}` can redirect to external manifests when `source_url` is set. * No database migrations required. --- ## Quick start (docker compose) Requirements: Docker 24+, docker compose plugin. ```bash # 1) Clone and enter the repo git clone https://github.com/agent-matrix/matrix-hub.git cd matrix-hub # 2) Create .env from template and adjust if needed cp .env.example .env # (Optionally) set MATRIX_REMOTES to your catalog index.json URLs in .env # 3) Build & run docker compose up -d --build # 4) Health check curl -s http://localhost:443/health | jq ``` You should see: ```json { "status": "ok" } ``` The app container waits for Postgres to be healthy, then starts the API. Ingest is scheduled (see INGEST\_INTERVAL\_MIN in .env). ## Environment All configuration is via environment variables; see `.env.example` for a documented template. Key settings (common): | Variable | Purpose | Example | | :----------------------- | :------------------------------------------- | :----------------------------------------------------------------------- | | `DATABASE_URL` | SQLAlchemy URL | `postgresql+psycopg://matrix:matrix@db:5432/matrixhub` | | `HOST` / `PORT` | Bind address/port | `0.0.0.0` / `443` | | `MATRIX_REMOTES` | CSV/JSON list of `index.json` URLs to ingest | `https://raw.githubusercontent.com/agent-matrix/catalog/main/index.json` | | `INGEST_INTERVAL_MIN` | Background ingest interval (minutes) | `15` | | `API_TOKEN` | Bearer token for admin/protected endpoints | `set_me` | | `MCP_GATEWAY_URL` | MCP Gateway base URL | `http://mcpgateway:7200` | | `MCP_GATEWAY_TOKEN` | Bearer for Gateway admin API | `supersecret` | | `SEARCH_LEXICAL_BACKEND` | `pgtrgm` or `none` | `pgtrgm` | | `SEARCH_VECTOR_BACKEND` | `pgvector` / `none` | `none` | | `SEARCH_HYBRID_WEIGHTS` | `sem:0.6,lex:0.4,rec:0.1,q:0.1` | | | `RAG_ENABLED` | `false` | | | `EMBED_MODEL` | Embedder identifier (informational) | `sentence-transformers/all-MiniLM-L6-v2` | **Security:** If you set `API_TOKEN`, pass `Authorization: Bearer ` on protected endpoints (e.g., manual ingest). Without it, admin endpoints are disabled or open depending on route configuration. ## API walk‑through Base URL defaults to `http://localhost:443`. ### GET /health Liveness/readiness. Optional DB probe: ```bash curl -s 'http://localhost:443/health?check_db=true' | jq ``` Returns: ```json { "status": "ok", "db": "ok" } ``` ### GET /catalog/search Hybrid search over the catalog with optional filters. **Query params** * `q` (required) — free text query * `type` — `agent` | `tool` | `mcp_server` * `capabilities` — CSV, e.g. `pdf,summarize` * `frameworks` — CSV, e.g. `langgraph,watsonx_orchestrate` * `providers` — CSV, e.g. `openai,watsonx` * `mode` — `keyword` | `semantic` | `hybrid` (default from settings) * `limit` — default `20` * `with_rag` — `true` to enrich results with a short “fit\_reason” * `rerank` — `none` | `llm` (default from settings) **Example** ```bash curl -s 'http://localhost:443/catalog/search?q=summarize%20pdfs&type=agent&capabilities=pdf,summarize&limit=5&with_rag=true' | jq ``` **Response (truncated)** ```json { "items": [ { "id": "agent:pdf-summarizer@1.4.2", "type": "agent", "name": "PDF Summarizer", "version": "1.4.2", "summary": "Summarizes long PDF documents into concise points.", "capabilities": ["pdf", "summarize"], "frameworks": ["langgraph"], "providers": ["watsonx"], "score_lexical": 0.82, "score_semantic": 0.76, "score_quality": 0.9, "score_recency": 0.88, "score_final": 0.83, "fit_reason": "Matches 'summarize pdfs'; strong README snippet about legal contracts." } ], "total": 1 } ``` ### GET /catalog/entities/{id} Fetch full entity details for a specific catalog ID. **Example** ```bash curl -s 'http://localhost:443/catalog/entities/agent:pdf-summarizer@1.4.2' | jq ``` ### POST /catalog/install Compute and execute an install plan (pip/uv, docker, git, zip), write adapters into your project, and (optionally) register tools/servers with MCP‑Gateway. Emits `matrix.lock.json`. **Body** ```json { "id": "agent:pdf-summarizer@1.4.2", "target": "./apps/pdf-bot" } ``` **Example** ```bash curl -s -X POST 'http://localhost:443/catalog/install' \ -H 'Content-Type: application/json' \ -d '{"id":"agent:pdf-summarizer@1.4.2","target":"./apps/pdf-bot"}' | jq ``` **Response (truncated)** ```json { "plan": { "artifacts": [ {"kind": "pypi", "command": "uv pip install --system --no-cache-dir pdf-summarizer-agent==1.4.2"} ], "adapters": [{"framework":"langgraph","template_key":"langgraph-node"}], "mcp_registration": { "tool": { "name": "pdf_summarize", "integration_type": "REST", "request_type": "POST", "url": "http://..." } } }, "results": [ {"step":"pypi","ok":true,"returncode":0,"elapsed_secs":5.91}, {"step":"adapters.write","ok":true,"extra":{"count":1}}, {"step":"gateway.register","ok":true,"extra":{"tool":{"id":"...","name":"pdf_summarize"}}}, {"step":"lockfile.write","ok":true,"extra":{"path":"apps/pdf-bot/matrix.lock.json"}} ], "files_written": ["apps/pdf-bot/src/flows/pdf_summarizer_node.py","apps/pdf-bot/matrix.lock.json"], "lockfile": { "version":1, "entities":[{ "id":"agent:pdf-summarizer@1.4.2", "...": "..." }] } } ``` ## Manifests & Schemas Catalog authors publish manifests in a separate content repo (e.g., `agent-matrix/catalog`) and generate an `index.json`. Matrix Hub ingests those manifests on a schedule or via an admin trigger. Supported schemas (JSON Schema Draft 2020‑12): * Agent: `schemas/agent.manifest.schema.json` * Tool: `schemas/tool.manifest.schema.json` * MCP Server: `schemas/mcp-server.manifest.schema.json` Minimal `index.json` formats accepted: ```json { "manifests": ["https://.../agents/pdf/1.4.2/agent.manifest.yaml"] } ``` or ```json { "items": [ {"manifest_url": "https://.../tool.manifest.yaml"} ] } ``` or ```json { "entries": [ {"base_url": "https://raw.githubusercontent.com/ORG/REPO/main/", "path": "agents/pdf/1.4.2/agent.manifest.yaml"} ] } ``` Example agent manifest (excerpt): ```yaml schema_version: 1 type: agent id: pdf-summarizer name: PDF Summarizer version: 1.4.2 description: Summarizes long PDF documents into concise points. capabilities: [pdf, summarize] compatibility: frameworks: [langgraph] providers: [watsonx] artifacts: - kind: pypi spec: { package: pdf-summarizer-agent, version: "==1.4.2" } adapters: - framework: langgraph template_key: langgraph-node mcp_registration: tool: name: pdf_summarize integration_type: REST request_type: POST url: https://example.com/invoke input_schema: { type: object, properties: { input: { type: string } }, required: [input] } ``` ## Local development ```bash # 1) Create a virtualenv (optional) python3 -m venv .venv && source .venv/bin/activate # 2) Install in editable mode pip install -U pip pip install -e . # 3) Run with auto-reload make dev # or uvicorn src.app:app --reload --port 443 ``` **Quality & tests** ```bash make fmt # ruff fix + format make lint # ruff check make test # pytest -q ``` **Where things live** ``` src/ app.py # FastAPI app, routers, scheduler routes/ # /health, /catalog/* services/ # ingest, search, install, adapters, gateway client... models.py # SQLAlchemy models schemas.py # Pydantic DTOs utils/ # logging, security, etag schemas/ # JSON Schemas for manifests tests/ # starter tests and smoke checks ``` ## Migrations This project includes Alembic scaffolding. To create and apply migrations: ```bash # Generate a new revision (edit message) make migrate m="add quality indexes" # Apply to head make upgrade ``` In production, run migrations as part of your deploy step (e.g., init container or CI job) before starting new app instances. ## Production notes **Container image:** See `Dockerfile` (multi‑stage). Run with: ```bash docker build -t ghcr.io/agent-matrix/matrix-hub:latest . docker run -p 443:443 --env-file .env ghcr.io/agent-matrix/matrix-hub:latest ``` **Database:** Use PostgreSQL in production. SQLite is supported for quick local trials only. **Scaling:** The API is stateless; scale horizontally behind a reverse proxy/load balancer. Ensure a single writer for migrations. **Observability:** Logs are JSON‑formatted with correlation IDs (see `src/utils/logging.py`). Forward stdout/stderr to your log stack (e.g., Loki/ELK). Add metrics/tracing as needed. **Security:** * Set `API_TOKEN` to protect admin routes. * Place the service behind TLS (e.g., NGINX/Envoy) and enforce client auth if needed. * Configure outbound access policies for `pip`, `docker`, and `git` if your environment restricts egress. **MCP Gateway:** Expose `MCP_GATEWAY_URL` and `MCP_GATEWAY_TOKEN`. The installer will register tools/servers after artifacts are installed. ## Troubleshooting **API returns 500 on `/catalog/search`** * Check DB connectivity (`/health?check_db=true`). * Ensure the entity table is populated (ingest logs). * If running SQLite, semantic/vector search is disabled by default. **No results after ingest** * Verify `MATRIX_REMOTES` points to a valid `index.json`. * Review container logs for validation errors (schema mismatches). **Install fails on `pip`/`docker`/`git`** * See the `results` array in the install response for command stdout/stderr. * Check that the container has network access and the artifact references are valid. **Gateway registration errors** * Confirm `MCP_GATEWAY_URL` is reachable and token is correct. * Some gateways perform discovery automatically; our client handles transport normalization. ## License Apache‑2.0. See [LICENSE](LICENSE). ## Docker Hub The official Docker image is available on Docker Hub: ```bash # Pull the latest image docker pull ruslanmv/matrix-hub:latest # Run with default settings (SQLite, port 8000) docker run -d -p 8000:8000 --name matrix-hub ruslanmv/matrix-hub:latest # Run with PostgreSQL and custom config docker run -d -p 8000:8000 \ -e DATABASE_URL=postgresql+psycopg://matrix:matrix@db:5432/matrixhub \ -e API_TOKEN=your-secret-token \ -e CATALOG_REMOTES='["https://raw.githubusercontent.com/agent-matrix/catalog/main/index.json"]' \ --name matrix-hub \ ruslanmv/matrix-hub:latest # Build locally make container-build make container-run ``` ## Frontend Integration The official web frontend is [MatrixHub](https://github.com/ruslanmv/matrixhub), deployed at [matrixhub.io](https://matrixhub.io). The frontend communicates with this backend via server-side proxy routes: | Frontend Route | Backend Route | Purpose | |:---------------|:-------------|:--------| | `GET /api/search` | `GET /catalog/search` | Hybrid search | | `GET /api/latest` | `GET /catalog` | List by type | | `GET /api/entities/[id]` | `GET /catalog/entities/{id}` | Entity detail | | `GET /api/manifest/[id]` | `GET /catalog/manifest/{id}` | Manifest JSON | | `POST /api/install` | `POST /catalog/install` | Install entity | To connect the frontend to your backend, set `MATRIX_HUB_BASE` in the frontend's `.env.local`: ```bash MATRIX_HUB_BASE=https://api.matrixhub.io ``` ## DNS Configuration (Production) | Record | Name | Value | Purpose | |:-------|:-----|:------|:--------| | A | `api.matrixhub.io` | `129.213.165.60` | Backend API | | A | `matrixhub.io` | `216.198.79.1` | Frontend (Vercel) | | CNAME | `www.matrixhub.io` | `*.vercel-dns-017.com` | Frontend www redirect | ## See also - **[matrixhub](https://github.com/ruslanmv/matrixhub)** — Web frontend (Next.js) - **[agent-generator](https://github.com/ruslanmv/agent-generator)** — AI agent scaffolding tool - **[catalog](https://github.com/agent-matrix/catalog)** — Curated manifest repository **Schemas:** * Agent — `schemas/agent.manifest.schema.json` * Tool — `schemas/tool.manifest.schema.json` * MCP Server — `schemas/mcp-server.manifest.schema.json` **Tests:** `tests/` — smoke tests and examples; run with `make test`.