AI SDK 5.0 MCP Enabled Next.js 16 Vercel Sandbox

# ๐Ÿš€ Programmatic Tool Calling with AI SDK ### A Universal LLM Optimization POC for Any Model **Programmatic Tool Calling** is a novel approach to dramatically reduce LLM inference costs and latency by replacing traditional multi-round-trip tool calling with code generation and sandboxed execution. > ๐Ÿ’ก Inspired by [Anthropic's beta capabilities](https://www.anthropic.com/engineering/advanced-tool-use) announced November 2025. This project extends that paradigm to work with **any LLM** through the Vercel AI SDK, including 100+ models via the AI Gateway. --- ## ๐ŸŽฏ The Problem Traditional LLM tool calling is **inherently inefficient**, especially with MCP: ``` User: "Get data for users 1-5 and find the highest scorer" Traditional Approach (N round-trips): โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Round 1: LLM โ†’ getUser(1) โ†’ result โ†’ LLM (context grows) โ”‚ โ”‚ Round 2: LLM โ†’ getUser(2) โ†’ result โ†’ LLM (context grows) โ”‚ โ”‚ Round 3: LLM โ†’ getUser(3) โ†’ result โ†’ LLM (context grows) โ”‚ โ”‚ Round 4: LLM โ†’ getUser(4) โ†’ result โ†’ LLM (context grows) โ”‚ โ”‚ Round 5: LLM โ†’ getUser(5) โ†’ result โ†’ LLM (context grows) โ”‚ โ”‚ Round 6: LLM โ†’ final answer โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โฌ‡๏ธ 6 LLM calls ร— full context each Accumulated results pollute context High latency, high token cost ``` ## โœจ The Solution PTC transforms tool orchestration into a single code generation + execution: ``` Programmatic Approach (1 round-trip): โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Round 1: LLM generates JavaScript: โ”‚ โ”‚ const users = await Promise.all([ โ”‚ โ”‚ getUser({ id: '1' }), getUser({ id: '2' }), โ”‚ โ”‚ getUser({ id: '3' }), getUser({ id: '4' }), โ”‚ โ”‚ getUser({ id: '5' }) โ”‚ โ”‚ ]); โ”‚ โ”‚ return users.sort((a,b) => b.score - a.score)[0]; โ”‚ โ”‚ โ”‚ โ”‚ โ†’ Execute in Sandbox โ†’ Return final result only โ”‚ โ”‚ โ”‚ โ”‚ Round 2: LLM receives final answer, responds to user โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โฌ‡๏ธ 2 LLM calls total Intermediate results never enter context Parallel execution, massive savings ``` --- ## ๐Ÿ“Š Proven Efficiency Gains | Metric | Traditional | PTC | Improvement | |--------|-------------|-----|-------------| | **LLM Round-trips** | N (per tool) | 2 (fixed) | 90% reduction | | **Context Growth** | Exponential | Constant | 85% efficiency | | **Token Usage** | ~70,000 (10 tools) | ~14,000 | 80% savings | | **Latency** | Sequential | Parallel | 3-5x faster | | **MCP Tool Calls** | N round-trips | 1 code_execution | 60-80% savings | --- ## ๐Ÿ—๏ธ Architecture ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ User Request โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Vercel AI SDK 5.0 + Programmatic Tool Wrapper โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ withProgrammaticCalling(tools) โ”‚ โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ Wraps local tools (Zod schemas) โ”‚ โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ Wraps MCP tools (JSON Schema) โ”‚ โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ Injects code_execution meta-tool โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ LLM (Any Provider via AI Gateway) โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ Generates JavaScript code orchestrating N tool calls โ”‚ โ”‚ โ”‚ โ”‚ Uses defensive helpers (toArray, safeGet, isSuccess...) โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Vercel Sandbox (Isolated Cloud Execution) โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ Local Tools โ”‚ โ”‚ MCP Bridge (File-based IPC) โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ getUser() โ”‚ โ”‚ mcp_firecrawl_scrape() โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ calculate() โ”‚ โ”‚ mcp_github_search() โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Main Process (MCP Tool Bridge Monitor) โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ - Routes sandbox MCP requests to real MCP servers โ”‚ โ”‚ โ”‚ โ”‚ - Supports HTTP, SSE, and Stdio transports โ”‚ โ”‚ โ”‚ โ”‚ - Normalizes responses for predictable code access โ”‚ โ”‚ โ”‚ โ”‚ - Parallel batch execution for efficiency โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Final Result Only โ†’ Back to LLM โ†’ User Response โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ``` --- ## ๐ŸŒŸ Key Features ### ๐Ÿ”ง Universal Model Support - **Direct Providers**: Anthropic Claude, OpenAI GPT - **Vercel AI Gateway**: 100+ models (Gemini, Mistral, Groq, DeepSeek, Meta, etc.) - Works with any model that supports tool calling ### ๐Ÿงช Vercel Sandbox Execution - Isolated cloud environment for LLM-generated code - Node.js 22 runtime with full async/await support - Automatic syntax validation before execution - Singleton pattern for cost optimization ### ๐Ÿ”Œ MCP Protocol Integration - First-class support for [Model Context Protocol](https://modelcontextprotocol.io) - HTTP, SSE, and Stdio transport support - Novel **MCP Bridge** architecture for sandboxโ†”MCP communication - Parameter normalization and response transformation ### ๐Ÿ“ˆ Real-Time Efficiency Metrics - Token savings breakdown (intermediate, context, overhead, decisions) - Execution time tracking - Visual metrics display in UI - Per-execution cost analysis ### ๐Ÿ›ก๏ธ Defensive Runtime Helpers Built-in utilities for handling unpredictable MCP responses: ```javascript toArray(value) // Safe array conversion safeGet(obj, 'path') // Safe nested property access safeMap(value, fn) // Safe iteration isSuccess(response) // Check MCP response success extractText(response) // Extract string output getCommandOutput(resp) // Parse command results ``` --- ## ๐Ÿš€ Getting Started ### Prerequisites - Node.js 18+ - Vercel account (for Sandbox) - At least one AI provider API key ### Installation #### Option 1: Use the Published Package (Recommended) Install the published npm package: ```bash npm install @task-orchestrator/programmatic-tools ``` **Peer Dependencies** (required): ```bash npm install ai@^5.0.0 @vercel/sandbox@^1.0.0 zod@^3.0.0 ms@^2.1.0 ``` **Optional Dependencies** (for MCP support): ```bash npm install @ai-sdk/mcp@^0.0.11 ``` #### Option 2: Clone and Develop ```bash # Clone the repository git clone https://github.com/your-repo/vercel-ptc-next.git cd vercel-ptc-next # Install dependencies npm install # Set up environment variables cp .env.example .env ``` ### Environment Configuration ```env # Required: At least one AI provider ANTHROPIC_API_KEY=sk-ant-... # OR OPENAI_API_KEY=sk-... # Optional: Vercel AI Gateway (100+ models) AI_GATEWAY_API_KEY=your_gateway_api_key # Vercel Sandbox (run `vercel link` or set token) VERCEL_TOKEN=your_vercel_token ``` ### Run Development Server ```bash npm run dev # Open http://localhost:3000 ``` --- ## ๐Ÿ“– Usage ### Using the Package in Your Project #### Basic Setup ```typescript import { streamText } from 'ai'; import { withProgrammaticCalling } from '@task-orchestrator/programmatic-tools'; import { tool } from 'ai'; import { z } from 'zod'; // Define your tools const myTools = { getUser: tool({ description: 'Get user by ID', inputSchema: z.object({ id: z.string() }), execute: async ({ id }) => ({ id, name: `User ${id}`, score: Math.random() * 100 }), }), calculateAverage: tool({ description: 'Calculate average of numbers', inputSchema: z.object({ numbers: z.array(z.number()) }), execute: async ({ numbers }) => ({ average: numbers.reduce((a, b) => a + b, 0) / numbers.length }), }), }; // Wrap tools for programmatic calling const { tools } = withProgrammaticCalling(myTools); // Use with streamText or generateText const result = await streamText({ model: yourModel, tools, messages: [{ role: 'user', content: 'Get users 1, 2, 3 and calculate their average score' }], }); ``` #### With MCP Integration ```typescript import { withProgrammaticCalling } from '@task-orchestrator/programmatic-tools'; import { createMCPManager } from '@task-orchestrator/programmatic-tools/mcp'; // Initialize MCP servers const mcpManager = createMCPManager({ servers: [ { name: 'firecrawl', type: 'http', url: 'https://mcp.firecrawl.dev/your-key/v2/mcp', }, ], }); await mcpManager.initialize(); const mcpTools = mcpManager.getTools(); // Combine with your local tools const allTools = { ...myTools, ...mcpTools }; // Wrap for programmatic calling const { tools } = withProgrammaticCalling(allTools); ``` #### With Context Management (Token Optimization) ```typescript import { ContextManager, withContextManagement } from '@task-orchestrator/programmatic-tools'; const contextManager = new ContextManager(); const result = await streamText({ model, tools, messages, ...withContextManagement({ contextManager, onStepFinish: (step) => { // Your custom step handling }, }), }); // Get token savings const tokensSaved = contextManager.getTokensSaved(); console.log(`Saved ${tokensSaved.totalSaved} tokens`); ``` ### Using the Demo Application If you've cloned the repository, you can run the full demo: #### Basic Chat 1. Select your model from the dropdown (โŒ˜K to open) 2. Type a prompt that requires multiple operations 3. Watch as PTC generates code and executes efficiently #### Example Prompts ``` "Get 5 users and calculate their average score" โ†’ Generates Promise.all() with 5 getUser calls + calculation "Scrape 3 URLs and summarize their content" โ†’ Parallel mcp_firecrawl_scrape calls + aggregation "Find top products on ProductHunt today" โ†’ MCP scraping with filtering and formatting ``` #### Debug Panel Click "Debug" to view: - Generated code - Individual tool call results - Token savings breakdown - Execution timeline --- ## ๐Ÿ”Œ MCP Server Configuration ### Via Config File (Recommended) Edit `lib/mcp/mcp-config.ts`: ```typescript export const mcpServers: MCPServerConfig[] = [ // HTTP transport { name: "Firecrawl MCP", type: "http", url: "https://mcp.firecrawl.dev/your-key/v2/mcp" }, // Stdio transport (local process) { name: "GitHub MCP", type: "stdio", command: "npx", args: ["-y", "@modelcontextprotocol/server-github"] }, // SSE transport { name: "Streaming MCP", type: "sse", url: "https://example.com/sse" } ]; export const enableMCP: boolean = true; ``` ### MCP Bridge: How It Works The MCP Bridge enables sandbox code to call external MCP tools: ``` โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Vercel Sandbox โ”‚ โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ โ”‚ โ”‚ // LLM-generated code โ”‚ โ”‚ โ”‚ โ”‚ const results = await Promise.all([ โ”‚ โ”‚ โ”‚ โ”‚ mcp_firecrawl_scrape({ url: '...' }), โ”‚ โ”‚ โ”‚ โ”‚ mcp_firecrawl_scrape({ url: '...' }) โ”‚ โ”‚ โ”‚ โ”‚ ]); โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ // Writes to /tmp/mcp_call_*.json โ”‚ โ”‚ โ”‚ โ”‚ // Polls /tmp/mcp_result_*.json โ”‚ โ”‚ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Main Process (Bridge Monitor) โ”‚ โ”‚ - Watches for MCP request files โ”‚ โ”‚ - Routes to real MCP client (HTTP/SSE/Stdio) โ”‚ โ”‚ - Normalizes responses โ”‚ โ”‚ - Writes results back to sandbox filesystem โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ ``` --- ## ๐Ÿ“ Project Structure ``` vercel-ptc-next/ โ”œโ”€โ”€ app/ โ”‚ โ”œโ”€โ”€ api/ โ”‚ โ”‚ โ”œโ”€โ”€ chat/route.ts # Main chat endpoint with PTC โ”‚ โ”‚ โ”œโ”€โ”€ mcp/route.ts # MCP server management โ”‚ โ”‚ โ””โ”€โ”€ models/route.ts # Gateway model discovery โ”‚ โ”œโ”€โ”€ layout.tsx โ”‚ โ”œโ”€โ”€ page.tsx โ”‚ โ””โ”€โ”€ globals.css โ”œโ”€โ”€ components/ โ”‚ โ”œโ”€โ”€ ChatInterface.tsx # Main UI with AI Elements โ”‚ โ”œโ”€โ”€ DebugPanel.tsx # Tool call inspection โ”‚ โ”œโ”€โ”€ EfficiencyMetrics.tsx # Token savings display โ”‚ โ”œโ”€โ”€ MCPServerManager.tsx # MCP configuration UI โ”‚ โ””โ”€โ”€ ai-elements/ # Modular AI UI components โ”‚ โ”œโ”€โ”€ conversation.tsx โ”‚ โ”œโ”€โ”€ message.tsx โ”‚ โ”œโ”€โ”€ tool.tsx โ”‚ โ”œโ”€โ”€ code-block.tsx โ”‚ โ”œโ”€โ”€ chain-of-thought.tsx โ”‚ โ””โ”€โ”€ ... โ”œโ”€โ”€ lib/ โ”‚ โ”œโ”€โ”€ tool-wrapper.ts # ๐Ÿ”‘ Core PTC implementation โ”‚ โ”œโ”€โ”€ sandbox.ts # Vercel Sandbox orchestration โ”‚ โ”œโ”€โ”€ mcp-bridge.ts # MCP โ†” Sandbox communication โ”‚ โ”œโ”€โ”€ mcp/ โ”‚ โ”‚ โ”œโ”€โ”€ client.ts # MCP client implementation โ”‚ โ”‚ โ”œโ”€โ”€ adapter.ts # MCP โ†’ AI SDK conversion โ”‚ โ”‚ โ”œโ”€โ”€ manager.ts # Multi-server management โ”‚ โ”‚ โ””โ”€โ”€ mcp-config.ts # Server configuration โ”‚ โ”œโ”€โ”€ providers.ts # AI provider factory โ”‚ โ”œโ”€โ”€ tools.ts # Example tool definitions โ”‚ โ””โ”€โ”€ context-manager.ts # Token optimization โ””โ”€โ”€ types/ โ””โ”€โ”€ chat.ts # TypeScript definitions ``` --- ## ๐Ÿ’ฐ Cost Analysis ### Vercel Sandbox Pricing | Metric | Rate | Free Tier (Hobby) | |--------|------|-------------------| | Active CPU Time | $0.128/hour | 5 hours/month | | Provisioned Memory | $0.0106/GB-hour | 420 GB-hours | | Network Bandwidth | $0.15/GB | 20 GB | | Sandbox Creations | $0.60/million | 5,000 | ### Cost Per Execution (2 vCPU, 4GB RAM) | Scenario | Duration | Est. Cost | |----------|----------|-----------| | Quick (3-5 tools) | 10 sec | ~$0.0004 | | Medium (5-10 tools) | 30 sec | ~$0.001 | | Heavy (MCP-heavy) | 2 min | ~$0.003 | ### ROI Analysis | Metric | Traditional (10 tools) | PTC | |--------|------------------------|-----| | LLM Round-trips | 10 | 2 | | Context tokens | ~70,000 | ~14,000 | | LLM cost (GPT-4) | $0.70-$2.10 | $0.14-$0.42 | | Sandbox cost | $0 | ~$0.002 | | **Net Savings** | - | **$0.50-$1.70** | **Result**: Sandbox overhead of ~$0.002 saves $0.50-$1.70 in LLM costs per complex workflow. --- ## ๐Ÿ”ฎ How Token Savings Are Calculated PTC tracks four categories of savings: ```typescript { // 1. Intermediate Results (never sent to LLM) intermediateResultTokens: 12500, // 2. Context Re-sends (base context ร— N-1 calls avoided) roundTripContextTokens: 35000, // 3. Tool Call Overhead (JSON structure per call) toolCallOverheadTokens: 400, // 4. LLM Decision Outputs (reasoning per step avoided) llmDecisionTokens: 720, // Total totalSaved: 48620 } ``` --- ## ๐Ÿ› ๏ธ Extending PTC ### Adding Local Tools When using the package: ```typescript import { tool } from 'ai'; import { z } from 'zod'; const myTools = { myCustomTool: tool({ description: 'Description for LLM', inputSchema: z.object({ param: z.string().describe('Parameter description'), }), execute: async ({ param }) => { // Your implementation return { result: '...' }; }, }), }; const { tools } = withProgrammaticCalling(myTools); ``` When developing locally (in this repo): ```typescript // lib/tools.ts export const tools = { myCustomTool: tool({ description: 'Description for LLM', inputSchema: z.object({ param: z.string().describe('Parameter description'), }), execute: async ({ param }) => { // Your implementation return { result: '...' }; }, }), }; ``` ### Adding MCP Servers When using the package: ```typescript import { createMCPManager } from '@task-orchestrator/programmatic-tools/mcp'; const mcpManager = createMCPManager({ servers: [ { name: "Your MCP Server", type: "http", url: "https://your-mcp-server.com/mcp" }, ], }); ``` When developing locally (in this repo): ```typescript // lib/mcp/mcp-config.ts export const mcpServers: MCPServerConfig[] = [ { name: "Your MCP Server", type: "http", url: "https://your-mcp-server.com/mcp" }, ]; ``` --- ## ๐Ÿงช Development ```bash # Run development server npm run dev # Type checking npm run build # Linting npm run lint ``` --- ## ๐Ÿ“ฆ Package Information The core functionality is available as an npm package: **Package:** [`@task-orchestrator/programmatic-tools`](https://www.npmjs.com/package/@task-orchestrator/programmatic-tools) **Installation:** ```bash npm install @task-orchestrator/programmatic-tools ``` **Documentation:** See the [package README](./package/README.md) for detailed API documentation. **Features:** - โœ… Programmatic tool calling with code generation - โœ… MCP (Model Context Protocol) integration - โœ… Context management for token optimization - โœ… Efficiency metrics tracking - โœ… Defensive helper functions for robust execution ## ๐Ÿ“š Resources - [Package Documentation](./package/README.md) - Detailed API reference - [npm Package](https://www.npmjs.com/package/@task-orchestrator/programmatic-tools) - Install and use in your projects - [Vercel AI SDK Documentation](https://sdk.vercel.ai/docs) - [Model Context Protocol Specification](https://modelcontextprotocol.io/specification/2025-11-25) - [Vercel Sandbox Documentation](https://vercel.com/docs/vercel-sandbox) - [Vercel Sandbox Pricing](https://vercel.com/docs/vercel-sandbox/pricing) - [MCP Server Examples](https://github.com/modelcontextprotocol/servers) --- ## ๐Ÿค Contributing Contributions are welcome! This is a novel pattern with room for: - Additional MCP server integrations - Performance optimizations - New defensive helper functions - Provider-specific optimizations - UI/UX improvements --- ## ๐Ÿ“„ License MIT ---

Built with โค๏ธ using Vercel AI SDK, Vercel Sandbox, and MCP
First-of-its-kind LLM optimization for the modern AI stack