# Illuma Agents

**Enterprise-grade TypeScript library for building and orchestrating LLM-powered agents.**

Built on [LangChain](https://js.langchain.com/) and [LangGraph](https://langchain-ai.github.io/langgraphjs/), Illuma Agents provides multi-agent orchestration, real-time streaming, tool integration, prompt caching, extended thinking, and structured output — supporting **12+ LLM providers** out of the box.

---

## Features

- **Multi-Agent Orchestration** — Handoff, sequential, parallel (fan-out/fan-in), conditional, and hybrid agent flows
- **12+ LLM Providers** — OpenAI, Anthropic, AWS Bedrock, Google Gemini, Vertex AI, Azure OpenAI, Mistral, DeepSeek, xAI, OpenRouter, Moonshot
- **Streaming-First** — Real-time token streaming with split-stream buffering and content aggregation
- **Built-in Tools** — Code execution (12+ languages), calculator, web search, browser automation, programmatic tool calling
- **Prompt Caching** — Anthropic and Bedrock cache control for reduced latency and cost
- **Extended Thinking** — Anthropic/Bedrock thinking blocks with proper tool-call sequencing
- **Structured Output** — JSON schema-constrained responses via tool calling, provider-native, or auto mode
- **Dynamic Tool Discovery** — BM25-ranked tool search for large tool registries (MCP servers)
- **Context Management** — Automatic message pruning, token counting, and context window optimization
- **Observability** — Langfuse + OpenTelemetry tracing
- **Dual Module Output** — ESM + CJS with full TypeScript declarations

---

## Installation

```bash
npm install @illuma-ai/agents
```

### Peer Dependencies

The library requires `@langchain/core` as a peer dependency. If not already installed:

```bash
npm install @langchain/core
```

---

## Environment Variables

Set API keys for the providers you plan to use:

```env
# LLM Providers (add the ones you need)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_REGION=us-east-1
GOOGLE_API_KEY=...
AZURE_OPENAI_API_KEY=...
DEEPSEEK_API_KEY=...
XAI_API_KEY=...
MISTRAL_API_KEY=...
OPENROUTER_API_KEY=...

# Code Executor (optional)
CODE_EXECUTOR_BASEURL=http://localhost:8088
CODE_EXECUTOR_API_KEY=your-api-key

# Observability (optional)
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_BASE_URL=https://cloud.langfuse.com
```

---

## Quick Start

### Single Agent

```typescript
import { HumanMessage } from '@langchain/core/messages';
import {
  Run,
  ChatModelStreamHandler,
  createContentAggregator,
  ToolEndHandler,
  ModelEndHandler,
  GraphEvents,
  Providers,
} from '@illuma-ai/agents';
import type * as t from '@illuma-ai/agents';

const { contentParts, aggregateContent } = createContentAggregator();

const run = await Run.create<t.IState>({
  runId: 'my-run-001',
  graphConfig: {
    type: 'standard',
    llmConfig: {
      provider: Providers.ANTHROPIC,
      model: 'claude-sonnet-4-20250514',
      apiKey: process.env.ANTHROPIC_API_KEY,
    },
    instructions: 'You are a helpful AI assistant.',
  },
  returnContent: true,
  customHandlers: {
    [GraphEvents.TOOL_END]: new ToolEndHandler(),
    [GraphEvents.CHAT_MODEL_END]: new ModelEndHandler(),
    [GraphEvents.CHAT_MODEL_STREAM]: new ChatModelStreamHandler(),
    [GraphEvents.ON_RUN_STEP]: {
      handle: (event: string, data: t.RunStep) => aggregateContent({ event, data }),
    },
    [GraphEvents.ON_RUN_STEP_DELTA]: {
      handle: (event: string, data: t.RunStepDeltaEvent) => aggregateContent({ event, data }),
    },
    [GraphEvents.ON_MESSAGE_DELTA]: {
      handle: (event: string, data: t.MessageDeltaEvent) => aggregateContent({ event, data }),
    },
  },
});

const result = await run.processStream(
  { messages: [new HumanMessage('What is the capital of France?')] },
  { version: 'v2', configurable: { user_id: 'user-123', thread_id: 'conv-1' } }
);

console.log('Response:', contentParts);
```

### Multi-Agent with Handoffs

```typescript
import { Run, Providers } from '@illuma-ai/agents';
import type * as t from '@illuma-ai/agents';

const run = await Run.create({
  runId: 'multi-agent-001',
  graphConfig: {
    type: 'multi-agent',
    agents: [
      {
        agentId: 'flight_assistant',
        provider: Providers.ANTHROPIC,
        clientOptions: { modelName: 'claude-haiku-4-5' },
        instructions: 'You are a flight booking assistant.',
      },
      {
        agentId: 'hotel_assistant',
        provider: Providers.ANTHROPIC,
        clientOptions: { modelName: 'claude-haiku-4-5' },
        instructions: 'You are a hotel booking assistant.',
      },
    ],
    edges: [
      {
        from: 'flight_assistant',
        to: 'hotel_assistant',
        description: 'Transfer when user needs hotel help',
      },
      {
        from: 'hotel_assistant',
        to: 'flight_assistant',
        description: 'Transfer when user needs flight help',
      },
    ],
  },
  customHandlers: { /* ...event handlers... */ },
  returnContent: true,
});
```

### Parallel Fan-out / Fan-in

```typescript
const run = await Run.create({
  runId: 'parallel-001',
  graphConfig: {
    type: 'multi-agent',
    agents: [
      { agentId: 'coordinator', provider: Providers.ANTHROPIC, clientOptions: { modelName: 'claude-haiku-4-5' }, instructions: 'Coordinate research tasks.' },
      { agentId: 'analyst_a', provider: Providers.ANTHROPIC, clientOptions: { modelName: 'claude-haiku-4-5' }, instructions: 'Financial analysis.' },
      { agentId: 'analyst_b', provider: Providers.ANTHROPIC, clientOptions: { modelName: 'claude-haiku-4-5' }, instructions: 'Technical analysis.' },
      { agentId: 'summarizer', provider: Providers.ANTHROPIC, clientOptions: { modelName: 'claude-haiku-4-5' }, instructions: 'Synthesize all findings.' },
    ],
    edges: [
      { from: 'coordinator', to: ['analyst_a', 'analyst_b'], edgeType: 'direct' },   // Fan-out (parallel)
      { from: ['analyst_a', 'analyst_b'], to: 'summarizer', edgeType: 'direct' },     // Fan-in
    ],
  },
  customHandlers: { /* ... */ },
});
```

### Using Tools

```typescript
import { createCodeExecutionTool, Calculator } from '@illuma-ai/agents';

const run = await Run.create<t.IState>({
  runId: 'tools-001',
  graphConfig: {
    type: 'standard',
    llmConfig: { provider: Providers.OPENAI, model: 'gpt-4o' },
    instructions: 'You can execute code and do math.',
    tools: [createCodeExecutionTool(), new Calculator()],
  },
  customHandlers: { /* ... */ },
});
```

### Extended Thinking

```typescript
const run = await Run.create<t.IState>({
  runId: 'thinking-001',
  graphConfig: {
    type: 'standard',
    llmConfig: {
      provider: Providers.ANTHROPIC,
      model: 'claude-3-7-sonnet-latest',
      thinking: { type: 'enabled', budget_tokens: 5000 },
    },
    instructions: 'Think through problems carefully.',
  },
  customHandlers: {
    // ...standard handlers...
    [GraphEvents.ON_REASONING_DELTA]: {
      handle: (event: string, data: t.ReasoningDeltaEvent) => {
        // Receive thinking/reasoning tokens as they stream
      },
    },
  },
});
```

### Structured Output

```typescript
const run = await Run.create<t.IState>({
  runId: 'structured-001',
  graphConfig: {
    type: 'standard',
    agents: [{
      agentId: 'analyzer',
      provider: Providers.OPENAI,
      clientOptions: { model: 'gpt-4o' },
      instructions: 'Analyze sentiment.',
      structuredOutput: {
        schema: {
          type: 'object',
          properties: {
            sentiment: { type: 'string', enum: ['positive', 'negative', 'neutral'] },
            confidence: { type: 'number' },
          },
          required: ['sentiment', 'confidence'],
        },
        mode: 'auto',
        strict: true,
      },
    }],
  },
  customHandlers: {
    [GraphEvents.ON_STRUCTURED_OUTPUT]: {
      handle: (_event: string, data: unknown) => console.log('Result:', data),
    },
  },
});
```

---

## Providers

| Provider | Enum | Notes |
|----------|------|-------|
| OpenAI | `Providers.OPENAI` | GPT-4o, o1, o3 |
| Anthropic | `Providers.ANTHROPIC` | Claude 4, Sonnet, Haiku — thinking, caching, web search |
| AWS Bedrock | `Providers.BEDROCK` | Claude via Bedrock — caching, reasoning |
| Google Gemini | `Providers.GOOGLE` | Gemini Pro, Flash |
| Vertex AI | `Providers.VERTEXAI` | Google models via GCP |
| Azure OpenAI | `Providers.AZURE` | OpenAI models via Azure |
| Mistral | `Providers.MISTRALAI` | Large, Medium, Small |
| DeepSeek | `Providers.DEEPSEEK` | Reasoning models |
| xAI | `Providers.XAI` | Grok |
| OpenRouter | `Providers.OPENROUTER` | Multi-model routing |
| Moonshot | `Providers.MOONSHOT` | Moonshot AI |

**Provider config examples:**

```typescript
// OpenAI
{ provider: Providers.OPENAI, clientOptions: { model: 'gpt-4o', apiKey: '...' } }

// Anthropic
{ provider: Providers.ANTHROPIC, clientOptions: { modelName: 'claude-sonnet-4-20250514', apiKey: '...' } }

// AWS Bedrock
{ provider: Providers.BEDROCK, clientOptions: { model: 'us.anthropic.claude-sonnet-4-20250514-v1:0', region: 'us-east-1' } }
```

---

## Multi-Agent Patterns

### Edge Types

| Type | Behavior |
|------|----------|
| **Handoff** (default) | LLM decides when to transfer — auto-generates `transfer_to_<agent>` tools |
| **Direct** | Fixed routing — agents run in sequence or parallel |

### Handoff (Dynamic)

```typescript
{ from: 'triage', to: 'billing', description: 'Transfer for billing questions' }
// → triage agent gets a transfer_to_billing tool it can call
```

### Sequential Pipeline

```typescript
{ from: 'drafter', to: 'reviewer', edgeType: 'direct', prompt: 'Review the draft above.' }
```

### Fan-out / Fan-in (Parallel)

```typescript
{ from: 'coordinator', to: ['analyst_a', 'analyst_b'], edgeType: 'direct' }          // parallel
{ from: ['analyst_a', 'analyst_b'], to: 'summarizer', edgeType: 'direct', prompt: '{results}' }  // join
```

Use `{results}` in prompts to inject collected output from parallel agents.

### Conditional Routing

```typescript
{
  from: 'router',
  to: ['fast_model', 'powerful_model'],
  condition: (state) => state.messages.at(-1)?.content.length > 500 ? 'powerful_model' : 'fast_model',
}
```

### Hybrid

Agents with both handoff and direct edges use **exclusive routing**: if a handoff fires, only the handoff destination runs; otherwise direct edges execute.

---

## Built-in Tools

| Tool | Import | Description |
|------|--------|-------------|
| **Code Executor** | `createCodeExecutionTool()` | Sandboxed execution in 12+ languages (Python, JS, TS, C, C++, Java, PHP, Rust, Go, D, Fortran, R) |
| **Calculator** | `new Calculator()` | Math expressions via [mathjs](https://mathjs.org/) |
| **Browser Tools** | `createBrowserTools()` | 12 browser actions (navigate, click, type, screenshot, etc.) |
| **Tool Search** | `createToolSearchTool()` | BM25-ranked discovery for large tool registries |
| **Programmatic Tool Calling** | `createProgrammaticToolCallingTool()` | LLM writes Python to call tools as async functions |

---

## Event System

Register handlers to receive real-time streaming events:

```typescript
const customHandlers = {
  [GraphEvents.CHAT_MODEL_STREAM]: new ChatModelStreamHandler(),   // Token-by-token streaming
  [GraphEvents.CHAT_MODEL_END]:    new ModelEndHandler(usageArray), // Usage metadata
  [GraphEvents.TOOL_END]:          new ToolEndHandler(),            // Tool results
  [GraphEvents.ON_RUN_STEP]:       { handle: (e, data) => ... },   // New run step
  [GraphEvents.ON_RUN_STEP_DELTA]: { handle: (e, data) => ... },   // Step delta (tool args)
  [GraphEvents.ON_MESSAGE_DELTA]:  { handle: (e, data) => ... },   // Text delta
  [GraphEvents.ON_REASONING_DELTA]:{ handle: (e, data) => ... },   // Thinking delta
  [GraphEvents.ON_AGENT_UPDATE]:   { handle: (e, data) => ... },   // Agent switch
  [GraphEvents.ON_STRUCTURED_OUTPUT]: { handle: (e, data) => ... },// Structured JSON
};
```

Use `createContentAggregator()` to automatically collect deltas into a complete response:

```typescript
const { contentParts, aggregateContent } = createContentAggregator();
// Pass aggregateContent into your handlers
// After streaming, contentParts has the full structured response
```

---

## Prompt Caching

Caching is **automatic** for Anthropic and Bedrock providers:

- System messages get cache control markers
- Last 2 conversation messages get cache breakpoints
- Use `dynamicContext` for per-request data (keeps system prompt cacheable):

```typescript
{
  instructions: 'You are a helpful assistant.',           // Cached
  dynamicContext: `Current time: ${new Date().toISOString()}`,  // Not cached
}
```

---

## Structured Output Modes

| Mode | Description |
|------|-------------|
| `'auto'` | Auto-selects best strategy per provider **(default)** |
| `'tool'` | Uses tool calling — universal compatibility |
| `'provider'` | Provider-native JSON mode |
| `'native'` | Constrained decoding — guaranteed schema compliance |

```typescript
structuredOutput: {
  schema: { /* JSON Schema */ },
  mode: 'auto',
  strict: true,
  handleErrors: true,  // Auto-retry on validation failure
  maxRetries: 2,
}
```

---

## Observability

Set these env vars to enable automatic Langfuse tracing:

```env
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_BASE_URL=https://cloud.langfuse.com
```

Each trace captures `userId`, `sessionId`, `messageId`, and full LangChain callback spans.

---

## Title Generation

Generate conversation titles from the first exchange:

```typescript
const { title, language } = await run.generateTitle({
  provider: Providers.ANTHROPIC,
  inputText: userMessage,
  contentParts,
  titleMethod: TitleMethod.COMPLETION,
  clientOptions: { model: 'claude-3-5-haiku-latest' },
});
```

---

## API Exports

```typescript
// Core
export { Run } from '@illuma-ai/agents';
export { ChatModelStreamHandler, createContentAggregator, SplitStreamHandler } from '@illuma-ai/agents';
export { HandlerRegistry, ModelEndHandler, ToolEndHandler } from '@illuma-ai/agents';

// Tools
export { createCodeExecutionTool, Calculator, createBrowserTools } from '@illuma-ai/agents';
export { createToolSearchTool, createProgrammaticToolCallingTool } from '@illuma-ai/agents';

// Graphs
export { StandardGraph, MultiAgentGraph } from '@illuma-ai/agents';

// LLM
export { getChatModelClass, llmProviders } from '@illuma-ai/agents';

// Enums & Constants
export { GraphEvents, Providers, ContentTypes, StepTypes, TitleMethod, Constants } from '@illuma-ai/agents';

// Types
export type { IState, RunConfig, AgentInputs, GraphEdge, StructuredOutputConfig } from '@illuma-ai/agents';
```

---

## License

UNLICENSED — Proprietary software. All rights reserved @TeamIlluma.