# ThinkHive SDK v4.2.2

The official JavaScript/TypeScript SDK for [ThinkHive](https://thinkhive.ai) - AI Agent Observability Platform.

## Features

- **OpenTelemetry-Based Tracing**: Built on OTLP for seamless integration with existing observability tools
- **Run-Centric Architecture**: Atomic unit of work tracking with claims, calibration, and linking
- **Facts vs Inferences**: Claims API for separating verified facts from inferences
- **Deterministic Ticket Linking**: 7 methods for linking runs to support tickets (Zendesk, Intercom, Salesforce, Freshdesk)
- **Calibrated Predictions**: Brier scores and ECE for prediction accuracy
- **Configurable ROI Engine**: Business impact calculation with customizable cost, deflection, and attribution models
- **Auto-Instrumentation**: Works with LangChain, OpenAI, Anthropic, and more
- **Multi-Format Support**: Normalizes traces from 25+ observability platforms

## Installation

```bash
npm install @thinkhive/sdk
```

## Quick Start

### Initialization

```typescript
import { init, runs, shutdown } from '@thinkhive/sdk';

init({
  apiKey: 'thk_your_api_key',
  serviceName: 'my-ai-agent',
  autoInstrument: true,
  frameworks: ['langchain', 'openai'],
});

// Create a run (atomic unit of work)
const run = await runs.create({
  agentId: 'weather-agent',
  conversationMessages: [
    { role: 'user', content: 'What is the weather in San Francisco?' },
    { role: 'assistant', content: 'The weather in San Francisco is currently 65F and sunny.' }
  ],
  outcome: 'success',
});

console.log(`Run ID: ${run.id}`);
await shutdown();
```

## API Reference

### Core APIs (V3)

| API | Import | Description |
|-----|--------|-------------|
| `runs` | `@thinkhive/sdk` | Create and manage runs (atomic work units) |
| `claims` | `@thinkhive/sdk` | Manage facts/inferences for runs |
| `calibration` | `@thinkhive/sdk` | Track prediction accuracy (Brier scores) |
| `linking` | `@thinkhive/sdk` | Connect runs to support tickets (7 methods) |
| `customerContext` | `@thinkhive/sdk` | Time-series customer snapshots |
| `roiAnalytics` | `@thinkhive/sdk` | Configurable ROI calculation engine |
| `agents` | `@thinkhive/sdk` | Agent CRUD management |
| `apiKeys` | `@thinkhive/sdk` | API key management with scoped permissions |

### Assessment APIs

| API | Description |
|-----|-------------|
| `humanReview` | Human-in-the-loop review queues |
| `nondeterminism` | Multi-sample reliability testing |
| `evalHealth` | Metric health monitoring |
| `deterministicGraders` | Rule-based grading |
| `conversationEval` | Multi-turn conversation grading |
| `transcriptPatterns` | Pattern detection in transcripts |

### Operations & Monitoring APIs

| API | Description |
|-----|-------------|
| `evalRuns` | Create, manage, and query evaluation runs |
| `signals` | Behavioral signal configuration, stats, and trends |
| `notifications` | Alert rules and notification management |
| `documents` | Agent RAG document upload and management |
| `shadowTests` | Shadow test creation and execution |
| `sessions` | Trace session grouping and querying |
| `drift` | Model/behavior drift detection |
| `llmCosts` | LLM usage cost tracking and optimization |

### Legacy APIs (V2)

| API | Description |
|-----|-------------|
| `issues` | Clustered failure patterns |
| `analyzer` | User-selected trace analysis with cost estimation |
| `businessMetrics` | Industry-driven metrics with historical tracking |
| `qualityMetrics` | RAG scoring and hallucination detection |

---

## Runs API

```typescript
import { runs, createRunWithContext } from '@thinkhive/sdk';

// Create a run
const run = await runs.create({
  agentId: 'support-agent',
  conversationMessages: [
    { role: 'user', content: 'Help me cancel my order' },
    { role: 'assistant', content: 'I can help you cancel your order...' }
  ],
  outcome: 'resolved',
  durationMs: 1500,
});

// List runs with filters
const result = await runs.list({
  agentId: 'support-agent',
  outcome: 'resolved',
  startedAfter: '2025-01-01',
  limit: 50,
});

// Get a specific run
const existing = await runs.get('run_abc123');

// Get run statistics
const stats = await runs.stats('support-agent');
```

## Claims API (Facts vs Inferences)

```typescript
import { claims, isFact, isInference, getHighConfidenceClaims } from '@thinkhive/sdk';

// List claims for a run
const claimList = await claims.list(runId);

// Filter by type
const facts = claimList.filter(isFact);
const inferences = claimList.filter(isInference);

// Get high confidence claims
const confident = getHighConfidenceClaims(claimList, 0.9);

// Group by category
const byCategory = groupClaimsByCategory(claimList);
```

## Calibration API

```typescript
import { calibration, calculateBrierScore, isWellCalibrated } from '@thinkhive/sdk';

// Get calibration status for a prediction type
const status = await calibration.status('agent-123', 'churn_risk');

// Get all calibration metrics
const metrics = await calibration.allMetrics('agent-123');

// Single prediction
const score = calculateBrierScore(0.8, true); // 0.04

// Batch calculation
const batchScore = calculateBrierScore([
  { predicted: 0.8, actual: 1 },
  { predicted: 0.3, actual: 0 },
]);

if (isWellCalibrated(score)) {
  console.log('Agent predictions are well calibrated');
}
```

## Ticket Linking (V3)

Deterministic linking between runs and support tickets using 7 methods with explicit confidence scores.

```typescript
import {
  linking,
  generateZendeskMarker,
  linkRunToTicket,
  linkRunToZendeskTicket,
  getBestLinkMethod,
  LINK_METHOD_CONFIDENCE,
} from '@thinkhive/sdk';

// SDK explicit linking (confidence: 1.0)
const link = await linking.create({
  runId: 'run_abc123',
  ticketId: 'ticket_xyz',
  method: 'sdk_explicit',
});

// Zendesk marker linking
const zdLink = await linking.create({
  runId: 'run_abc123',
  externalTicketId: '12345',
  platform: 'zendesk',
  method: 'zendesk_marker',
});

// Convenience functions
await linkRunToTicket('run_abc123', 'ticket_xyz');
await linkRunToZendeskTicket('run_abc123', '12345');

// Get links for a run
const links = await linking.getForRun('run_abc123');

// Get links for a ticket
const ticketLinks = await linking.getForTicket('ticket_xyz');

// Verify a link
await linking.verify('link_id', { verified: true, notes: 'Confirmed by agent' });

// Auto-link using all available evidence
const autoLinked = await linking.autoLink('run_abc123');

// Get link statistics
const stats = await linking.stats();
// { totalLinks, byMethod, avgConfidence, verifiedCount, unverifiedCount }

// Generate a Zendesk marker to embed in responses
const marker = generateZendeskMarker('run_abc123');
// Returns: '[THID:run_abc123]'

// Server-side marker generation with format options
const serverMarker = await linking.generateMarker({
  traceId: 'trace_123',
  format: 'html_comment',
});

// Delete a link
await linking.delete('link_id');
```

### Link Methods and Confidence

| Method | Confidence | Description |
|--------|-----------|-------------|
| `sdk_explicit` | 1.0 | Direct SDK call with ticket ID |
| `zendesk_marker` | 1.0 | Embedded THID marker in response |
| `custom_field` | 1.0 | Zendesk custom field |
| `manual` | 1.0 | Human-assigned |
| `middleware_stamp` | 0.98 | Middleware-injected trace ID |
| `session_match` | 0.95 | Session ID correlation |
| `email_time_window` | 0.6 | Email + 15min window |

## ROI Analytics (V3)

Configurable ROI calculation engine with customizable cost models, deflection definitions, and attribution rules.

### Configuration

```typescript
import { roiAnalytics } from '@thinkhive/sdk';

// Get active ROI config (auto-provisions default if none exists)
const config = await roiAnalytics.getConfig();

// Create custom ROI config
const newConfig = await roiAnalytics.createConfig({
  costConfig: {
    humanAgentCostPerHour: 35,
    aiCostPer1kTokens: 0.01,
    escalationCostMultiplier: 2.5,
    infrastructureMonthlyCost: 200,
  },
  deflectionConfig: {
    definition: 'resolved_without_human',
    outcomesThatCount: ['resolved'],
    minConfidence: 0.8,
    excludeEscalated: true,
  },
  resolutionConfig: {
    definition: 'customer_confirmed',
    requiresSatisfaction: false,
    minSatisfactionScore: null,
    noRecontactWindowHours: 24,
  },
  attributionConfig: {
    timeSavedValuePerHour: 50,
    churnPreventionAttribution: 0.2,
    upsellAttribution: 0.1,
    useActualARR: true,
    defaultARRIfUnknown: 10000,
  },
});

// Update config (creates new version, keeps history)
const updated = await roiAnalytics.updateConfig({
  costConfig: { humanAgentCostPerHour: 40 },
});

// List config version history
const versions = await roiAnalytics.configVersions({ limit: 10 });
```

### Calculate ROI

```typescript
// Calculate ROI for a date range
const result = await roiAnalytics.calculateV3({
  startDate: '2025-01-01',
  endDate: '2025-02-01',
  includeBreakdown: true,
  includeConfidenceIntervals: false,
});

console.log(`Net ROI: $${result.summary.netROI}`);
console.log(`ROI %: ${result.summary.roiPercentage}%`);
console.log(`Deflection Rate: ${result.metrics.deflectionRate}`);
console.log(`Total Runs: ${result.metrics.totalRuns}`);

// Filter by agent
const agentROI = await roiAnalytics.calculateV3({
  agentId: 'support-agent',
  startDate: '2025-01-01',
  endDate: '2025-02-01',
});
```

### ROI Trends

```typescript
// Get ROI trend over time
const trend = await roiAnalytics.trendV3({
  granularity: 'day', // 'day' | 'week' | 'month'
  startDate: '2025-01-01',
  endDate: '2025-02-01',
});

for (const bucket of trend.trend) {
  console.log(`${bucket.periodStart}: ${bucket.runCount} runs, $${bucket.netROI} ROI`);
}

if (trend.comparison) {
  console.log(`Improving: ${trend.comparison.isImproving}`);
  console.log(`Change: ${trend.comparison.changePercent}%`);
}
```

### Legacy V1 ROI (still available)

```typescript
// V1 summary endpoint
const summary = await roiAnalytics.summary({
  startDate: '2025-01-01',
  endDate: '2025-01-31',
});

// V1 trends
const trends = await roiAnalytics.trends({
  startDate: '2025-01-01',
  endDate: '2025-01-31',
});

// V1 per-trace ROI calculation
const impact = await roiAnalytics.calculate({
  traceId: 'trace_abc123',
});

// V1 industry configs
const industries = await roiAnalytics.industries();

// V1 correlation analysis
const correlations = await roiAnalytics.correlations();
```

## Manual Tracing

```typescript
import { traceLLM, traceRetrieval, traceTool, traceChain } from '@thinkhive/sdk';

// Trace an LLM call
const response = await traceLLM({
  name: 'generate-response',
  modelName: 'gpt-4',
  provider: 'openai',
  input: { prompt: 'Hello!' }
}, async () => {
  return await openai.chat.completions.create({...});
});

// Trace a retrieval operation
const docs = await traceRetrieval({
  name: 'search-knowledge-base',
  query: 'refund policy',
  topK: 5
}, async () => {
  return await vectorStore.similaritySearch(query, 5);
});

// Trace a tool call
const result = await traceTool({
  name: 'lookup-order',
  toolName: 'order_lookup',
  parameters: { orderId: '12345' }
}, async () => {
  return await lookupOrder('12345');
});

// Trace a chain/workflow
const output = await traceChain({
  name: 'support-workflow',
  input: { query: 'help with refund' }
}, async () => {
  const docs = await retrieveContext(query);
  return await generateResponse(docs);
});
```

## Analyzer API

```typescript
import { analyzer } from '@thinkhive/sdk';

// Estimate cost before running analysis
const estimate = await analyzer.estimateCost({
  traceIds: ['trace-1', 'trace-2'],
  tier: 'standard',
});
console.log(`Estimated cost: $${estimate.estimatedCost}`);

// Analyze specific traces
const analysis = await analyzer.analyze({
  traceIds: ['trace-1', 'trace-2'],
  tier: 'standard',
  includeRootCause: true,
  includeLayers: true,
});

// Analyze traces by time window with smart sampling
const windowAnalysis = await analyzer.analyzeWindow({
  agentId: 'support-agent',
  startDate: new Date('2025-01-01'),
  endDate: new Date('2025-01-31'),
  filters: { outcomes: ['failure'], minSeverity: 'medium' },
  sampling: { strategy: 'smart', samplePercent: 10 },
});

// Get aggregated insights
const summary = await analyzer.summarize({
  agentId: 'support-agent',
  startDate: new Date('2025-01-01'),
  endDate: new Date('2025-01-31'),
});
```

## Issues API

```typescript
import { issues } from '@thinkhive/sdk';

const issueList = await issues.list('support-agent', { status: 'open', limit: 10 });
const issue = await issues.get('issue-123');
const fixes = await issues.getFixes('issue-123');
```

## API Keys

```typescript
import { apiKeys, hasPermission, canAccessAgent } from '@thinkhive/sdk';

const result = await apiKeys.create({
  name: 'CI Pipeline Key',
  permissions: { read: true, write: true, delete: false },
  scopeType: 'agent',
  allowedAgentIds: ['agent-prod-001'],
  environment: 'production',
  expiresAt: new Date(Date.now() + 90 * 24 * 60 * 60 * 1000),
});

if (hasPermission(key, 'write')) { /* can write */ }
if (canAccessAgent(key, 'agent-123')) { /* can access */ }
```

## Business Metrics

```typescript
import { businessMetrics, isMetricReady, getStatusMessage } from '@thinkhive/sdk';

const metric = await businessMetrics.current('agent-123', 'Deflection Rate');
const history = await businessMetrics.history('agent-123', 'Deflection Rate', {
  startDate: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000),
  granularity: 'daily',
});

await businessMetrics.record('agent-123', {
  metricName: 'CSAT/NPS',
  value: 4.5,
  unit: 'score',
  periodStart: '2025-01-01T00:00:00Z',
  periodEnd: '2025-01-07T23:59:59Z',
  source: 'survey_system',
});
```

## Auto-Instrumentation

```typescript
import { init } from '@thinkhive/sdk';

init({
  apiKey: 'thk_your_api_key',
  serviceName: 'my-ai-agent',
  autoInstrument: true,
  frameworks: ['langchain', 'openai', 'anthropic'],
});
// All LangChain, OpenAI, and Anthropic calls are now automatically traced
```

## Sub-package Imports

```typescript
// OpenAI instrumentation
import { instrumentOpenAIClient } from '@thinkhive/sdk/instrumentation/openai';

// LangChain instrumentation
import { ThinkHiveCallbackHandler } from '@thinkhive/sdk/instrumentation/langchain';

// Ticket linking (standalone)
import { linking, generateZendeskMarker } from '@thinkhive/sdk/integrations/ticket-linking';

// Customer context (standalone)
import { customerContext, captureCustomerContext } from '@thinkhive/sdk/integrations/customer-context';
```

## Environment Variables

| Variable | Description |
|----------|-------------|
| `THINKHIVE_API_KEY` | Your ThinkHive API key |
| `THINKHIVE_ENDPOINT` | Custom API endpoint (default: `https://app.thinkhive.ai`) |
| `THINKHIVE_SERVICE_NAME` | Service name for traces (optional) |
| `THINKHIVE_AGENT_ID` | Default agent ID (optional) |

## Eval Runs

```typescript
import { evalRuns } from '@thinkhive/sdk';

// Create an evaluation run
const run = await evalRuns.create('agent-123', { confidenceLevel: 'medium' });

// Get results
const results = await evalRuns.getResults(run.id, { limit: 50 });

// Estimate cost before running
const cost = await evalRuns.estimateCost('agent-123', { confidenceLevel: 'high' });

// List recent runs
const runs = await evalRuns.list({ agentId: 'agent-123', limit: 10 });
```

## Signals

```typescript
import { signals } from '@thinkhive/sdk';

// List all signals
const allSignals = await signals.list();

// Create a custom behavioral signal
await signals.create('Escalation Request', 'negative', {
  type: 'keywords',
  keywords: ['speak to manager', 'escalate'],
});

// Get signal stats and trends
const stats = await signals.getStats({ agentId: 'agent-123' });
const trends = await signals.getTrends({ granularity: 'daily' });
```

## Notifications

```typescript
import { notifications } from '@thinkhive/sdk';

// Create an alert rule
await notifications.createRule({
  agentId: 'agent-123',
  name: 'High failure rate alert',
  eventType: 'failure_spike',
  condition: { threshold: 0.3 },
  channel: 'email',
  target: 'team@company.com',
});

// List notifications
const alerts = await notifications.list('agent-123', true);
// or: await notifications.listNotifications('agent-123', true);
```

## Documents (RAG)

```typescript
import { documents } from '@thinkhive/sdk';

// Upload a document
await documents.upload('agent-123', 'faq.txt', 'text/plain', 1024);

// List agent documents
const docs = await documents.list('agent-123');

// Delete a document
await documents.remove('agent-123', 'doc-456');
```

## Shadow Tests

```typescript
import { shadowTests } from '@thinkhive/sdk';

// Create a shadow test
await shadowTests.create({
  fixId: 'fix-456',
  agentId: 'agent-123',
  testName: 'Refund policy test',
  inputData: { message: 'How do I get a refund?' },
  expectedOutput: 'You can request a refund within 30 days.',
});

// List tests for an agent
const tests = await shadowTests.list('agent-123');
```

## Sessions

```typescript
import { sessions } from '@thinkhive/sdk';

// List conversation sessions
const allSessions = await sessions.list('agent-123', { limit: 20 });

// Get all traces in a session
const traces = await sessions.getTraces('session-789', 'agent-123');
```

## Drift Detection

```typescript
import { drift, hasDrift, getDriftSeverity } from '@thinkhive/sdk';

// Detect drift for an agent
const report = await drift.detect('agent-123');
if (hasDrift(report)) {
  console.log(`Drift severity: ${getDriftSeverity(report)}`);
}

// Detect drift across all agents
const allDrift = await drift.detectAll();
```

## LLM Costs

```typescript
import { llmCosts, formatCost } from '@thinkhive/sdk';

// Get cost summary
const summary = await llmCosts.summary({ period: '30d' });
// or: await llmCosts.getSummary({ period: '30d' });
console.log(`Total cost: ${formatCost(summary.totalCost)}`);

// Get per-agent breakdown
const breakdown = await llmCosts.getBreakdown('agent-123');

// Get optimization savings
const savings = await llmCosts.getSavings();
```

## Error Handling

```typescript
import { ThinkHiveError, RateLimitError } from '@thinkhive/sdk';

try {
  await runs.create({ /* ... */ });
} catch (error) {
  if (error instanceof RateLimitError) {
    console.log(`Retry after ${error.retryAfter}ms`);
  }
}

// Both forms work:
throw new RateLimitError(60000);
throw new RateLimitError('Custom message', 60000);
```

## Upgrading

### v4.2.0 → v4.2.2

Improvements in v4.2.2:
- **`RateLimitError`** now accepts both `new RateLimitError(retryAfterMs)` and `new RateLimitError('message', retryAfterMs)` signatures
- **Claims helpers** (`isFact`, `isInference`, `isComputed`) now accept both `{claimType: 'observed'}` (API format) and `{type: 'fact'}` (shorthand)
- **`groupClaimsByType`** returns both API names (`observed`/`inferred`/`computed`) and shorthand names (`fact`/`inference`)
- **`calculateBrierScore`** now accepts both single prediction `(probability, outcome)` and batch `([{predicted, actual}])` forms
- **`calculateECE`** now accepts pre-computed bucket objects `[{averageConfidence, accuracy, count}]` in addition to raw predictions
- **`isWellCalibrated`** accepts both a raw number and `{brierScore: number}` object
- **`calculatePassAtK`** now supports both `(passRate, k)` and `(n, c, k)` calling conventions
- **`isReliableEvaluation`** accepts both a CriterionAnalysis object and a raw reliability score
- **Rule helpers** (`createRegexRule`, `createContainsRule`, `createLengthRule`, `createJsonSchemaRule`) now return named rule objects with `{type, name, config}` structure
- **`allRulesPassed`** and **`getFailedRules`** accept any `{passed: boolean}` objects, not just `RuleResult[]`
- **`aggregateWorst`** / **`aggregateAverage`** now accept simple `number[]` arrays in addition to `TurnEvaluation[]`
- **Method aliases added** for cross-SDK consistency:
  - `signals.delete()` (alias for `remove()`)
  - `notifications.list()` (alias for `listNotifications()`)
  - `llmCosts.summary()` (alias for `getSummary()`)
  - `qualityMetrics.evaluate()` (alias for `evaluateRag()`)
  - `guardrails.evaluate()` (alias for `scan()`)
  - `customerContext.capture()` (alias for `captureSnapshot()`)

### v4.1.0 → v4.2.0

New in v4.2.0:
- **`evalRuns`** — create, list, and manage evaluation runs programmatically
- **`signals`** — behavioral signal CRUD with stats and trends
- **`notifications`** — alert rule configuration and notification management
- **`documents`** — agent RAG document upload/list/delete
- **`shadowTests`** — shadow test creation and management
- **`sessions`** — trace session grouping and querying
- **`drift`** — model/behavior drift detection with helpers
- **`llmCosts`** — LLM usage cost tracking and optimization savings

### v4.0 → v4.1.0

- `roiAnalytics` now includes V3 methods: `getConfig()`, `createConfig()`, `updateConfig()`, `configVersions()`, `calculateV3()`, `trendV3()`
- `linking` now includes: `autoLink()`, `stats()`, `generateMarker()`
- GET `/roi/config` auto-provisions a default configuration if none exists
- ROI date range limit increased to 92 days

## License

MIT License - see [LICENSE](LICENSE) for details.