# Ask: Multi-LLM Interface

A TypeScript/Node module (Bun-compatible) for connecting to **11 LLM providers** with **560+ models** via a single unified interface. Use it as a CLI, code library, Cloudflare Worker, web server, or **client-side SDK**.

---

## Features
- ✅ **11 LLM Providers**: OpenAI, Anthropic, Google, Groq, Mistral, OpenRouter, Cohere, XAI, DeepSeek, AI21, Cloudflare
- ✅ **560+ Models** with unified API
- ✅ **Web Interface** 🆕: Modern chat UI built with Vue 3 and bare-v2 (see `/web` directory)
- ✅ **Client-side SDK**: Browser-compatible JavaScript/TypeScript SDK for web apps
- ✅ **CLI with 45+ aliases** (e.g., `kimi`, `claude`, `gpt4o`, `gemini`)
- ✅ **Beautiful model listing** with emoji icons and capability badges
- ✅ **Streaming & non-streaming** support
- ✅ **Context injection** from files/directories
- ✅ **Conversation mode** with history management
- ✅ **Custom API Key Management**: One key to rule them all - map multiple provider keys to a single Ask API key
- ✅ **Usage tracking & analytics** with token metering
- ✅ **Security features**: Content redaction (BlackoutManager), Input protection (LLM Shield)
- ✅ **Request logging** with latency tracking and statistics
- ✅ **Cloudflare Workers deployment** ready with D1 database support
- ✅ **Library usage** or standalone CLI

## Quick Start

### Web Interface (NEW!)

```bash
# Start the API worker
bun run worker:dev

# Start the web interface
bun run web:dev

# Open http://localhost:3000
```

See [WEB_INTERFACE_SETUP.md](./WEB_INTERFACE_SETUP.md) for details.

## Installation
```sh
bun install @livx.cc/ask
# or clone and bun install
```

## CLI Usage
```sh
bun run bin/localAsk.ts [options] <question>
# or if installed globally:
ask [options] <question>
# or:
bunx @livx.cc/ask [options] <question>
```

### Options
- `-h`         Show help (use `-h` for detailed help)
- `--list-models, -M`  List/filter supported models
- `--model, -m`        Model to use (e.g. gpt-4, gemini-1.5-pro-latest)
- `--maxTokens, -k`    Max tokens (default: 8192)
- `--temperature, -t`  Sampling temperature
- `--stream, -S`       Enable streaming mode
- `--save, -s`         Save Q&A trace
- `--output, -o`       Custom output folder for traces (default: .tmp/desk)
- `--conversation, -C` Conversation mode: load all previous Q&A as message history
- `--latest, -L`       Use only latest Q&A pair as message history
- `--context, -c`      Context file(s)/dir(s), comma-separated
- `--input, -i`        JSON/YAML input file with execution parameters
- `--promptid, -p`     Prompt ID
- `--workspaceid, -w`  Workspace ID
- `--verbose, -v`      Verbose logging

### Examples
```sh
ask --model gpt-4 --context src/ "What does this code do?"
ask -m gemini-1.5-pro-latest "Summarize the project."
ask --list-models gpt-4
ask --stream "Stream the answer"
ask -h  # Show detailed help
ask -s -o "my-traces/" "Save to custom folder"
ask -C "Continue our previous conversation"
ask -L "Elaborate on your last answer"
ask -C -o "conversations/" "Multi-turn chat in custom folder"
ask -i config.json  # Use JSON configuration file
ask -i config.json "Override question from JSON"
```

## JSON/YAML Input Files

You can use JSON or YAML input files to store and reuse execution configurations. This is perfect for reproducible executions and complex setups.

### JSON Configuration Format
```json
{
  "model": "gpt-4",
  "temperature": 0.7,
  "maxTokens": 2048,
  "stream": true,
  "save": true,
  "outputFolder": "custom-traces",
  "conversationMode": true,
  "latestOnly": false,
  "context": ["src/", "README.md"],
  "question": "What does this code do and how is it structured?",
  "verbose": false,
  "promptId": "example-prompt",
  "workspaceId": "example-workspace"
}
```

### YAML Configuration Format
```yaml
model: gpt-4
temperature: 0.7
maxTokens: 2048
stream: true
save: true
outputFolder: custom-traces
conversationMode: true
latestOnly: false
context:
  - src/
  - README.md
question: "What does this code do and how is it structured?"
verbose: false
promptId: example-prompt
workspaceId: example-workspace
```

### Parameter Precedence
When using input files, parameters are resolved in this order (highest to lowest priority):
1. **Command-line arguments** (`--model`, `--maxTokens`, etc.)
2. **Input file** (`-i config.json` or `-i config.yaml`)
3. **Environment variables** (`MODEL`, `MAX_TOKENS`, etc.)
4. **Local YAML prompt settings** (if using local prompt)
5. **Defaults**

### Input File Examples
```sh
# Use all settings from JSON
ask -i my-config.json

# Use all settings from YAML
ask -i my-config.yaml

# Override specific parameters
ask -i my-config.yaml --model gpt-3.5-turbo --verbose

# Override the question from input file
ask -i my-config.json "Different question than in file"

# Custom output folder via CLI
ask -s -o "my-traces/" "Save to custom folder"

# Custom output folder via input file (outputFolder in JSON/YAML)
ask -i config-with-output.yaml

# Override input file output folder
ask -i config.yaml -o "override-folder/"
```

## Local vs Remote Prompt Usage

- **Remote prompt** (default): Uses a workspace and prompt ID (from args, env, or defaults) to fetch the prompt from a remote server.
- **Local prompt**: If you pass a local path (e.g. `./ask.yaml` or `/path/to/ask.yaml`) as `--promptid` or `-p`, the CLI will load the prompt and settings from that YAML file and ignore remote workspace/prompt IDs.

### Example: Local Prompt YAML Structure
```yaml
---
prompt: |-
  context: {{context}}
  respond with capital letters only.
  ----
# respond in Hebrew.
settings:
  maxTokens: 1024
  model: meta-llama/llama-4-scout-17b-16e-instruct
  provider: groq
  temperature: 0.9
```
- `prompt`: The system prompt (supports `{{context}}` template).
- `settings`: Optional. Overrides for model, maxTokens, provider, temperature, etc.

Comand line example for local context and prompt usage:
```sh
ask what is this file about -S -c tsconfig.json -p ./workspaces/6ab03c56d720b34033c2d06ef9ba6cdf/ask.yaml
```

## Conversation Mode

The CLI supports conversation mode to maintain context across multiple interactions by loading previous Q&A pairs as message history.

### Conversation Flags
- **`-C, --conversation`**: Load ALL previous Q&A pairs from trace files as conversation history
- **`-L, --latest`**: Load only the LATEST Q&A pair as conversation history

### How It Works
1. **Save traces**: Use `-s` to save Q&A pairs to trace files
2. **Continue conversation**: Use `-C` or `-L` to load previous conversations
3. **Message history**: Previous Q&A pairs become user/assistant messages in the conversation

### Conversation Examples
```bash
# Start a conversation (save is enabled by default)
ask "My name is Alice and I'm a software engineer"

# Continue with latest context only
ask -L "What was my name again?"

# Continue with full conversation history  
ask -C "Tell me everything I've shared about myself"

# Conversation with custom output folder
ask -C -o "my-conversations/" "Continue our discussion"
```

### Input File Support
Conversation mode can also be configured in JSON/YAML input files:

```json
{
  "conversationMode": true,
  "save": true,
  "question": "Continue our previous discussion"
}
```

```yaml
latestOnly: true
save: true
question: "What did I just tell you?"
```

## Library Usage

### Node.js/Bun Backend

```ts
import { AskModule, AskModuleOptions } from '@livx.cc/ask';
import { FeedoxAIModule } from '@livx.cc/ask/dist/providers/FeedoxAI';

const ai = new FeedoxAIModule();
const ask = new AskModule(ai, { maxTokens: 2048 });
const answer = await ask.ask('What is the meaning of life?');
console.log(answer);
```

### Client-Side SDK (Browser/Web Apps)

**NPM/Yarn/Bun:**
```typescript
import { AskClient } from '@livx.cc/ask/client';

// Initialize client
const client = new AskClient({
  baseUrl: 'https://your-worker.workers.dev/v1',
  askApiKey: 'ask_abc123...',
  defaultProvider: 'openai',
  defaultModel: 'gpt-4'
});

// Non-streaming request
const response = await client.completion({
  messages: [{ role: 'user', content: 'Hello!' }]
});
console.log(response.content);

// Streaming request
const stream = await client.streamCompletion({
  messages: [{ role: 'user', content: 'Tell me a story' }]
});

for await (const chunk of stream) {
  process.stdout.write(chunk);
}
```

**CDN (Browser-only):**
```html
<script type="module">
  // ESM import from CDN
  import { AskClient } from 'https://cdn.jsdelivr.net/npm/@livx.cc/ask@latest/dist/ask-client.esm.min.js';
  
  const client = new AskClient({
    baseUrl: 'https://your-worker.workers.dev/v1',
    askApiKey: 'ask_abc123...'
  });
  
  const response = await client.completion({
    messages: [{ role: 'user', content: 'Hello!' }]
  });
</script>
```

**📚 Documentation:**
- **[CLIENT_SDK.md](docs/CLIENT_SDK.md)** - Complete SDK documentation with React/Vue examples
- **[CDN_USAGE.md](docs/CDN_USAGE.md)** - Detailed CDN usage guide with examples

## Supported Models/Providers

### 11 Providers, 560+ Models

| Provider | Models | Features |
|----------|--------|----------|
| **OpenAI** | GPT-3.5, GPT-4, GPT-4o, GPT-5, o1, o3 | Reasoning, Vision |
| **Anthropic** | Claude 2/3/4, Haiku, Sonnet, Opus | Long context |
| **Google** | Gemini 1.x/2.x/2.5, PaLM, Gemma | Vision, Reasoning |
| **Groq** | Llama 3/4, Mixtral, Gemma, Kimi, DeepSeek distilled | Fast inference |
| **Mistral** | Mistral/Mixtral, Codestral, Pixtral | Coding, Vision |
| **OpenRouter** | 50+ aggregated models | Multi-provider access |
| **Cohere** | Command-R, Command-R-Plus, Aya | Multilingual |
| **XAI** | Grok 2/3/4 | Vision, Image gen |
| **DeepSeek** | DeepSeek V3, DeepSeek R1 | Reasoning |
| **AI21** | Jurassic-2, Jamba | *Partially supported* |
| **Cloudflare** | Llama, TinyLlama via Workers AI | Edge deployment |

### View All Models

```bash
# List all models grouped by provider with emoji icons
ask --list-models

# Filter by provider or name
ask --list-models gpt
ask --list-models gemini
ask -M claude
```

### CLI Model Aliases

45+ shortcuts for common models:
- `kimi` → `groq/moonshotai/kimi-k2-instruct`
- `claude` → `anthropic/claude-3-5-sonnet-latest`
- `gpt4o` → `openai/gpt-4o`
- `gemini` → `google/models/gemini-2.0-flash`
- `mistral` → `mistral/mistral-large-latest`
- `grok` → `xai/grok-beta`
- `deepseek` → `deepseek/deepseek-chat`
- And many more!

```bash
# Use aliases instead of full names
ask -m kimi "Count to 10"
ask -m claude "Explain quantum computing"
ask -m gpt4o "Write a poem"
```

## Environment Variables

Configure API keys in `.env`:

```bash
# Required: At least one provider
OPENAI_API_KEY=sk-...
CLAUDE_API_KEY=sk-ant-...
GROQ_API_KEY=gsk_...

# Optional: Additional providers
GOOGLE_AI_API_KEY=...
MISTRAL_API_KEY=...
OPENROUTER_API_KEY=...
COHERE_API_KEY=...
XAI_API_KEY=...
DEEPSEEK_API_KEY=...
AI21_API_KEY=...

# Cloudflare Workers AI (requires both)
CLOUDFLARE_API_KEY=...
CLOUDFLARE_ACCOUNT_ID=...

# Worker mode (optional)
USE_WORKER=true
WORKER_API_URL=https://your-worker.workers.dev/v1
```

## Cloudflare Workers Deployment

Deploy the worker for edge-based LLM access:

```bash
# Configure secrets
wrangler secret put OPENAI_API_KEY
wrangler secret put CLAUDE_API_KEY
wrangler secret put GROQ_API_KEY
# ... add all providers you want to use

# Deploy
wrangler deploy

# Test
curl https://your-worker.workers.dev/v1/models
```

## Security Features

### Content Redaction (BlackoutManager)
Redact sensitive content during streaming:
```typescript
import { applyBlackoutToStream } from './worker/modules/BlackoutManager';

// Content within <|...|> will be redacted
const stream = await applyBlackoutToStream(originalStream, true);
```

### Input Protection (LLM Shield)
Scan for prompt injection and data exfiltration attempts:
```typescript
import { createLlmShield } from './worker/modules/LlmShield';

const shield = createLlmShield(true);
const result = shield.scanInput(userInput);
if (!result.safe) {
  console.warn('Security issues detected:', result.issues);
}
```

### Request Logging
Track latency, token usage, and provider statistics:
```typescript
import { getRequestLogger } from './worker/modules/RequestLogger';

const logger = getRequestLogger(true);
const tracker = logger.startRequest('openai', 'gpt-4');
// ... execute request ...
logger.logRequest(tracker.end(true, tokensUsed));

// Get statistics
const stats = logger.getStats();
console.log(stats.averageLatency, stats.providerBreakdown);
```

## API Usage

### REST API Endpoints

```bash
# List all models
GET /v1/models

# List models by provider
GET /v1/models/openai
GET /v1/models/anthropic

# Execute completion
POST /v1/completion
{
  "provider": "openai",
  "model": "gpt-4",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ],
  "config": {
    "temperature": 0.7,
    "maxTokens": 2000,
    "stream": true
  }
}
```

### Custom API Keys via Headers

```bash
curl -X POST https://your-worker.workers.dev/v1/completion \
  -H "Content-Type: application/json" \
  -H "x-openai-api-key: sk-..." \
  -H "x-anthropic-api-key: sk-ant-..." \
  -d '{
    "provider": "openai",
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

## Custom API Key Management

Ask now supports custom API key management, allowing users to create a single Ask API key that maps to multiple provider keys. This simplifies key management and enables usage tracking.

### Benefits

- 🔑 **One Key for All**: Manage one Ask API key instead of juggling multiple provider keys
- 📊 **Usage Tracking**: Automatic logging of requests, tokens, latency, and costs
- 🔒 **Secure Storage**: Provider keys are encrypted in the database
- 📈 **Analytics**: View usage statistics by provider, model, and time period
- 🚀 **Easy Integration**: Works with existing endpoints via Authorization header
- 💰 **Future Ready**: Prepared for metered billing and paid tiers

### Quick Start with Sync Script

The easiest way to get started is using the sync script:

```bash
# 1. Setup database (one-time)
./scripts/setup-db.sh dev

# 2. Add your provider keys to .env
cp .env.example .env
# Edit .env with your API keys

# 3. Start worker
npm run worker:dev

# 4. Sync your keys (creates Ask API key)
npm run sync-keys
```

**That's it!** Your Ask API key is saved in `.env.ask` and ready to use.

See [QUICK_START.md](QUICK_START.md) for a complete walkthrough.

### Manual Setup

1. **Create D1 Database**:
```bash
# Run the setup script
./scripts/setup-db.sh dev

# Or manually:
wrangler d1 create ask-api-keys-dev
wrangler d1 execute ask-api-keys-dev --file=src/worker/db/schema.sql
```

2. **Set Encryption Key**:
```bash
wrangler secret put API_KEY_ENCRYPTION_KEY --env=dev
# Enter a strong random key (recommended: 32+ characters)
```

3. **Deploy Worker**:
```bash
wrangler deploy --env=dev
```

### API Key Management Endpoints

#### Create API Key
```bash
POST /v1/apikeys
{
  "userId": "user-123",
  "name": "My App Key",
  "description": "Production API key for my app",
  "providerKeys": {
    "openai": "sk-...",
    "anthropic": "sk-ant-...",
    "groq": "gsk_..."
  },
  "rateLimitRpm": 60,
  "rateLimitTpm": 100000
}

Response:
{
  "success": true,
  "data": {
    "apiKey": "ask_a1b2c3d4e5f6...",
    "id": 1,
    "userId": "user-123",
    "name": "My App Key",
    "providers": ["openai", "anthropic", "groq"],
    "createdAt": 1704567890
  }
}
```

#### List User's API Keys
```bash
GET /v1/apikeys?userId=user-123

Response:
{
  "success": true,
  "data": [
    {
      "id": 1,
      "apiKey": "ask_a1b2c3d4...",
      "name": "My App Key",
      "usageCount": 150,
      "lastUsedAt": 1704567890,
      "isActive": true
    }
  ]
}
```

#### Get API Key Details
```bash
GET /v1/apikeys/{apiKey}

Response:
{
  "success": true,
  "data": {
    "id": 1,
    "apiKey": "ask_a1b2c3d4...",
    "userId": "user-123",
    "name": "My App Key",
    "usageCount": 150,
    "providers": ["openai", "anthropic", "groq"],
    "rateLimitRpm": 60,
    "rateLimitTpm": 100000
  }
}
```

#### Add/Update Provider Key
```bash
POST /v1/apikeys/{apiKey}/providers
{
  "userId": "user-123",
  "provider": "mistral",
  "providerApiKey": "..."
}
```

#### Remove Provider Key
```bash
DELETE /v1/apikeys/{apiKey}/providers/{provider}
{
  "userId": "user-123"
}
```

#### Revoke API Key
```bash
DELETE /v1/apikeys/{apiKey}
{
  "userId": "user-123"
}
```

#### Get Usage Statistics
```bash
GET /v1/apikeys/{apiKey}/usage?startTime=1704000000&endTime=1704999999

Response:
{
  "success": true,
  "data": {
    "apiKey": "ask_a1b2c3...",
    "period": {
      "start": 1704000000,
      "end": 1704999999
    },
    "byProvider": [
      {
        "provider": "openai",
        "request_count": 100,
        "success_count": 98,
        "total_tokens": 50000,
        "avg_latency_ms": 1250
      }
    ]
  }
}
```

### Using Ask API Keys

Once you have an Ask API key, use it in the Authorization header:

```bash
# Using Bearer token format
curl -X POST https://your-worker.workers.dev/v1/completion \
  -H "Authorization: Bearer ask_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Or using custom header
curl -X POST https://your-worker.workers.dev/v1/completion \
  -H "x-ask-api-key: ask_a1b2c3d4e5f6..." \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

When using an Ask API key:
- ✅ Provider keys are automatically resolved from the database
- ✅ Usage is automatically logged for analytics
- ✅ Rate limits are checked (when configured)
- ✅ All provider headers are ignored (security)

### Database Schema

The system uses three main tables:

1. **api_keys**: Stores Ask API keys with metadata
   - User ID, name, description
   - Rate limits (RPM, TPM)
   - Usage counts and timestamps
   - Billing tier and token balance (for future use)

2. **provider_keys**: Maps Ask keys to encrypted provider keys
   - Supports all 11 providers
   - Keys are encrypted at rest
   - Can be added/removed independently

3. **usage_logs**: Tracks every request
   - Provider, model, tokens
   - Success/failure, latency
   - Timestamps for analytics
   - Cost tracking (future)

### Security Considerations

- 🔒 Provider API keys are encrypted in the database using AES encryption
- 🔑 Set a strong `API_KEY_ENCRYPTION_KEY` secret (32+ random characters)
- 🛡️ Ask API keys use cryptographically secure random generation
- 🚫 Never expose Ask API keys in public repositories
- ✅ Keys can be revoked instantly
- 📝 All usage is logged for audit trails

### Future Enhancements

The system is designed to support:
- 💰 **Paid Tiers**: Charge users for usage without exposing provider keys
- 🎯 **Token Metering**: Track and bill based on actual token usage
- 💳 **Balance Management**: Prepaid credits and automatic top-ups
- 📊 **Advanced Analytics**: Cost optimization and usage insights
- 🔄 **Automatic Failover**: Switch providers based on availability
- 🌐 **Multi-region**: Geographic key management

## Tests

Comprehensive test coverage for all components:
- Handler tests (OpenAI, Anthropic, Groq, etc.)
- Streaming tests with chunk reassembly
- Router integration tests
- CLI tests

```bash
# Run all tests
bun test

# Run worker tests
bun test tests/worker

# Run handler tests
bun test tests/worker/handlers
```

## Architecture

```
CLI (bin/localAsk.ts)
  ↓
  ├─> Worker Mode → WorkerClient → Cloudflare Worker
  └─> Direct Mode → LlmManager → Provider Handlers

Cloudflare Worker (src/worker/)
  ↓
  Router → Completion Controller → LlmManager → Provider Handlers
  
Provider Handlers:
  - OpenAiHandler
  - AnthropicHandler  
  - GroqHandler
  - GoogleHandler
  - MistralHandler
  - OpenRouterHandler
  - CohereHandler
  - XAIHandler
  - DeepSeekHandler
  - AI21Handler
  - CloudflareHandler
```

## Development/Contributing

- **Runtime**: Bun + TypeScript
- **Main code**: `src/`, CLI: `bin/localAsk.ts`
- **Tests**: `tests/`
- **Workers**: `src/worker/`
- PRs welcome!

### Project Structure

```
src/
├── ask.ts                    # Main Ask module
├── worker/
│   ├── modules/
│   │   ├── handlers/         # Provider handlers (11 total)
│   │   ├── LlmManager.ts     # Handler orchestration
│   │   ├── BlackoutManager.ts # Content redaction
│   │   ├── LlmShield.ts      # Security scanning
│   │   └── RequestLogger.ts  # Monitoring
│   ├── controllers/
│   │   └── completion.ts     # Worker completion endpoint
│   └── routes/               # Worker routing
├── providers/
│   ├── FeedoxAI.ts          # Legacy provider
│   └── WorkerClient.ts      # Worker client
└── supported-models.json     # Model catalog

bin/
└── localAsk.ts              # CLI entrypoint

tests/
├── worker/
│   ├── handlers/            # Handler tests
│   └── integration/         # Integration tests
└── *.spec.ts                # Unit tests
```

---

**Built with** [🏗 TS-scaffold](https://github.com/Livshitz/ts-scaffold.git)

**License**: MIT