/** * Batches Service Resource * Provides access to batch processing for chat completions */ import { type AIMLAPIClient } from '../client'; import { buildPath } from './_paths'; // ===== VERSION ===== const VERSION = 'v1' as const; export interface BatchRequest { /** Custom identifier for this request */ custom_id: string; /** Chat completion parameters */ params: { /** Model to use for the completion */ model: string; /** Maximum number of tokens to generate */ max_tokens?: number; /** Array of messages for the conversation */ messages: Array<{ /** Role of the message sender */ role: 'user' | 'assistant' | 'system'; /** Content of the message */ content: | string | Array<{ /** Type of content */ type: 'text' | 'image'; /** Text content or image URL */ text?: string; /** Image URL (for vision models) */ image_url?: { /** URL of the image */ url: string; }; }>; }>; /** Sampling temperature */ temperature?: number; /** Nucleus sampling parameter */ top_p?: number; /** Stop sequences */ stop_sequences?: string[]; /** System message */ system?: string; /** Metadata for the request */ metadata?: Record; /** Tool choice */ tool_choice?: any; /** Available tools */ tools?: any[]; /** Top-k sampling */ top_k?: number; /** Thinking configuration */ thinking?: { /** Type of thinking (always "enabled") */ type: 'enabled'; /** Budget tokens for thinking */ budget_tokens: number; }; }; } export interface CreateBatchParams { /** Array of batch requests (1-100000) */ requests: BatchRequest[]; /** Optional array of models (max 5) - auto-populated from requests */ models?: string[]; } export interface BatchRequestCounts { /** Number of requests currently processing */ processing: number; /** Number of requests that succeeded */ succeeded: number; /** Number of requests that failed */ errored: number; /** Number of requests that were canceled */ canceled: number; /** Number of requests that expired */ expired: number; } export interface BatchResponse { /** Unique identifier for the batch */ id: string; /** Type of resource (always "message_batch") */ type: 'message_batch'; /** Current processing status */ processing_status: | 'validating' | 'in_progress' | 'completed' | 'failed' | 'canceling' | 'canceled' | 'expired'; /** Request counts by status */ request_counts: BatchRequestCounts; /** When the batch ended (null if still active) */ ended_at: string | null; /** When the batch was created */ created_at: string; /** When the batch expires */ expires_at: string; /** When cancellation was initiated (null if not canceled) */ cancel_initiated_at: string | null; /** URL to download results (null if not completed) */ results_url: string | null; } export interface BatchResult { /** Custom ID from the original request */ custom_id: string; /** Result of the request */ result: { /** Result type */ type: 'succeeded' | 'failed'; /** The generated message (if successful) */ message?: { /** Model used */ model: string; /** Message ID */ id: string; /** Message type */ type: 'message'; /** Message role */ role: 'assistant'; /** Message content */ content: Array<{ /** Content type */ type: 'text'; /** Text content */ text: string; }>; /** Reason for stopping */ stop_reason: 'max_tokens' | 'end_turn' | 'stop_sequence'; /** Stop sequence used (if any) */ stop_sequence: string | null; /** Token usage information */ usage: { /** Input tokens used */ input_tokens: number; /** Cache creation tokens */ cache_creation_input_tokens: number; /** Cache read tokens */ cache_read_input_tokens: number; /** Cache creation details */ cache_creation: { /** 5-minute cache tokens */ ephemeral_5m_input_tokens: number; /** 1-hour cache tokens */ ephemeral_1h_input_tokens: number; }; /** Output tokens used */ output_tokens: number; /** Service tier (always "batch") */ service_tier: 'batch'; }; }; /** Error information (if failed) */ error?: { /** Error type */ type: string; /** Error message */ message: string; }; }; } export class Batches { constructor(private readonly _client: AIMLAPIClient) {} /** * Create a batch of messages for asynchronous processing * All usage is charged at 50% of the standard API prices * * @example * ```typescript * const batch = await client.batches.create({ * requests: [ * { * custom_id: 'test-01', * params: { * model: 'claude-3-5-haiku-20241022', * max_tokens: 1024, * messages: [ * { * role: 'user', * content: 'How to learn NestJS?' * } * ] * } * } * ] * }); * console.log(`Batch created with ID: ${batch.id}`); * ``` */ async create(params: CreateBatchParams): Promise { return this._client.post(buildPath('/batches', VERSION), { body: params as unknown as Record, }); } /** * Get batch status if in progress, or stream results if completed in JSONL format * * @param batchId The ID of the batch to get results for * * @example * ```typescript * const result = await client.batches.retrieve('msgbatch_01AbYVLPKqi8HuSe6sFJV7ZP'); * * if (typeof result === 'string') { * // Parse JSONL results * const lines = result.split('\n').filter(line => line.trim()); * const results = lines.map(line => JSON.parse(line)); * console.log(`Got ${results.length} batch results`); * } else { * // Batch status information * console.log(`Batch status: ${result.processing_status}`); * console.log(`Progress: ${result.request_counts.succeeded}/${result.request_counts.processing + result.request_counts.succeeded}`); * } * ``` */ async retrieve(batchId: string): Promise { const response = await this._client.get<{ status: string; batch?: BatchResponse; message?: string; results_url?: string; }>(buildPath('/batches', VERSION), { query: { batch_id: batchId }, }); // Handle the new response format where batch data is nested if (response.batch) { return response.batch; } // Handle completed batch results (JSONL string or direct results) if (response.results_url) { // If we have a results_url, we need to fetch the actual results // For now, return a placeholder - this would need implementation // to fetch from the results_url throw new Error('Results URL handling not implemented yet'); } // Handle direct string response (backward compatibility) if (typeof response === 'string') { return response as string; } // Fallback for backward compatibility - convert to unknown first return response as unknown as BatchResponse | string; } /** * Cancel a message batch that is currently in progress * Requests that have already started processing will complete, but no new requests will be processed * * @param batchId The ID of the batch to cancel * * @example * ```typescript * const canceled = await client.batches.cancel('msgbatch_01AbYVLPKqi8HuSe6sFJV7ZP'); * console.log(`Batch status: ${canceled.processing_status}`); * ``` */ async cancel(batchId: string): Promise { return this._client.post(buildPath(`/batches/cancel/${batchId}`, VERSION)); } /** * Parse JSONL batch results into an array of BatchResult objects * * @param jsonlString The JSONL string returned from retrieve() when batch is completed * * @example * ```typescript * const result = await client.batches.retrieve('msgbatch_01AbYVLPKqi8HuSe6sFJV7ZP'); * * if (typeof result === 'string') { * const results = client.batches.parseResults(result); * console.log(`Successfully processed ${results.filter(r => r.result.type === 'succeeded').length} requests`); * console.log(`Failed requests: ${results.filter(r => r.result.type === 'failed').length}`); * } * ``` */ parseResults(jsonlString: string): BatchResult[] { const lines = jsonlString.split('\n').filter((line) => line.trim()); return lines.map((line) => JSON.parse(line)); } /** * Wait for a batch to complete and return the results * * @param batchId The ID of the batch to wait for * @param options Polling options * @param options.maxAttempts Maximum number of polling attempts (default: 60) * @param options.pollInterval Interval between polls in seconds (default: 10) * * @example * ```typescript * const results = await client.batches.waitForCompletion( * 'msgbatch_01AbYVLPKqi8HuSe6sFJV7ZP', * { maxAttempts: 120, pollInterval: 30 } * ); * console.log(`Batch completed with ${results.length} results`); * ``` */ async waitForCompletion( batchId: string, options: { maxAttempts?: number; pollInterval?: number } = {} ): Promise { const { maxAttempts = 60, pollInterval = 10 } = options; for (let attempt = 0; attempt < maxAttempts; attempt++) { const result = await this.retrieve(batchId); if (typeof result === 'string') { // Batch is completed, return parsed results return this.parseResults(result); } // Check if batch is in a final state if (['completed', 'failed', 'canceled', 'expired'].includes(result.processing_status)) { if (result.processing_status === 'completed' && result.results_url) { // If we have a results URL, we need to retrieve the actual results const finalResult = await this.retrieve(batchId); if (typeof finalResult === 'string') { return this.parseResults(finalResult); } } throw new Error(`Batch finished with status: ${result.processing_status}`); } // Wait before next poll if (attempt < maxAttempts - 1) { await new Promise((resolve) => setTimeout(resolve, pollInterval * 1000)); } } throw new Error(`Batch did not complete within ${maxAttempts * pollInterval} seconds`); } }