# lumic-utility-functions

lumic-utility-functions is a versatile npm package that provides a comprehensive collection of utility functions for Node.js projects. It simplifies common tasks including:
- Making LLM API calls (unified interface for OpenAI models)
- Managing AWS services (Lambda, S3)
- Sending Slack messages
- Handling file operations (ZIP files, S3 storage)
- Working with JSON data
- Interacting with Google Sheets

## Installation

```bash
npm install lumic-utility-functions
```

## Usage

Import the lumic namespace:
```javascript
import { lumic } from "lumic-utility-functions";

// Access functions through their namespaces
const openaiResult = await lumic.llm.call("Your prompt", "text", { model: "GPT-5-mini" }); // OpenAI model
const deepseekResult = await lumic.llm.seek("Your prompt", "json"); // Deepseek model
const s3Result = await lumic.s3.upload("bucket", "file.txt");
const slackResult = await lumic.slack.sendMessage("#channel", "message");
const fixedJson = lumic.json.fix(brokenJsonString);
const gsResult = await lumic.gs.addRow({ spreadsheetId: "id", sheetName: "Sheet1" }, ["Value1", "Value2"]);
```

## Required Environment Variables

The following environment variables are recommended for full functionality:

* `MY_AWS_ACCESS_KEY_ID` - For AWS services
* `MY_AWS_REGION` - For AWS services
* `MY_AWS_SECRET_ACCESS_KEY` - For AWS services
* `OPENAI_API_KEY` - For OpenAI models
* `DEEPSEEK_API_KEY` - For Deepseek models
* `SLACK_BOT_TOKEN` - For Slack messaging
* `GOOGLE_SHEETS_CLIENT_EMAIL` - For Google Sheets service account
* `GOOGLE_SHEETS_PRIVATE_KEY` - For Google Sheets service account
* `AI_MODEL` - Default model for LLM calls (optional, defaults to 'GPT-5-mini')

Most functions also accept explicit authentication parameters if you prefer not to use environment variables.

### Table of Contents
- [Installation](#installation)
- [Usage](#usage)
- [LLM Interface](#llm-interface)
- [sendMessageToSlack](#sendmessagetoslack)
- [callPerplexityAPI](#callperplexityapi)
- [invokeLambdaFunction](#invokelambdafunction)
- [Zip File Utilities](#zip-file-utilities)
- [AWS S3 Utilities](#aws-s3-utilities)
- [JSON Utilities](#json-utilities)
- [Google Sheets Utilities](#google-sheets-utilities)
- [Performance Timer](#performance-timer)

## LLM Interface

The package provides a unified interface for interacting with various LLM providers through the `lumic.llm.call` function.

### Supported Models

#### OpenAI Models
- `GPT-5-mini`: Optimized GPT-4 model for general use (default)
- `GPT-5`: Full GPT-4 model with enhanced capabilities
- `o1-mini`: Compact O1 model
- `o1`: Full O1 model
- `o3-mini`: Compact O3 model
- `gpt-4.1`: GPT-4.1 model
- `gpt-4.1-mini`: Compact GPT-4.1 model
- `o4-mini`: Compact O4 model

#### Deepseek Models
- `deepseek-chat`: Versatile chat model with JSON output and function calling support (via `lumic.llm.seek`)
- `deepseek-reasoner`: Advanced reasoning model for complex tasks (via `lumic.llm.seek`)

### Input Parameters

- `content` (string | ContentPart[]): The content to send to the LLM. Can be either:
  - A string: Traditional text prompt
  - An array of content parts: For multi-modal inputs, supporting:
    - Text content: `{ type: 'text', text: string }`
    - Image content: `{ type: 'image_url', image_url: { url: string, detail?: 'auto' | 'low' | 'high' } }`

- `responseFormat` (string | object): The desired format of the response. Can be:
  - "text": Returns plain text response
  - "json": Returns parsed JSON object with advanced parsing strategies
  - JSON schema object: Returns parsed object conforming to the provided schema

- `options` (object, optional): Additional options for the API call
  - `model` (string, optional): The model to use (default: "GPT-5-mini" for OpenAI, "deepseek-chat" for Deepseek)
  - `developerPrompt` (string, optional): Custom developer context prompt
  - `apiKey` (string, optional): Custom API key
  - `reasoning_effort` (string, optional): Specify reasoning depth ('low' | 'medium' | 'high')
  - `temperature` (number, optional): Controls randomness of the output
  - `top_p` (number, optional): Controls diversity of token selection
  - `frequency_penalty` (number, optional): Reduces repetition of tokens
  - `presence_penalty` (number, optional): Encourages discussing new topics
  - `max_completion_tokens` (number, optional): Limits response length
  - `store` (boolean, optional): Option to store conversation
  - `metadata` (object, optional): Additional metadata for stored conversations
  - `tools` (Tool[], optional): Function calling tools

### Response Format

All responses follow a consistent format:
```typescript
interface LLMResponse<T> {
  response: T;                 // The model's response
  usage: {
    prompt_tokens: number;     // Number of tokens in the prompt
    completion_tokens: number; // Number of tokens in the completion
    reasoning_tokens?: number; // Number of tokens for reasoning (Deepseek models)
    provider: string;         // 'openai' or 'deepseek'
    model: string;            // The model name (e.g., 'GPT-5-mini', 'deepseek-chat')
    cache_hit_tokens?: number; // Cache hit tokens for Deepseek models
    cost: number;             // Cost of the API call
  };
  tool_calls?: ToolCall[];    // Optional tool calls if tools were provided
}
```

### Examples

1. Basic text completion with OpenAI:
```javascript
const result = await lumic.llm.call(
  "What is the capital of France?",
  "text",
  { model: "GPT-5-mini" }
);
```

2. JSON output:
```javascript
const result = await lumic.llm.call(
  "List three famous scientists in JSON format",
  "json"
);
```

3. Using function calling:
```javascript
const weatherTool = {
  type: 'function',
  function: {
    name: 'get_weather',
    description: 'Get the current weather for a location',
    parameters: {
      type: 'object',
      properties: {
        location: {
          type: 'string',
          description: 'The city name',
        },
      },
      required: ['location'],
    },
  },
};

const result = await lumic.llm.call(
  "What's the weather like in Tokyo?",
  "text",
  {
    model: "GPT-5-mini",
    tools: [weatherTool],
  }
);
```

4. Using Deepseek models:
```javascript
// Using Deepseek chat model for JSON output
const result = await lumic.llm.seek(
  "List three famous scientists in JSON format",
  "json"
);

// Using Deepseek reasoner model for complex reasoning tasks
const result = await lumic.llm.seek(
  "Explain quantum entanglement in simple terms",
  "text",
  { model: "deepseek-reasoner" }
);
```

### Model Features and Compatibility

| Feature | OpenAI Models | Deepseek Chat | Deepseek Reasoner |
|---------|---------------|---------------|-------------------|
| Text output | ✅ | ✅ | ✅ |
| JSON output | ✅ | ✅ | ❌ |
| Function calling | ✅ | ✅ | ❌ |
| Context length | Up to 128K | 64K | 64K |
| Cost | See [OpenAI pricing](https://openai.com/pricing) | $0.27/1M input tokens, $1.10/1M output tokens | $0.55/1M input tokens, $2.19/1M output tokens |

Note: When attempting to use unsupported features with a model (e.g., JSON output with Deepseek Reasoner), the API will return a graceful error response with details about the incompatibility.

## sendMessageToSlack

The `sendMessageToSlack` function is designed to send a message to a specified Slack channel.

### Input Parameters

- `channel` (string): The Slack channel to send the message to.
- `message` (string): The message content to send.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const success = await lumic.slack.sendMessage("#general", "Hello, Slack!");
console.log(success); // true if message was sent successfully
```

## callPerplexityAPI

The `callPerplexityAPI` function is designed to send a query to the Perplexity API and return the response.

### Input Parameters

- `query` (string): The text query to send to the Perplexity API.
- `maxTokens` (number, optional): The maximum number of tokens for the response (default: 4000).
- `apiKey` (string, optional): The API key to use for the Perplexity API (defaults to the configured PERPLEXITY_API_KEY).

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const result = await lumic.perplexity.call("What is the capital of France?");
console.log(result); // "The capital of France is Paris."
```

## invokeLambdaFunction

The `invokeLambdaFunction` function is designed to invoke an AWS Lambda function.

### Input Parameters

- `functionName` (string): The name of the Lambda function to invoke.
- `payload` (object, optional): The payload to send to the Lambda function (default: empty object).
- `invocationType` (string, optional): The type of invocation (default: 'RequestResponse').
- `auth` (object, optional): AWS credentials for Lambda invocation. If not provided, falls back to environment variables.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

// Using environment variables
const result = await lumic.lambda.invoke(
  "myLambdaFunction",
  { key: "value" }
);

// Using explicit auth credentials
const resultWithAuth = await lumic.lambda.invoke(
  "myLambdaFunction",
  { key: "value" },
  'RequestResponse',
  {
    AWS_ACCESS_KEY_ID: 'your-access-key',
    AWS_SECRET_ACCESS_KEY: 'your-secret-key',
    AWS_REGION: 'us-west-2'
  }
);
```

## Zip File Utilities

The package includes a set of file management utilities to work with zip files, along with other general utility functions.

### createZipFile

The `createZipFile` function compresses a directory into a zip file.

### Input Parameters

- `sourceDir` (string): The path of the directory to compress. It must start with `/tmp/` for security purposes.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const zipFilePath = await lumic.zip.create("/tmp/myDirectory");
console.log(`Zip file created at: ${zipFilePath}`);
```

### extractZipFile

The `extractZipFile` function extracts a zip archive into a specified directory.

### Input Parameters

- `zipPath` (string): The path to the zip file. It must start with `/tmp/`.
- `destPath` (string): The directory where the zip content will be extracted. It must start with `/tmp/`.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const destPath = await lumic.zip.extract("/tmp/myZipFile.zip", "/tmp/extractedFolder");
console.log(`Files extracted to: ${destPath}`);
```

### addToZipFile

The `addToZipFile` function allows adding new files or directories to an existing zip file.

### Input Parameters

- `zipFilePath` (string): The path to the existing zip file (must start with `/tmp/`).
- `itemPath` (string): The file or directory to add to the zip (must start with `/tmp/`).
- `destPath` (string, optional): The destination folder inside the zip.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const updatedZip = await lumic.zip.addTo("/tmp/myZipFile.zip", "/tmp/newFile.txt");
console.log(`File added to zip: ${updatedZip}`);
```

## AWS S3 Utilities

The package includes comprehensive S3 management functions:

### Core Functions
- `create(bucketName, auth?)`: Creates a new S3 bucket
- `destroy(bucketName, auth?)`: Deletes a bucket and all its contents
- `generateBucketName(prefix?)`: Generates a valid S3 bucket name
- `ls(options?)`: Lists buckets or objects in a bucket, sorted by most recent first. Options include:
  - `bucket?`: Optional bucket name. If provided, lists objects in the bucket. If not provided, lists all buckets.
  - `auth?`: Optional AWS credentials
  - `limit?`: Maximum number of items to return (default: 10)

### File Operations
- `upload(bucketName, sourcePath, destinationPath?, auth?, isPublic?)`: Uploads files/directories with optional public access
- `download(bucketName, s3Path?, localPath?, auth?)`: Downloads from S3
- `readFile(bucket, key, auth?)`: Reads a single file as a stream
- `saveTo(bucket, content, filename?, auth?, isPublic?)`: Saves content directly to S3 with optional public access
- `deleteFile(s3FileUrl, auth?)`: Deletes a file using its S3 URL

### Utility Functions
- `generateBucketName(prefix?)`: Creates a valid bucket name with optional prefix
- `checkBucketForFileNamePart(bucketName, fileNamePart, auth?)`: Searches for files in a bucket

### Usage Examples

#### Creating and Managing Buckets
```javascript
import { lumic } from "lumic-utility-functions";

// Generate a unique bucket name with prefix
const bucketName = await lumic.s3.generateBucketName("myproject");

// Create the bucket
await lumic.s3.create(bucketName);

// Clean up when done
await lumic.s3.destroy(bucketName);

// List all buckets in the account (up to 10, most recent first)
const buckets = await lumic.s3.ls();
console.log(buckets.objects); // Array of { key: string, createdAt: number }

// List objects in a specific bucket with custom limit
const objects = await lumic.s3.ls({
  bucket: "my-bucket",
  limit: 5
});
console.log(objects.objects); // Array of { key: string, createdAt: number }
```

#### Uploading and Downloading Files
```javascript
// Upload a file (private by default)
const uploadResult = await lumic.s3.upload(
  "my-bucket",
  "/tmp/myfile.txt"
);

// Upload a publicly accessible file
const publicUploadResult = await lumic.s3.upload(
  "my-bucket",
  "/tmp/myfile.txt",
  {
    isPublic: true,
  }
);

// Download files
const downloadResult = await lumic.s3.download(
  "my-bucket",
  "uploads/",
  "/tmp/downloads"
);

// Stream a file
const fileStream = await lumic.s3.readFile(
  "my-bucket",
  "uploads/myfile.txt"
);
```

#### Working with Content
```javascript
// Save JSON directly to S3 (private by default)
const jsonContent = { name: "John", age: 30 };
const saveResult = await lumic.s3.saveTo(
  "my-bucket",
  jsonContent,
  "data.json"
);

// Save JSON as a publicly accessible file
const publicSaveResult = await lumic.s3.saveTo(
  "my-bucket",
  jsonContent,
  "public-data.json",
  null,  // using default AWS credentials
  true   // make the file public
);

// Check if a file exists
const searchResult = await lumic.s3.checkBucketForFileNamePart(
  "my-bucket",
  "data-2024"
);

// Delete a file using its URL
await lumic.s3.deleteFile(
  "https://my-bucket.s3.region.amazonaws.com/uploads/myfile.txt"
);
```

## JSON Utilities

The package includes utility functions for working with JSON data, particularly for handling and fixing malformed JSON.

### fixBrokenJson

The `fixBrokenJson` function is a robust JSON parser that attempts to repair and parse malformed JSON strings.

### Input Parameters

- `jsonStr` (string): The potentially malformed JSON string to fix and parse

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

// Fix malformed JSON
const brokenJson = `{
  name: "John",
  age: 30,
  hobbies: ['reading' "writing", coding],
  address: {
    street: '123 Main St,
    city: "Springfield"
  },
}`;

const fixed = lumic.json.fix(brokenJson);
console.log(fixed);
```

### parseResponse

The `parseResponse` function is a sophisticated JSON parser that attempts to extract and parse JSON content from various formats, including code blocks.

### Input Parameters

- `content` (string): The content string that may contain JSON
- `responseFormat` ('text' | 'json'): The expected format of the response. Use 'json' to attempt JSON parsing.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

// Parse JSON from a string that might contain code blocks or extra text
const content = `
\`\`\`json
{
  "name": "John",
  "age": 30
}
\`\`\`
`;

const parsed = await lumic.json.parse(content, 'json');
console.log(parsed); // { name: "John", age: 30 }
```

The function employs multiple parsing strategies:
1. Direct JSON parsing
2. Extracting JSON between first {} or []
3. Removing leading/trailing text
4. AI-assisted JSON fixing (using GPT-4)

### isValidJson

The `isValidJson` function checks if a string contains valid JSON.

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

const validJson = '{"name": "John", "age": 30}';
const invalidJson = '{name: John, age: 30}';

console.log(lumic.json.isValid(validJson));                         // true
console.log(lumic.json.isValid(invalidJson));                       // false
```

## Google Sheets Utilities

The package provides utilities for working with Google Sheets through the `lumic.gs` namespace:

### Functions

- `addRow(config, values, startColumn?)`: Add a row of values to a Google Sheet
- `addSheet(spreadsheetId, sheetTitle)`: Add a new sheet to an existing spreadsheet
- `writeCell(config, cell, value)`: Write a value to a specific cell
- `getCell(config, cell)`: Get the value from a specific cell
- `convertA1ToRowCol(a1Notation)`: Convert A1 notation (e.g., 'A1') to row and column numbers
- `writeArray(config, data, startCell?)`: Write a 2D array of data to a sheet

### Usage Example

```javascript
import { lumic } from "lumic-utility-functions";

// Configure the sheet
const config = {
  spreadsheetId: "your-spreadsheet-id",
  sheetName: "Sheet1"
};

// Add a row
await lumic.gs.addRow(config, ["Value1", "Value2", "Value3"]);

// Write to a specific cell
await lumic.gs.writeCell(config, { row: 1, column: 1 }, "Hello World");

// Write an array of data
const data = [
  ["Header1", "Header2"],
  ["Value1", "Value2"],
  ["Value3", "Value4"]
];
await lumic.gs.writeArray(config, data, "A1");
```

## Performance Timer

The `PerformanceTimer` utility helps track execution time and performance metrics across your application. It's accessible through `lumic.utils.Timer`.

### Usage

```javascript
import { lumic } from "lumic-utility-functions";

// Get the global Timer instance
const timer = lumic.utils.Timer;

// Timer starts automatically upon instantiation
timer.checkpoint("start-processing"); // Record a checkpoint

// Do some work...
await someAsyncOperation();
timer.checkpoint("after-async-operation");

// Get current elapsed time in seconds
console.log(`Elapsed time: ${timer.getElapsedSeconds()} seconds`);

// Do more work...
await anotherOperation();
timer.checkpoint("after-another-operation");

// Stop the timer when done
timer.stop();

// Generate a performance report
const report = timer.generateReport();
console.log(report);
// Output:
// {
//   totalTime: 1234.56, // Total time in milliseconds
//   phases: [
//     { label: "start-processing", duration: 100.5 },
//     { label: "after-async-operation", duration: 534.2 },
//     { label: "after-another-operation", duration: 599.86 }
//   ]
// }
```

The PerformanceTimer provides:
- Automatic timing start upon instantiation
- Checkpoint recording with custom labels
- Total execution time tracking
- Quick elapsed time in seconds via `getElapsedSeconds()`
- Detailed performance report with phase-by-phase timing

Note: Use the global Timer instance across your application to track the complete execution flow, rather than creating new instances.

## TypeScript Support

This package includes TypeScript definitions. When using TypeScript, you'll get full type support and autocompletion for all functions and their parameters.

```typescript
import { lumic } from "lumic-utility-functions";

// TypeScript will provide type checking and autocompletion
const result = await lumic.llm.call(
  "Your prompt",
  "json",
  { model: "GPT-5-mini" }
);
```

## Author

The utility-functions project is a product of [Lumic.ai](https://lumic.ai).