# @ccas-produits-numeriques/common-utilities

Universal utilities library for SIJ applications with **conditional exports** for browser and Node.js environments.

## 🌟 Features

- **Universal**: Works in both browser and Node.js environments
- **Conditional Exports**: Automatically serves the right version based on environment
- **Modular**: Import only what you need
- **Zero Breaking Changes**: Seamless integration with existing projects
- **TypeScript**: Full type definitions included

## Installation

### Requirements

- Node.js >= 24
- pnpm (recommended package manager)

```bash
pnpm add @ccas-produits-numeriques/common-utilities
```

## 📁 File Structure

```
src/
├── core/                    # Universal utilities (browser + Node.js)
│   ├── assertUnreachable.ts     # TypeScript exhaustiveness checking
│   ├── asyncUtils.ts            # timeout(), sleep() with AbortSignal
│   ├── diacritics.ts            # removeDiacritics() for text processing
│   ├── fixtureHelper.ts         # getFixtureValue() for testing
│   ├── generateUri.ts           # URI generation with parameters
│   ├── intervals.utils.ts       # Date interval operations
│   ├── routes.ts                # Route schemas (universal)
│   └── sortAlphabetically.ts    # Alphabetical sorting utilities
│
├── http/                    # Universal HTTP utilities
│   └── schemas/
│       └── errorResponse.ts     # HTTP error response schemas
│
├── types/                   # Universal type definitions
│   ├── environment.ts           # Environment enums
│   └── errors.ts                # Error type definitions
│
├── zod/                     # Universal Zod utilities
│   └── date.ts                  # Date validation and parsing
│
├── logging/                 # Universal logging utilities (Node.js-specific)
│   ├── logger.ts                # Pino logger with MongoDB support
│   ├── sentry.ts                # Sentry error tracking
│   ├── sentry.fastify.ts        # Sentry Fastify integration
│   └── withCause.ts             # Error causation tracking
│
├── node/                    # Node.js-specific utilities
│   ├── core/
│   │   ├── cryptoUtils.ts       # Password hashing, key generation
│   │   └── esmUtils.ts          # __dirname() for ES modules
│   ├── http/
│   │   ├── apiUtils.ts          # File operations, rate limiting
│   │   └── fetchClient.ts       # Fetch wrapper with proxy support
│   ├── logging/
│   │   ├── logger.ts            # Pino logger with MongoDB support
│   │   ├── sentry.ts            # Sentry error tracking
│   │   ├── sentry.fastify.ts    # Sentry Fastify integration
│   │   └── withCause.ts         # Error causation tracking
│   ├── fastify/
│   │   └── middlewares/
│   │       ├── errorMiddleware.ts # Error handling middleware
│   │       └── logMiddleware.ts   # Request logging middleware
│   ├── mongodb/                 # MongoDB utilities
│   │   ├── utils.ts             # Collection operations
│   │   ├── schema.ts            # Schema validation with Zod
│   │   └── indexes.ts           # Index management
│   └── types/
│       └── models.ts            # MongoDB model descriptors
│
└── browser/                 # Browser-specific exports
    ├── core/                    # Re-exports universal core utilities
    ├── http/                    # Re-exports universal HTTP utilities
    ├── types/                   # Re-exports universal types
    ├── zod/                     # Re-exports universal Zod utilities
    ├── logging/                 # Empty (Node.js-only)
    └── fastify/                 # Empty (Node.js-only)
```

## 🚀 Quick Start

### Universal Utilities (Browser + Node.js)

```typescript
// String and text processing
import { removeDiacritics, sortAlphabeticallyBy } from "@ccas-produits-numeriques/common-utilities/core";

// Async utilities with AbortSignal support
import { timeout, sleep } from "@ccas-produits-numeriques/common-utilities/core";

// URI generation with type safety
import { generateUri } from "@ccas-produits-numeriques/common-utilities/core";

// TypeScript exhaustiveness checking
import { assertUnreachable } from "@ccas-produits-numeriques/common-utilities/core";

// ESM module utilities (Node.js only)
import { __dirname } from "@ccas-produits-numeriques/common-utilities";

// Route schemas (works in browser and server)
import { zCoreRoutes } from "@ccas-produits-numeriques/common-utilities/core";
```

### HTTP Utilities (Universal)

```typescript
// HTTP error response schemas
import { IResError, IResErrorJson, ZResError } from "@ccas-produits-numeriques/common-utilities/http";
import { zResBadRequest, zResNotFound } from "@ccas-produits-numeriques/common-utilities/http";
```

### Type Definitions (Universal)

```typescript
// Environment types
import { AppEnvironmentEnum } from "@ccas-produits-numeriques/common-utilities/types";

// Error types
import type { IResError, IResErrorJson } from "@ccas-produits-numeriques/common-utilities/types";
```

### Node.js-Only Utilities

```typescript
// Crypto operations (Node.js only)
import { hashPassword, generateKey, verifyPassword } from "@ccas-produits-numeriques/common-utilities";

// File operations and rate limiting (Node.js only)
import { apiRateLimiter, downloadFileAsStream } from "@ccas-produits-numeriques/common-utilities";

// HTTP client with proxy support (Node.js only)
import { getClientFetch, type FetchOptions, type FetchRequestInit } from "@ccas-produits-numeriques/common-utilities";

// Logging with Sentry integration (Node.js only)
import {
  createLogger,
  initSentry,
  withCause,
  initSentryFastify,
} from "@ccas-produits-numeriques/common-utilities/logging";

// Fastify middleware (Node.js only)
import { errorMiddleware, logMiddleware } from "@ccas-produits-numeriques/common-utilities/fastify";

// MongoDB utilities (Node.js only)
import {
  clearCollection,
  clearAllCollections,
  collectionExistInDb,
  createCollectionIfNotExists,
  configureSchemaValidation,
  createIndexes,
} from "@ccas-produits-numeriques/common-utilities/mongodb";
```

## 🔄 How Conditional Exports Work

This library automatically serves the correct version based on your environment:

```typescript
// Same import, different behavior:
import { assertUnreachable } from "@ccas-produits-numeriques/common-utilities/core";

// In Browser: Gets universal version (src/core/assertUnreachable.ts)
// In Node.js: Gets full version including Node.js-specific utilities
```

### Import Paths Guide

**Universal utilities (Browser + Node.js):**

- `/core` - Core utilities including:
  - String utilities: `removeDiacritics`, `sortAlphabeticallyBy`
  - Async utilities: `timeout`, `sleep`
  - URI utilities: `generateUri`, `generatePath`, `generateQueryString`
  - Validation: `assertUnreachable`
  - Testing: `getFixtureValue`
  - Intervals: `substractInterval`, `substractIntervals`
- `/http` - HTTP error schemas only (`ZResError`, `IResError`, etc.)
- `/types` - Type definitions (`AppEnvironmentEnum`, etc.)
- `/zod` - Zod date utilities (`ParisDate`, `zParisLocalDateString`, etc.)

**Node.js-specific utilities:**

- From root (`@ccas-produits-numeriques/common-utilities`) - Node.js-specific utilities:
  - `getClientFetch` - HTTP client with proxy support
  - `apiRateLimiter` - API rate limiting
  - `downloadFileAsStream`, `downloadFileAsTmp` - File operations
  - `computeFileHash` - File hashing
  - `__dirname` - ESM module directory helper
  - `hashPassword`, `generateKey`, `verifyPassword` - Crypto utilities
  - Plus all exports from `/core`, `/http`, `/types`, `/zod`
- `/logging` - Logging utilities (Node.js only)
- `/fastify` - Fastify middleware (Node.js only)

### Browser Environment

- ✅ Gets universal utilities only
- ❌ No Node.js dependencies (crypto, fs, async_hooks, etc.)
- ✅ Prevents "Module not found: Can't resolve 'async_hooks'" errors

### Node.js Environment

- ✅ Gets all utilities (universal + Node.js-specific)
- ✅ Full crypto, file system, and Sentry integration
- ✅ Fastify middleware and logging capabilities

## 📦 Dependencies

### Always Available

- **@hapi/boom**: Error handling utilities (included as dependency)

### Peer Dependencies (install as needed)

- **zod** (>= 4.0.0): Schema validation (for `/zod` and route schemas)
- **luxon**: Date handling (for `/zod` date utilities)
- **type-fest**: Type utilities (for advanced TypeScript types)
- **fastify**: Web framework (for `/fastify` middleware)
- **pino**: Logging (for `/logging` utilities)
- **@sentry/node**: Error tracking (for `/logging` utilities)
- **mongodb**: Database (for model descriptors in `/types` and MongoDB utilities)
- **zod-mongodb-schema**: Zod to MongoDB schema conversion (for MongoDB utilities)
- **rate-limiter-flexible**: API rate limiting (for `/http` utilities)

### Installation Examples

```bash
# Basic usage (universal utilities only)
pnpm add @ccas-produits-numeriques/common-utilities

# With HTTP error schemas and validation
pnpm add @ccas-produits-numeriques/common-utilities zod

# With date utilities and timezone handling
pnpm add @ccas-produits-numeriques/common-utilities zod luxon

# Node.js server with Fastify integration
pnpm add @ccas-produits-numeriques/common-utilities fastify zod pino @sentry/node

# Full installation (all features)
pnpm add @ccas-produits-numeriques/common-utilities zod luxon fastify pino @sentry/node mongodb zod-mongodb-schema rate-limiter-flexible type-fest
```

## API Documentation

### Core Utilities

#### String Utilities

- `removeDiacritics(str: string): string` - Remove diacritical marks from strings
- `sortAlphabeticallyBy<T>(key: string, array: T[]): T[]` - Sort arrays alphabetically by property

#### URI Utilities

- `generateUri(path: string, options?: PathOptions): string` - Generate URIs with parameters
- `generatePath(path: string, params: Record<string, string>): string` - Generate paths with parameters
- `generateQueryString(query: Record<string, string | string[]>): string` - Generate query strings

#### Validation Utilities

- `assertUnreachable(x: never): never` - TypeScript exhaustiveness checking

#### Async Utilities

- `timeout<T>(promise: Promise<T>, millis: number): Promise<T>` - Add timeout to promises
- `sleep(ms: number, signal?: AbortSignal): Promise<void>` - Abortable sleep function

#### Crypto Utilities

- `hashPassword(password: string, rounds: number, salt?: string): string` - PBKDF2 password hashing
- `generateKey(size: number, format: string): string` - Generate secure API keys

#### Interval Utilities

- `substractInterval(a: Interval, b: Interval): Interval[]` - Subtract date intervals
- `substractIntervals(a: Interval[], b: Interval[]): Interval[]` - Subtract multiple intervals

#### ESM Utilities

- `__dirname(filePath: string): string` - Get directory name from import.meta.url in ESM modules

```typescript
import { __dirname } from "@ccas-produits-numeriques/common-utilities";

// Usage in an ESM module
const currentDir = __dirname(import.meta.url);
console.log(currentDir); // e.g., "/Users/username/project/src"

// Use it to resolve paths relative to the current file
import { join } from "path";
const configPath = join(__dirname(import.meta.url), "../config.json");
```

### Types

#### Environment Types

- `AppEnvironmentEnum` - Environment enumeration (local, recette, production, preview, test)
- `Environment` - Type alias for AppEnvironmentEnum

#### Error Types

- `ZResError` - Zod schema for standardized error responses
- `IResError` - TypeScript interface for error responses

### Routes

Route schemas are available universally from the `/core` module for use in both browser and Node.js environments.

#### Route Schema Types

- `IHealthcheckRoutesDef` - Interface for healthcheck route definitions
- `zCoreRoutes` - Core route schemas including healthcheck endpoint

### Zod Utilities

#### Date Classes

- `ParisDate` - Date class with Paris timezone JSON serialization
- `parisTimezoneDate(parts: DateParts): ParisDate` - Create Paris timezone date from parts

#### Date Parsers

- `parseParisLocalDate(date: string, time?: string, offset?: number): ParisDate` - Parse local dates
- `parseNullableParisLocalDate(date: string | null, time?: string, offset?: number): ParisDate | null` - Parse nullable dates

#### Zod Schemas

- `zParisLocalDateString` - Validate DD/MM/YYYY format and convert to ParisDate
- `zLocalDate` - Transform Date to ParisDate
- `zParisLocalDate` - Parse ISO date strings to ParisDate

### HTTP Utilities

#### Fetch Client (Node.js only)

- `getClientFetch(options: FetchOptions): FetchWithParams` - Create a fetch client with proxy support

```typescript
import { getClientFetch } from "@ccas-produits-numeriques/common-utilities";

// Create a configured fetch client
const client = getClientFetch({
  baseURL: "https://api.example.com",
  headers: {
    Accept: "application/json",
    "User-Agent": "MyApp/1.0",
  },
  proxy: "http://proxy.example.com:8080",
  env: "production",
  timeout: 10000, // 10 seconds
});

// Make requests with params support
const response = await client("/users", {
  method: "GET",
  params: {
    page: 1,
    limit: 20,
  },
});

// The client is a wrapped fetch function with additional features:
// - Automatic proxy configuration based on environment
// - Timeout support with AbortSignal
// - Base URL support for relative paths
// - Query parameters as an object
// - Default headers
```

##### FetchOptions

- `baseURL?: string` - Base URL for relative paths
- `headers?: Record<string, string>` - Default headers for all requests
- `proxy?: string` - Proxy URL (only used in non-local/test environments)
- `env?: string` - Environment name (local, test, production, etc.)
- `timeout?: number` - Request timeout in milliseconds (default: 5000)

##### FetchRequestInit

Extends the standard `RequestInit` with:

- `params?: Record<string, string | number | boolean>` - Query parameters as an object

#### Rate Limiting

- `apiRateLimiter(name: string, options: RateLimitOptions): RateLimiterFn` - Advanced API rate limiting

#### File Operations

- `downloadFileAsTmp(stream: Readable, filename: string, env: string): Promise<string>` - Download to temp file
- `downloadFileAsStream(stream: Readable, filename: string, env: string): Promise<ReadStream>` - Download as stream
- `computeFileHash(stream: Readable): Promise<string>` - Compute SHA-256 hash

### Logging Utilities

#### Logger Creation

- `createLogger(config: LogConfig): Logger` - Create configured Pino logger
- `createJobProcessorLogger(logger: PinoLogger): ILogger` - Create job processor logger adapter

#### Error Handling

- `withCause<T>(error: T, cause: Error, level?: string): T` - Add error causation

#### Sentry Integration

- `initSentry(config: SentryConfig): void` - Initialize Sentry error tracking
- `initSentryFastify(app: FastifyInstance): void` - Initialize Sentry Fastify integration
- `closeSentry(): Promise<void>` - Graceful Sentry shutdown

### Fastify Utilities

#### Middleware

- `errorMiddleware(server: FastifyInstance, config: Config): void` - Standardized error handling
- `logMiddleware(config: Config): FastifyLoggerOptions` - Request/response logging with security filtering

### MongoDB Utilities (Node.js only)

#### Collection Operations

- `clearCollection(collection: Collection): Promise<void>` - Clear all documents from a collection
- `clearAllCollections(db: Db): Promise<void>` - Clear all collections in a database
- `collectionExistInDb(collections: CollectionInfo[], name: string): boolean` - Check if collection exists
- `createCollectionIfNotExists(db: Db, collectionName: string): Promise<void>` - Create collection if it doesn't exist

```typescript
import { clearCollection, collectionExistInDb } from "@ccas-produits-numeriques/common-utilities/mongodb";

// Clear a single collection
await clearCollection(db.collection("users"));

// Check if collection exists
const collections = await db.listCollections().toArray();
if (collectionExistInDb(collections, "products")) {
  console.log("Products collection exists");
}
```

#### Schema Validation

- `configureSchemaValidation(db: Db, modelDescriptors: ModelDescriptor[], options: SchemaValidationOptions): Promise<void>` - Configure Zod-based schema validation for MongoDB collections

```typescript
import { configureSchemaValidation } from "@ccas-produits-numeriques/common-utilities/mongodb";
import { z } from "zod";

const userSchema = z.object({
  name: z.string(),
  email: z.string().email(),
  age: z.number().positive(),
});

const modelDescriptors = [
  { collectionName: "users", zod: userSchema },
  { collectionName: "products" }, // No schema validation
];

await configureSchemaValidation(db, modelDescriptors, {
  logger,
  validationLevel: "strict",
  validationAction: "error",
  onError: (error) => console.error("Schema validation error:", error),
});
```

#### Index Management

- `createIndexes(db: Db, indexDefinitions: IndexDefinition[], options: CreateIndexesOptions): Promise<void>` - Create and manage indexes with conflict resolution

```typescript
import { createIndexes } from "@ccas-produits-numeriques/common-utilities/mongodb";

const indexDefinitions = [
  {
    collectionName: "users",
    indexes: [
      [{ email: 1 }, { unique: true, name: "email_unique" }],
      [{ name: 1, createdAt: -1 }, { name: "name_date_compound" }],
    ],
  },
  {
    collectionName: "products",
    indexes: [
      [{ sku: 1 }, { unique: true }],
      [{ category: 1, price: -1 }, { name: "category_price" }],
    ],
  },
];

// Create indexes (with automatic conflict resolution)
await createIndexes(db, indexDefinitions, {
  logger,
  drop: false, // Set to true to drop all indexes first
  onError: (error) => captureException(error),
});

// The function handles:
// - Creating new indexes
// - Resolving conflicts (error codes 85, 86)
// - Dropping outdated indexes
// - Skipping collections without indexes
```

## Development

### Scripts

```bash
# Build the library
pnpm build

# Watch mode for development
pnpm dev

# Run tests
pnpm test

# Run tests in CI mode
pnpm test:ci

# Run tests with coverage
pnpm test:ci:coverage

# Run linting
pnpm lint

# Fix linting issues
pnpm lint:fix

# Format code
pnpm prettier:fix

# Check formatting
pnpm prettier:check

# Type checking
pnpm typecheck

# Run all checks (typecheck, lint, test)
pnpm check

# Clean build directory
pnpm clean
```

## Migration from Shared Package

If you're migrating from a shared package, update your imports:

```typescript
// Before (shared package)
import { AppEnvironmentEnum } from "shared/types";
import { IRouteSchema } from "shared/routes/common.routes";
import { ZResError } from "shared/src/types/errors";
import { apiRateLimiter } from "./utils/apiUtils";
import { createLogger } from "./services/logger";
import { errorMiddleware } from "./middlewares/errorMiddleware";

// After (@ccas-produits-numeriques/common-utilities)
import { AppEnvironmentEnum } from "@ccas-produits-numeriques/common-utilities/types";
import { zCoreRoutes, IHealthcheckRoutesDef } from "@ccas-produits-numeriques/common-utilities/core";
import { ZResError } from "@ccas-produits-numeriques/common-utilities/types";
import { apiRateLimiter } from "@ccas-produits-numeriques/common-utilities";
import { createLogger } from "@ccas-produits-numeriques/common-utilities/logging";
import { errorMiddleware } from "@ccas-produits-numeriques/common-utilities/fastify";
```

## Recent Updates

### v3.1.0 - Dependency Updates

- **Updated** `rate-limiter-flexible` peer dependency from v7 to v9
- **Updated** various peer dependencies to latest versions

### v1.5.0 - MongoDB Utilities

- **Added** comprehensive MongoDB utilities for common operations
- **Features**:
  - Collection operations: `clearCollection`, `clearAllCollections`, `collectionExistInDb`, `createCollectionIfNotExists`
  - Schema validation: `configureSchemaValidation` with Zod integration
  - Index management: `createIndexes` with automatic conflict resolution
  - Bug fix: Fixed early exit issue in `createIndexes` (using `continue` instead of `return`)
- **Dependencies**: Added `zod-mongodb-schema` as peer dependency for schema conversion

### v1.4.0 - Native Fetch Client

- **Added** `getClientFetch()` - A lightweight fetch wrapper with proxy support
- **Features**:
  - Native Node.js fetch (no axios dependency)
  - Automatic proxy configuration via undici's ProxyAgent
  - Built-in timeout support with AbortSignal
  - Base URL support for relative paths
  - Query parameters as objects
  - Default headers configuration
- **Removed** deprecated HTTP clients (`createHttpClient`, `createHttpClientNative`)
- **Removed** `node-fetch-native` dependency in favor of native Node.js fetch

### v1.1.0 - Enhanced Logging and Middleware

- **Added** `initSentryFastify()` for Fastify-specific Sentry integration
- **Enhanced** `logMiddleware()` with security-aware field filtering
- **Improved** error handling with better causation tracking
- **Updated** API rate limiter to support both Axios and Fetch clients
- **Added** comprehensive file operations utilities
- **Enhanced** crypto utilities with configurable hash rounds

### Migration Examples

#### Migrating from Axios to Fetch Client

```typescript
// Before: Using axios
import axios from "axios";

const client = axios.create({
  baseURL: "https://api.example.com",
  headers: { "User-Agent": "MyApp/1.0" },
  timeout: 10000,
  httpsAgent: new HttpsProxyAgent(config.proxy.https),
});

const response = await client.get("/users", {
  params: { page: 1, limit: 20 },
});
const data = response.data;

// After: Using getClientFetch
import { getClientFetch } from "@ccas-produits-numeriques/common-utilities";

const client = getClientFetch({
  baseURL: "https://api.example.com",
  headers: { "User-Agent": "MyApp/1.0" },
  timeout: 10000,
  proxy: config.proxy.https,
  env: config.env,
});

const response = await client("/users", {
  method: "GET",
  params: { page: 1, limit: 20 },
});
const data = await response.json();
```

#### Tools Project Migration

```typescript
// Before: Local duplicate utilities
import { apiRateLimiter } from "./utils/apiUtils";
import { timeout } from "./utils/asyncUtils";
import { createLogger } from "./services/logger";
import { errorMiddleware } from "./middlewares/errorMiddleware";
import { assertUnreachable } from "shared/src/utils/assertUnreachable";

// After: Direct imports from common-utilities
import { apiRateLimiter } from "@ccas-produits-numeriques/common-utilities";
import { timeout } from "@ccas-produits-numeriques/common-utilities/core";
import { createLogger } from "@ccas-produits-numeriques/common-utilities/logging";
import { errorMiddleware } from "@ccas-produits-numeriques/common-utilities/fastify";
import { assertUnreachable } from "@ccas-produits-numeriques/common-utilities/core";
```

#### Benefits of Migration

- ✅ **Zero code duplication** - All utilities centralized in common-utilities
- ✅ **Consistent behavior** - Same implementation across all projects
- ✅ **Better maintenance** - Updates propagate from single source
- ✅ **Type safety** - Full TypeScript support with proper exports
- ✅ **Performance** - Tree-shakable imports, only load what you need