# @aikotools/datacompare

Advanced data comparison engine with directive-based matching for E2E testing.

[![npm version](https://badge.fury.io/js/@aikotools%2Fdatacompare.svg)](https://www.npmjs.com/package/@aikotools/datacompare)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

## Overview

`@aikotools/datacompare` provides sophisticated recursive object and array comparison with support for:

- **Deep Recursive Comparison**: Navigate nested object and array structures
- **Directive-Based Matching**: Flexible comparison directives with `{{compare:...}}` syntax
- **Time Range Comparisons**: Compare timestamps with tolerance ranges (±5 minutes, etc.)
- **Number Range Comparisons**: Validate numbers within ranges or tolerances
- **String Pattern Matching**: Regex, startsWith, endsWith, contains patterns
- **Array Flexibility**: Order-independent, partial matching, wildcards
- **Type Preservation**: Maintains proper types during comparison

## Installation

```bash
npm install @aikotools/datacompare
```

## Quick Start

```typescript
import { compareData } from '@aikotools/datacompare';

const result = await compareData({
  expected: {
    userId: 'user_12345',
    email: '{{compare:endsWith:@example.com}}',
    score: '{{compare:number:range:0:100}}',
    timestamp: '{{compare:time:range:-60:+60:seconds}}',
  },
  actual: {
    userId: 'user_12345',
    email: 'john.doe@example.com',
    score: 85,
    timestamp: '2023-12-01T10:30:00Z',
    extra: 'ignored by default',
  },
  context: {
    startTimeTest: '2023-12-01T10:30:00Z',
  },
});

console.log(result.success); // true
console.log(result.stats);   // { totalChecks: 4, passedChecks: 4, ... }
```

## Features

### 1. Basic Object Comparison

```typescript
await compareData({
  expected: { name: 'John', age: 30 },
  actual: { name: 'John', age: 30, extra: 'ignored' },
});
// ✅ Passes - extra properties ignored by default
```

### 2. Array Comparison

```typescript
// Ordered comparison
await compareData({
  expected: [1, 2, 3],
  actual: [1, 2, 3],
});
// ✅ Exact match

// Order-independent
await compareData({
  expected: ['{{compare:ignoreOrder}}', 'apple', 'banana'],
  actual: ['banana', 'apple'],
});
// ✅ Order doesn't matter

// Partial matching
await compareData({
  expected: [1, 2, '{{compare:ignoreRest}}'],
  actual: [1, 2, 3, 4, 5],
});
// ✅ Only validates first 2 elements
```

### 3. String Pattern Directives

```typescript
await compareData({
  expected: {
    message: '{{compare:startsWith:Hello}}',
    email: '{{compare:endsWith:@example.com}}',
    userId: '{{compare:regex:user_[0-9]{5}}}',
    log: '{{compare:contains:ERROR}}',
  },
  actual: {
    message: 'Hello World',
    email: 'john@example.com',
    userId: 'user_12345',
    log: '[2023-12-01] ERROR: Failed',
  },
});
// ✅ All patterns match
```

### 4. Time Range Comparisons (NEW!)

```typescript
const now = new Date().toISOString();

// Combined range: past and future (±5 minutes)
await compareData({
  expected: { timestamp: '{{compare:time:range:-300:+300:seconds}}' },
  actual: { timestamp: now },
  context: { startTimeTest: now },
});

// Future only: up to 1 hour in the future
await compareData({
  expected: { timestamp: '{{compare:time:range:+60:minutes}}' },
  actual: { timestamp: futureTime },
  context: { startTimeTest: now },
});

// Past only: up to 1 hour in the past
await compareData({
  expected: { timestamp: '{{compare:time:range:-60:minutes}}' },
  actual: { timestamp: pastTime },
  context: { startTimeTest: now },
});

// Exact time match (no offset = baseTime itself)
await compareData({
  expected: { timestamp: '{{compare:time:exact}}' },
  actual: { timestamp: now },
  context: { startTimeTest: now },
});

// Exact time match with offset (baseTime + offset)
await compareData({
  expected: { abfahrt: '{{compare:time:exact:630:seconds}}' },
  actual: { abfahrt: departureTime }, // exactly baseTime + 630 seconds
  context: { startTimeTest: baseTime },
});

// Exact time match with negative offset (baseTime - offset)
await compareData({
  expected: { arrival: '{{compare:time:exact:-10:minutes}}' },
  actual: { arrival: arrivalTime }, // exactly baseTime - 10 minutes
  context: { startTimeTest: baseTime },
});
```

### 5. Number Range Comparisons (NEW!)

```typescript
// Range validation
await compareData({
  expected: { score: '{{compare:number:range:0:100}}' },
  actual: { score: 85 },
});
// ✅ 85 is within [0, 100]

// Tolerance (absolute)
await compareData({
  expected: { value: '{{compare:number:tolerance:42:±5}}' },
  actual: { value: 44 },
});
// ✅ 44 is within 42±5 (37-47)

// Tolerance (percentage)
await compareData({
  expected: { value: '{{compare:number:tolerance:100:±10%}}' },
  actual: { value: 95 },
});
// ✅ 95 is within 100±10% (90-110)
```

### 6. Ignore Paths

Skip entire subtrees during comparison, regardless of type differences or nested mismatches.
Useful for fields that are known to differ (e.g. direction fields populated by external systems).

```typescript
// Ignore a specific field
await compareData({
  expected: { name: 'John', secret: 'expectedValue' },
  actual: { name: 'John', secret: 'completelyDifferent' },
  options: {
    ignorePaths: [
      { path: ['secret'], doc: ['Secret field ignored for testing'] }
    ]
  }
});
// ✅ Passes - 'secret' is completely skipped

// Wildcard matching with '*' (matches any segment including array indices)
await compareData({
  expected: {
    data: {
      allFahrtereignis: [
        { name: 'event1', richtung: 'NORD' },
        { name: 'event2', richtung: 'SUED' }
      ]
    }
  },
  actual: {
    data: {
      allFahrtereignis: [
        { name: 'event1', richtung: 'WEST' },
        { name: 'event2', richtung: 'OST' }
      ]
    }
  },
  options: {
    ignorePaths: [
      { path: ['data', 'allFahrtereignis', '*', 'richtung'], doc: ['Richtung wird ignoriert'] }
    ]
  }
});
// ✅ Passes - 'richtung' in all array elements is skipped

// Subtree ignoring (entire nested structure is skipped)
await compareData({
  expected: { data: { nested: { deep: 'a', another: [1, 2, 3] } } },
  actual: { data: { nested: { deep: 'b', another: [4, 5] } } },
  options: {
    ignorePaths: [
      { path: ['data', 'nested'], doc: ['Entire subtree ignored'] }
    ]
  }
});
// ✅ Passes - everything under 'data.nested' is skipped
```

**IgnorePathConfig:**
- `path`: Array of path segments. Use `'*'` as wildcard to match any single segment (including array indices like `[0]`).
- `doc`: Optional array of documentation strings explaining why this path is ignored.

### 7. Special Keywords

```typescript
// Exact matching (no extra properties allowed)
await compareData({
  expected: {
    '{{compare:exact}}': true,
    name: 'John',
    age: 30,
  },
  actual: {
    name: 'John',
    age: 30,
    extra: 'property', // ❌ Will cause error in exact mode
  },
});

// Ignore specific values
await compareData({
  expected: {
    userId: 'user_123',
    timestamp: '{{compare:ignore}}', // Ignored
  },
  actual: {
    userId: 'user_123',
    timestamp: '2023-12-01T10:30:00Z', // Any value OK
  },
});
```

## API Reference

### compareData(request: CompareRequest): Promise<CompareResult>

Main comparison function.

**Parameters:**
- `expected`: Expected object with compare directives
- `actual`: Actual object to compare against
- `context`: Optional context (startTimeTest, startTimeScript, etc.)
- `options`: Optional comparison options
  - `format`: Data format (`'json'`, `'text'`, `'xml'`)
  - `strictMode`: Exact matching everywhere (default: `false`)
  - `ignoreExtraProperties`: Ignore extra properties in actual (default: `true`)
  - `maxDepth`: Maximum nesting depth
  - `maxErrors`: Stop after N errors
  - `ignorePaths`: Array of `IgnorePathConfig` to skip entire subtrees (see section 6)

**Returns:** `CompareResult` with:
- `success`: boolean - Overall status
- `errors`: CompareError[] - List of all errors
- `details`: CompareDetail[] - Detailed check information
- `stats`: CompareStats - Comparison statistics

### createDefaultEngine(): CompareEngine

Creates a CompareEngine with all built-in directives registered.

### CompareEngine

Core comparison engine. Use for custom directive registration:

```typescript
import { createDefaultEngine, MyCustomDirective } from '@aikotools/datacompare';

const engine = createDefaultEngine();
engine.getRegistry().registerDirective(new MyCustomDirective());

const result = await engine.compare({
  expected: { /* ... */ },
  actual: { /* ... */ },
});
```

## Directive Reference

### String Patterns
- `{{compare:startsWith:pattern}}` - String must start with pattern
- `{{compare:endsWith:pattern}}` - String must end with pattern
- `{{compare:regex:pattern}}` - String must match regex
- `{{compare:contains:substring}}` - String must contain substring

### Time Ranges
- `{{compare:time:range:-N:+M:unit}}` - Time within range (N units in past, M units in future)
- `{{compare:time:range:+N:unit}}` - Time within N units in the future only
- `{{compare:time:range:-N:unit}}` - Time within N units in the past only
- `{{compare:time:exact}}` - Exact time match with baseTime (offset = 0)
- `{{compare:time:exact:N:unit}}` - Exact time match with baseTime + N units (supports negative values)

**Units:** milliseconds, seconds, minutes, hours, days, weeks, months, years

**Base Time Priority:** startTimeTest > startTimeScript > current time (from context)

### Number Ranges
- `{{compare:number:range:min:max}}` - Number within [min, max]
- `{{compare:number:tolerance:value:±N}}` - Number within value±N
- `{{compare:number:tolerance:value:±N%}}` - Number within value±N%

### Special Keywords
- `{{compare:exact}}` - Enable exact property matching
- `{{compare:ignore}}` - Ignore this value
- `{{compare:ignoreRest}}` - Ignore remaining array elements
- `{{compare:ignoreOrder}}` - Ignore array element order

## Examples

See the integration tests for comprehensive examples:
- [BasicComparison.test.ts](tests/integration/BasicComparison.test.ts) - Core comparison features
- [IgnorePaths.test.ts](tests/integration/IgnorePaths.test.ts) - ignorePaths feature

## Migration from e2e-tool-util-compare-object

```typescript
// Old syntax
const expected = {
  "__EXACT__": true,
  userId: "<compare:ref:userId>",
  email: "<CS:ENDS_WITH>@example.com",
};

// New syntax
const expected = {
  "{{compare:exact}}": true,
  userId: "{{compare:ref:userId}}", // Coming in future release
  email: "{{compare:endsWith:@example.com}}",
};
```

## Architecture

```
CompareEngine
├── CompareParser (parses {{compare:...}} directives)
├── CompareRegistry (manages directives, matchers, transforms)
└── RecursiveComparer (deep object/array comparison)
    ├── Object comparison (partial/exact matching)
    ├── Array comparison (ordered/unordered/partial)
    └── Directive evaluation (startsWith, time, number, etc.)
```

## Development

```bash
# Install dependencies
npm install

# Build
npm run build

# Test
npm test

# Lint
npm run lint

# Format
npm run format
```

## Test Results

✅ 39/39 tests passing (4 test files)
- ✅ Basic object/array comparison
- ✅ String pattern directives (startsWith, endsWith, contains, regex)
- ✅ Time range directives
- ✅ Number range directives
- ✅ Array flexibility (ignoreOrder, ignoreRest)
- ✅ ignorePaths (exact, wildcard, subtree, nested)

## Roadmap

### Implemented ✅
- [x] Core comparison engine
- [x] Recursive deep comparison
- [x] String pattern directives
- [x] Time range comparisons
- [x] Number range comparisons
- [x] Array flexibility
- [x] ignorePaths (skip subtrees with wildcard support)

### Coming Soon
- [ ] Reference directives (`{{compare:ref:anId}}`)
- [ ] Custom transform pipeline
- [ ] JSON/Text processors
- [ ] Performance optimizations

## License

MIT - See [LICENSE](LICENSE) file for details.

## Contributing

This is part of the @aikotools ecosystem. For issues and contributions, please see the repository.

---

**Built with ❤️ for E2E Testing**