# Elasticsearch Client

Search, index, and manage documents in Elasticsearch clusters.

## Methods

| Method                                   | Description                                               |
| ---------------------------------------- | --------------------------------------------------------- |
| `apiRequest(options, schema, metadata?)` | Make any Elasticsearch API request with schema validation |

## Usage

### Search Documents

```typescript
import { api, z, elasticsearch } from "@superblocksteam/sdk-api";

// Integration ID from the integrations panel
const PROD_ELASTICSEARCH = "a1b2c3d4-5678-90ab-cdef-elastic00001";

const HitSchema = z.object({
  _index: z.string(),
  _id: z.string(),
  _score: z.number().nullable(),
  _source: z.record(z.unknown()),
  highlight: z.record(z.array(z.string())).optional(),
});

const SearchResponseSchema = z.object({
  took: z.number(),
  timed_out: z.boolean(),
  hits: z.object({
    total: z.object({
      value: z.number(),
      relation: z.string(),
    }),
    max_score: z.number().nullable(),
    hits: z.array(HitSchema),
  }),
  aggregations: z.record(z.unknown()).optional(),
});

export default api({
  name: "ElasticsearchExample",
  integrations: {
    es: elasticsearch(PROD_ELASTICSEARCH),
  },
  input: z.object({
    index: z.string(),
    query: z.string(),
    size: z.number().default(10),
  }),
  output: z.object({
    total: z.number(),
    hits: z.array(
      z.object({
        id: z.string(),
        score: z.number().nullable(),
        source: z.record(z.unknown()),
      }),
    ),
  }),
  async run(ctx, { index, query, size }) {
    const result = await ctx.integrations.es.apiRequest(
      {
        method: "POST",
        path: `/${index}/_search`,
        body: {
          query: {
            multi_match: {
              query: query,
              fields: ["title^2", "content", "tags"],
            },
          },
          size: size,
          highlight: {
            fields: {
              content: {},
            },
          },
        },
      },
      { response: SearchResponseSchema },
    );

    return {
      total: result.hits.total.value,
      hits: result.hits.hits.map((hit) => ({
        id: hit._id,
        score: hit._score,
        source: hit._source,
      })),
    };
  },
});
```

### Index a Document

```typescript
const IndexResponseSchema = z.object({
  _index: z.string(),
  _id: z.string(),
  _version: z.number(),
  result: z.string(), // created, updated
  _shards: z.object({
    total: z.number(),
    successful: z.number(),
    failed: z.number(),
  }),
});

// Index with auto-generated ID
const result = await ctx.integrations.es.apiRequest(
  {
    method: "POST",
    path: `/products/_doc`,
    body: {
      name: "Wireless Headphones",
      price: 149.99,
      category: "Electronics",
      tags: ["audio", "wireless", "bluetooth"],
      created_at: new Date().toISOString(),
    },
  },
  { response: IndexResponseSchema },
);

console.log(`Indexed document: ${result._id}`);

// Index with specific ID
const resultWithId = await ctx.integrations.es.apiRequest(
  {
    method: "PUT",
    path: `/products/_doc/product_123`,
    body: {
      name: "Wireless Headphones",
      price: 149.99,
    },
  },
  { response: IndexResponseSchema },
);
```

### Get a Document

```typescript
const GetResponseSchema = z.object({
  _index: z.string(),
  _id: z.string(),
  _version: z.number(),
  found: z.boolean(),
  _source: z.record(z.unknown()),
});

const doc = await ctx.integrations.es.apiRequest(
  {
    method: "GET",
    path: `/products/_doc/${documentId}`,
  },
  { response: GetResponseSchema },
);

if (doc.found) {
  console.log(`Document: ${JSON.stringify(doc._source)}`);
}
```

### Update a Document

```typescript
const UpdateResponseSchema = z.object({
  _index: z.string(),
  _id: z.string(),
  _version: z.number(),
  result: z.string(), // updated, noop
});

// Partial update
const result = await ctx.integrations.es.apiRequest(
  {
    method: "POST",
    path: `/products/_update/${documentId}`,
    body: {
      doc: {
        price: 129.99,
        updated_at: new Date().toISOString(),
      },
    },
  },
  { response: UpdateResponseSchema },
);

// Update with script
const scriptResult = await ctx.integrations.es.apiRequest(
  {
    method: "POST",
    path: `/products/_update/${documentId}`,
    body: {
      script: {
        source: "ctx._source.views += params.count",
        params: { count: 1 },
      },
    },
  },
  { response: UpdateResponseSchema },
);
```

### Delete a Document

```typescript
const DeleteResponseSchema = z.object({
  _index: z.string(),
  _id: z.string(),
  _version: z.number(),
  result: z.string(), // deleted, not_found
});

const result = await ctx.integrations.es.apiRequest(
  {
    method: "DELETE",
    path: `/products/_doc/${documentId}`,
  },
  { response: DeleteResponseSchema },
);
```

### Bulk Operations

```typescript
const BulkResponseSchema = z.object({
  took: z.number(),
  errors: z.boolean(),
  items: z.array(
    z.object({
      index: z
        .object({
          _index: z.string(),
          _id: z.string(),
          status: z.number(),
          result: z.string().optional(),
          error: z
            .object({
              type: z.string(),
              reason: z.string(),
            })
            .optional(),
        })
        .optional(),
      delete: z
        .object({
          _index: z.string(),
          _id: z.string(),
          status: z.number(),
          result: z.string().optional(),
        })
        .optional(),
    }),
  ),
});

// Bulk index
const result = await ctx.integrations.es.apiRequest(
  {
    method: "POST",
    path: "/_bulk",
    headers: {
      "Content-Type": "application/x-ndjson",
    },
    body:
      [
        { index: { _index: "products", _id: "1" } },
        { name: "Product 1", price: 10 },
        { index: { _index: "products", _id: "2" } },
        { name: "Product 2", price: 20 },
      ]
        .map(JSON.stringify)
        .join("\n") + "\n",
  },
  { response: BulkResponseSchema },
);

if (result.errors) {
  result.items.forEach((item) => {
    if (item.index?.error) {
      console.error(`Failed to index: ${item.index.error.reason}`);
    }
  });
}
```

### Aggregations

```typescript
const result = await ctx.integrations.es.apiRequest(
  {
    method: "POST",
    path: `/products/_search`,
    body: {
      size: 0, // Don't return documents, only aggregations
      aggs: {
        categories: {
          terms: {
            field: "category.keyword",
            size: 10,
          },
        },
        avg_price: {
          avg: {
            field: "price",
          },
        },
        price_ranges: {
          range: {
            field: "price",
            ranges: [{ to: 50 }, { from: 50, to: 100 }, { from: 100 }],
          },
        },
      },
    },
  },
  { response: SearchResponseSchema },
);

// Access aggregation results
const categories = result.aggregations?.categories;
const avgPrice = result.aggregations?.avg_price;
```

### Create an Index

```typescript
const CreateIndexResponseSchema = z.object({
  acknowledged: z.boolean(),
  shards_acknowledged: z.boolean(),
  index: z.string(),
});

const result = await ctx.integrations.es.apiRequest(
  {
    method: "PUT",
    path: "/products",
    body: {
      settings: {
        number_of_shards: 1,
        number_of_replicas: 1,
      },
      mappings: {
        properties: {
          name: { type: "text", analyzer: "standard" },
          price: { type: "float" },
          category: { type: "keyword" },
          tags: { type: "keyword" },
          description: { type: "text" },
          created_at: { type: "date" },
        },
      },
    },
  },
  { response: CreateIndexResponseSchema },
);
```

### Check Index Exists

```typescript
try {
  await ctx.integrations.es.apiRequest(
    {
      method: "HEAD",
      path: "/products",
    },
    { response: z.void() },
  );
  console.log("Index exists");
} catch (error) {
  console.log("Index does not exist");
}
```

## Trace Metadata

All methods accept an optional `metadata` parameter as the last argument for diagnostics labeling. See the [root SDK README](../../../README.md#trace-metadata) for details.

## Common Pitfalls

### No Specialized Methods

```typescript
// WRONG - These methods do not exist
await es.search({ ... });
await es.index({ ... });

// CORRECT - Use apiRequest
await ctx.integrations.es.apiRequest(
  { method: "POST", path: "/products/_search", body: { ... } },
  { response: SearchResponseSchema }
);
```

### Index Name in Path

The index name is part of the URL path:

```typescript
// Search in specific index
const path = "/products/_search";

// Search across multiple indices
const path = "/products,orders/_search";

// Search all indices
const path = "/_search";

// Search with wildcards
const path = "/logs-*/_search";
```

### Method Matters

Different HTTP methods have different meanings:

```typescript
// POST with _doc - Auto-generate ID
const path = "/products/_doc";
const method = "POST";

// PUT with _doc/{id} - Specific ID
const path = "/products/_doc/123";
const method = "PUT";

// POST with _search - Search
const path = "/products/_search";
const method = "POST";
```

### Field Types for Filtering

Use `.keyword` suffix for exact matching on text fields:

```typescript
// WRONG - Text field doesn't support term queries
const body = {
  query: {
    term: { category: "Electronics" },
  },
};

// CORRECT - Use keyword sub-field
const body = {
  query: {
    term: { "category.keyword": "Electronics" },
  },
};

// Or use match for text search
const body = {
  query: {
    match: { category: "Electronics" },
  },
};
```

### Boolean Query Structure

Combine queries correctly:

```typescript
const body = {
  query: {
    bool: {
      must: [
        // All conditions must match (AND)
        { match: { title: "headphones" } },
      ],
      should: [
        // At least one should match (OR, affects score)
        { term: { "brand.keyword": "Sony" } },
        { term: { "brand.keyword": "Bose" } },
      ],
      must_not: [
        // Must not match (NOT)
        { term: { "status.keyword": "discontinued" } },
      ],
      filter: [
        // Must match, but doesn't affect score
        { range: { price: { gte: 50, lte: 200 } } },
      ],
      minimum_should_match: 1,
    },
  },
};
```

### Bulk Format

Bulk operations require NDJSON format:

```typescript
// WRONG - Regular JSON array
const body = [{ index: { _index: "products" } }, { name: "Product" }];

// CORRECT - Newline-delimited JSON
const body = '{"index":{"_index":"products"}}\n{"name":"Product"}\n';

// Or build programmatically
const operations = [{ index: { _index: "products" } }, { name: "Product" }];
const body = operations.map(JSON.stringify).join("\n") + "\n";
```

### Total Hits Accuracy

For large result sets, total is approximate by default:

```typescript
// Default behavior (faster but approximate for >10000 hits)
const response = {
  hits: {
    total: { value: 10000, relation: "gte" }, // "gte" means >= 10000
  },
};

// Get accurate count
const body = {
  track_total_hits: true, // Exact count
  // or
  track_total_hits: 100000, // Accurate up to 100000
};
```

### Scroll vs Search After

For large result sets, use proper pagination:

```typescript
// search_after for deep pagination (recommended)
const body = {
  size: 100,
  sort: [{ created_at: "desc" }, { _id: "asc" }],
  search_after: [lastHit.sort[0], lastHit.sort[1]], // From previous response
};

// DON'T use from/size for deep pagination
// This is inefficient for large offsets:
const body = {
  from: 10000, // Bad - loads all 10000 docs in memory
  size: 10,
};
```

### Date Format

Elasticsearch expects ISO 8601 dates:

```typescript
// CORRECT formats
const date = "2024-01-15T10:30:00Z";
const date = "2024-01-15";
const date = 1705315800000; // Milliseconds since epoch

// In range queries
const body = {
  query: {
    range: {
      created_at: {
        gte: "2024-01-01",
        lte: "2024-01-31",
        format: "yyyy-MM-dd", // Optional format specification
      },
    },
  },
};
```

## Error Handling

```typescript
import { RestApiValidationError } from "@superblocksteam/sdk-api";

try {
  const result = await ctx.integrations.es.apiRequest(
    { method: "POST", path: "/products/_search", body: { ... } },
    { response: SearchResponseSchema }
  );
} catch (error) {
  if (error instanceof RestApiValidationError) {
    console.error("Validation failed:", error.details.zodError);
  }
}
```

## API Reference

- [Elasticsearch API Documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/rest-apis.html)
- [Search API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html)
- [Document APIs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs.html)
- [Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
- [Aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html)