# Snowflake Streaming Ingest Node.js SDK

A high-performance Node.js SDK for streaming data ingestion into Snowflake, built with Rust for optimal performance and reliability.

- [Overview](#overview)
- [Supported Platforms](#supported-platforms)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Features](#features)
- [Architecture](#architecture)
- [Dependencies](#dependencies)
- [Support](#support)
- [License](#license)

## Overview

The Snowflake Streaming Ingest Node.js SDK provides a Node.js interface for real-time data streaming into Snowflake tables. It leverages a Rust core for high performance while providing a familiar JavaScript API for easy integration.

## Supported Platforms

| OS | Architecture | C Library | Notes |
|----|-------------|-----------|-------|
| Linux | x64 | glibc (>= 2.26) | |
| Linux | x64 | musl | Alpine Linux, etc. |
| Linux | arm64 | glibc (>= 2.26) | |
| Linux | arm64 | musl | Alpine Linux, etc. |
| macOS | arm64 | — | Apple Silicon, macOS 11.0+ |
| Windows | x64 | MSVC | Windows 10+ |

### Node.js
- **Node.js 20** or later

## Installation

```bash
npm install snowpipe-streaming
```

The correct native binary for your platform is installed automatically via `optionalDependencies`.

## Quick Start

```javascript
const { createClient } = require("snowpipe-streaming");

// Create a client
const client = await createClient({
  clientName: "my-client",
  dbName: "MY_DB",
  schemaName: "MY_SCHEMA",
  pipeName: "MY_PIPE",
  properties: {
    account: "your_account",
    user: "your_user",
    private_key: "-----BEGIN PRIVATE KEY-----\n...",
  },
});

// Open a channel
const { channel } = await client.openChannel({ name: "my_channel" });

// Insert data
await channel.insertRows([
  { id: 1, name: "John Doe", timestamp: "2024-01-01T00:00:00Z" },
  { id: 2, name: "Jane Smith", timestamp: "2024-01-02T00:00:00Z" },
]);

// Wait for data to be committed to Snowflake
await channel.waitForCommit();

// Close resources
await channel.close();
await client.close();
```

## Features

- **High Performance**: Rust-based core for optimal throughput and low latency
- **Memory Efficient**: Minimal memory footprint with efficient data handling
- **Automatic Retries**: Built-in retry logic for transient failures
- **Backpressure Handling**: Intelligent backpressure management
- **Comprehensive Logging**: Detailed logging for debugging and monitoring
- **Cross-Platform**: Native support for Linux, macOS, and Windows
- **Zero Dependencies**: No JavaScript runtime dependencies

## Architecture

The SDK uses a hybrid Rust-Node.js architecture:
- **Rust Core**: High-performance data processing, networking, and Snowflake communication
- **Node.js Bindings**: JavaScript API using napi-rs for seamless integration
- **Platform Packages**: Architecture-specific native binaries distributed as npm optional dependencies

## Dependencies

- **None**: The SDK has zero JavaScript runtime dependencies
- **Native Extensions**: Rust-based binary modules (automatically installed for your platform)

## Support

- **Documentation**: [Snowpipe Streaming Overview](https://docs.snowflake.com/en/user-guide/snowpipe-streaming/data-load-snowpipe-streaming-overview)

## License

This project is licensed under the Apache License 2.0.
