# @kyuda/n8n-nodes-databricks

Comprehensive Databricks integration for n8n. Provides nodes for SQL, Unity Catalog, Jobs, Clusters, Files, Genie AI, Vector Search, and Model Serving.

## Features

- **Genie AI Assistant** -- Start conversations, send messages, and execute SQL queries through Databricks' AI assistant
- **Databricks SQL** -- Execute SQL queries with automatic chunked result retrieval
- **Unity Catalog** -- Manage catalogs, schemas, tables, volumes, and functions
- **Model Serving** -- Query deployed ML/LLM endpoints with auto-detected input format
- **Files** -- Upload, download, list, and manage files in Unity Catalog volumes (up to 5 GiB)
- **Vector Search** -- Semantic similarity search with reranking support
- **Jobs** -- Run, create, list, cancel, and monitor Databricks job runs
- **Clusters** -- List, create, start, stop, and delete Databricks compute clusters

## Prerequisites

- Node.js 18+
- pnpm
- A Databricks workspace with credentials (PAT or OAuth Service Principal)

## Installation

### For Development

```bash
git clone https://github.com/kyudahq/n8n.git
cd n8n/@kyuda/n8n-nodes-databricks
pnpm install
pnpm build
```

Link to your n8n installation:

```bash
npm link
cd ~/.n8n/custom
npm link @kyuda/n8n-nodes-databricks
```

### From npm

```bash
npm install @kyuda/n8n-nodes-databricks
```

## Credentials

Two authentication methods are supported:

### Personal Access Token (default)

1. **Host** -- Your Databricks workspace URL (e.g. `https://adb-1234567890123456.7.azuredatabricks.net`)
2. **Token** -- A Databricks personal access token (generate at User Settings > Access Tokens)

### OAuth M2M (Service Principal)

1. **Host** -- Your Databricks workspace URL
2. **Client ID** -- The application (client) ID of a Databricks service principal
3. **Client Secret** -- The secret associated with the service principal

The OAuth flow uses `grant_type=client_credentials` against your workspace's `/oidc/v1/token` endpoint. Tokens are cached and refreshed automatically.

## Resources and Operations

### Databricks (main node)

| Resource | Operations |
|---|---|
| **Genie** | Start Conversation, Create Message, Get Message, Execute Message Query, Get Query Results, Get Space |
| **Databricks SQL** | Execute Query |
| **Unity Catalog** | Create/Get/Update/Delete Catalog, List Catalogs, Create/Get/Update/Delete Schema, List Schemas, Create/Get/Delete Volume, List Volumes, Get/List Tables, Create/Get/Delete Function, List Functions |
| **Model Serving** | Query Endpoint (auto-detects chat/completions/embeddings/dataframe formats) |
| **Files** | Upload File, Download File, Delete File, Get File Info, List Directory, Create Directory, Delete Directory |
| **Vector Search** | Query Index, Get Index, List Indexes |
| **Jobs** | Run Job, Get Run Status, Get Run Output, Create Job, List Jobs, Cancel Run, List Runs |
| **Clusters** | List Clusters, Get Cluster, Create Cluster, Start Cluster, Stop Cluster, Delete Cluster |

## Architecture

```
@kyuda/n8n-nodes-databricks/
├── credentials/
│   └── Databricks.credentials.ts      # PAT + OAuth M2M
└── nodes/
    └── Databricks/
        ├── Databricks.node.ts          # Main node (slim dispatcher)
        ├── helpers.ts                  # Shared HTTP helper, cache, security utils
        └── resources/
            ├── index.ts                # Barrel exports
            ├── clusters/              # operations, parameters, handler
            ├── databricksSql/          # operations, parameters, handler
            ├── files/                  # operations, parameters, handler
            ├── genie/                  # operations, parameters, handler
            ├── jobs/                   # operations, parameters, handler
            ├── modelServing/           # operations, parameters, handler
            ├── unityCatalog/           # operations, parameters, handler
            └── vectorSearch/           # operations, parameters, handler
```

### Execution Flow

```
User selects Resource → Operation → fills Parameters
        ↓
    execute() dispatcher (Databricks.node.ts)
        ↓
    handlers[resource](ctx, operation, itemIndex)
        ↓
    Per-resource handler file (e.g. resources/jobs/handler.ts)
        ↓
    databricksApiRequest() (shared helper with auth, host normalization)
        ↓
    Databricks REST API
```

### Security

- **Token masking** -- Bearer tokens and PAT values are scrubbed from all error messages and logs
- **URL encoding** -- All user-provided path segments are passed through `encodeURIComponent`
- **Path traversal protection** -- File operations reject paths containing `..`
- **Cache isolation** -- Cache keys include a hash of the credential token, preventing cross-user cache leaks

## Development

```bash
pnpm build      # Build the node
pnpm test       # Run tests
pnpm lint       # Lint
pnpm lintfix    # Auto-fix lint issues
```

## Contributing

Contributions are welcome. Please submit a Pull Request at [github.com/kyudahq/n8n](https://github.com/kyudahq/n8n).

## Resources

- [n8n Documentation](https://docs.n8n.io/)
- [Databricks API Documentation](https://docs.databricks.com/api/)
- [n8n Community](https://community.n8n.io/)

## License

[MIT](LICENSE.md)