---
name: neon-postgres
description: Guides and best practices for working with Neon Serverless Postgres. Covers setup, connection methods, branching, autoscaling, scale-to-zero, read replicas, connection pooling, Neon Auth, and the Neon CLI, MCP server, REST API, TypeScript SDK, and Python SDK. Use when users ask about...
risk: critical
source: https://github.com/neondatabase/agent-skills/tree/main/skills/neon-postgres
source_repo: neondatabase/agent-skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/neondatabase/agent-skills/blob/main/LICENSE
---

# Neon Serverless Postgres
## When to Use

Use this skill when you need guides and best practices for working with Neon Serverless Postgres. Covers setup, connection methods, branching, autoscaling, scale-to-zero, read replicas, connection pooling, Neon Auth, and the Neon CLI, MCP server, REST API, TypeScript SDK, and Python SDK. Use when users ask about...

Guide the user through any Neon-related task: setup, connections, branching, and advanced features. Deliver a working Neon connection, a completed feature configuration, or a specific answer from the official Neon docs.

Neon is a serverless Postgres platform that separates compute and storage to offer autoscaling, branching, instant restore, and scale-to-zero. It's fully compatible with Postgres and works with any language, framework, or ORM that supports Postgres.

## Neon Documentation

The Neon documentation is the source of truth for all Neon-related information. Always verify claims against the official docs before responding. Neon features and APIs evolve, so prefer fetching current docs over relying on training data.

### Fetching Docs as Markdown

Any Neon doc page can be fetched as markdown in two ways:

1. **Append `.md` to the URL** (simplest): https://neon.com/docs/introduction/branching.md
2. **Request `text/markdown`** on the standard URL: `curl -H "Accept: text/markdown" https://neon.com/docs/introduction/branching`

Both return the same markdown content. Use whichever method your tools support.

### Finding the Right Page

The docs index lists every available page with its URL and a short description:

```
https://neon.com/docs/llms.txt
```

Common doc URLs are organized in the topic links below. If you need a page not listed here, search the docs index: https://neon.com/docs/llms.txt. Don't guess URLs.

## What Is Neon

Use this for architecture explanations and terminology (organizations, projects, branches, endpoints) before giving implementation advice.

Link: https://neon.com/docs/introduction/architecture-overview.md

## Getting Started

Use this section when guiding a user through first-time Neon setup.

### Check Status Quo

Before starting setup, inspect the user's codebase and environment:

- Existing database connection code
- Existing Neon MCP server or Neon CLI configuration
- Existence of a `.env` file and `DATABASE_URL` environment variable
- Existing ORM (Prisma, Drizzle, TypeORM) configuration

### Self-Driving Setup With Neon's CLI or MCP Server

Offer to inspect existing connected Neon projects or create new ones using the Neon CLI or MCP server. If neither is set up yet, run init with the `--agent` flag. Use `npx -y` to skip the package install prompt. Auth is handled automatically. If the user is not logged in, it opens their browser for OAuth and waits for completion before proceeding.

```bash
npx -y neon@latest init --agent <agent-name>
```

Supported `--agent` values: `cursor`, `copilot`, `claude`, `claude-desktop`, `codex`, `opencode`, `cline`, `gemini-cli`, `goose`, `zed`.

This installs the Neon extension (for Cursor/VS Code) or MCP server (for other agents), creates an API key, and adds the `neon-postgres` agent skill to the project.

If `init` is not suitable, the individual steps can be run non-interactively:

- **Extension:** `cursor --install-extension databricks.neon-local-connect`
- **MCP server:** `npx -y add-mcp https://mcp.neon.tech/mcp -g -n Neon -y -a <agent-name>`
- **Agent skill:** `npx skills add neondatabase/agent-skills --skill neon-postgres --agent <agent-name> -y`

For full CLI installation options, see https://neon.com/docs/reference/cli-install.md

### Setup Flow

**1. Select Organization and Project**

Use MCP server or CLI to list organizations and projects. Let the user select an existing project or create a new one.

**2. Get Connection String**

Use MCP server or CLI to get the connection string. Store it in `.env` as `DATABASE_URL`. Read the file first before modifying to avoid overwriting existing values.

**3. Pick Connection Method & Driver**

Refer to the connection methods guide to pick the correct driver based on deployment platform: https://neon.com/docs/connect/choose-connection.md

**4. User Authentication with Neon Auth (if needed)**

Skip for CLI tools, scripts, or apps without user accounts. If the app needs auth: use MCP server `provision_neon_auth` tool, then see the auth overview (https://neon.com/docs/auth/overview.md) for setup. For auth + database queries, see the JavaScript SDK reference (https://neon.com/docs/reference/javascript-sdk.md).

**5. ORM Setup (optional)**

Check for existing ORM (Prisma, Drizzle, TypeORM). If none, ask if they want one. For Drizzle integration, see https://neon.com/docs/guides/drizzle.md.

**6. Schema Setup**

- Check for existing migration files or ORM schemas
- If none: offer to create an example schema or design one together

### Resume Support

If resuming setup, check what's already configured (MCP connection, `.env` with `DATABASE_URL`, dependencies, schema) and continue from the next incomplete step.

### Security Reminders

Remind users to use environment variables for credentials, never commit connection strings, and use least-privilege database roles.

## Connection Methods & Drivers

Use this when you need to pick the correct transport and driver based on runtime constraints (TCP, HTTP, WebSocket, edge, serverless, long-running).

Link: https://neon.com/docs/connect/choose-connection.md

### Recommended: Drizzle + the right driver for your runtime

Always pair Neon with an ORM such as **Drizzle** for easy schema management and migrations. Pick the driver based on how the runtime treats your code:

- **Long-running or shared-runtime environments → node-postgres (`pg`).** Neon Functions, and any host where the function runtime is shared across requests / runs on fluid compute (e.g. **Vercel** with Fluid compute), keep a module-scope process alive across many requests. Open a `pg` pool **once at module scope** and reuse it across requests.
- **Fully isolated serverless (Lambda-style) → Neon's serverless driver (`@neondatabase/serverless`).** Hosts like **Netlify** spin up a fresh, isolated instance per request, so a persistent TCP pool can't be reused; the serverless driver queries over HTTP and is built for this.

**Neon Functions / Vercel / fluid compute — Drizzle + node-postgres:**

```typescript
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import * as schema from "./schema";

// Created once at module scope; reused by every request the instance handles.
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });
const db = drizzle({ client: pool, schema });
```

On **Vercel** (Fluid compute) also attach the pool with `attachDatabasePool` from `@vercel/functions`, so the function runtime drains idle connections before an instance suspends:

```typescript
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import { attachDatabasePool } from "@vercel/functions";
import * as schema from "./schema";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
attachDatabasePool(pool); // let the Vercel runtime manage the pooled connections
const db = drizzle({ client: pool, schema });
```

**Netlify and other fully-isolated serverless — Drizzle + Neon serverless driver:**

```typescript
import { drizzle } from "drizzle-orm/neon-http";
import { neon } from "@neondatabase/serverless";

const sql = neon(process.env.DATABASE_URL!);
const db = drizzle({ client: sql });
```

### Serverless Driver

Use this for `@neondatabase/serverless` patterns, including HTTP queries, WebSocket transactions, and runtime-specific optimizations.

Link: https://neon.com/docs/serverless/serverless-driver.md

### Neon JS SDK

Use this for combined Neon Auth + Data API workflows with PostgREST-style querying and typed client setup.

Link: https://neon.com/docs/reference/javascript-sdk.md

## Developer Tools

Use this for local development enablement with `npx -y neon@latest init --agent <agent-name>`, VSCode extension setup, and Neon MCP server configuration.

| Tool             | URL                                             |
| ---------------- | ----------------------------------------------- |
| CLI Init Command | https://neon.com/docs/reference/cli-init.md     |
| VSCode Extension | https://neon.com/docs/local/vscode-extension.md |
| MCP Server       | https://neon.com/docs/ai/neon-mcp-server.md     |
| Neon CLI         | https://neon.com/docs/reference/neon-cli.md     |

### Neon CLI

Use this for terminal-first workflows, scripts, and CI/CD automation with `neon`.

Link: https://neon.com/docs/reference/neon-cli.md

## Neon Admin API

The Neon Admin API can be used to manage Neon resources programmatically. It is used behind the scenes by the Neon CLI and MCP server, but can also be used directly for more complex automation workflows or when embedding Neon in other applications.

### Neon REST API

Use this for direct HTTP automation, endpoint-level control, API key auth, rate-limit handling, and operation polling.

Link: https://neon.com/docs/reference/api-reference.md

### Neon TypeScript SDK

Use this when implementing typed programmatic control of Neon resources in TypeScript via `@neondatabase/api-client`.

Link: https://neon.com/docs/reference/typescript-sdk.md

### Neon Python SDK

Use this when implementing programmatic Neon management in Python with the `neon-api` package.

Link: https://neon.com/docs/reference/python-sdk.md

## Neon Auth

Use this for managed user authentication setup, UI components, auth methods, and Neon Auth integration pitfalls in Next.js and React apps.

Link: https://neon.com/docs/auth/overview.md

Neon Auth is also embedded in the Neon JS SDK. Depending on your use case, you may want to use the Neon JS SDK instead of Neon Auth alone. See https://neon.com/docs/connect/choose-connection.md for more details.

## Neon Infrastructure as Code (`neon.ts`)

`neon.ts` is Neon's branch config and infrastructure-as-code file: declare which services your branches have, get type-safe env vars, and program per-branch compute — all in TypeScript (see the `neon` skill for the full reference). Postgres always exists on every branch, so you never declare the database itself; what you codify here is the Postgres-adjacent surface — Neon Auth, the Data API, and per-branch compute settings (autoscaling and scale-to-zero).

Add it with `@neon/config`:

```bash
npm i @neon/config
```

```typescript
// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  auth: true, // Neon Auth (adds NEON_AUTH_* env vars)
  dataApi: true, // Data API (adds NEON_DATA_API_URL); requires auth: true (or an external IdP)
  // Postgres exists on every branch; tune its compute per branch:
  branch: (branch) => {
    if (branch.exists) return {}; // leave existing branches untouched
    if (branch.isDefault) return { protected: true }; // prod keeps default compute
    return {
      ttl: "7d", // non-prod branches auto-expire (max 30d)
      postgres: {
        computeSettings: {
          autoscalingLimitMinCu: 0.25, // scale to zero
          autoscalingLimitMaxCu: 1, // keep dev/preview cheap
          suspendTimeout: "5m",
        },
      },
    };
  },
});
```

Reconcile the declaration from the CLI — the Neon equivalent of `terraform plan` / `apply`:

```bash
neon config status   # print the branch's live config
neon config plan     # dry-run diff of what apply would change
neon config apply    # provision the declared services / settings
neon deploy          # alias for `neon config apply`
```

Because `neon checkout` applies the policy as it **creates** a branch, a fresh branch comes up with these compute settings (and Auth / Data API) already in place. Checking out an _existing_ branch never reconciles it — run `neon deploy` to apply changes.

Since `neon.ts` is TypeScript, invalid combinations fail to compile with an actionable message: the Data API verifies requests with Neon Auth by default, so `dataApi: true` without `auth: true` is a type error (the fix — `auth: true`, or `authProvider: 'external'` with a `jwksUrl` — is in the message). See the `neon` skill's type-safe config note.

Read the resulting env back, typed and validated against the policy, with `parseEnv` from `@neon/env`:

```typescript
import { parseEnv } from "@neon/env";
import config from "./neon";

const env = parseEnv(config);
env.postgres.databaseUrl; // typed; enabling auth / dataApi above surfaces env.auth / env.dataApi
```

## Branching

Use this when the user is planning isolated environments, schema migration testing, preview deployments, or branch lifecycle automation.

Key points:

- Branches are instant, copy-on-write clones (no full data copy).
- Each branch has its own compute endpoint.
- Use the neon CLI or MCP server to create, inspect, and compare branches.

Link: https://neon.com/docs/introduction/branching.md

For detailed branch creation workflows (normal vs schema-only branches, reset-from-parent, CLI/MCP selection), use the `neon-postgres-branches` skill if available

Or fetch the full branching skill from the following URL:

https://neon.com/docs/ai/skills/neon-postgres-branches/SKILL.md

If this skill is not installed you can use the following command to install it:

```bash
npx skills add neondatabase/agent-skills --skill neon-postgres-branches
```

## Autoscaling

Use this when the user needs compute to scale automatically with workload and wants guidance on CU sizing and runtime behavior.

Link: https://neon.com/docs/introduction/autoscaling.md

## Scale to Zero

Use this when optimizing idle costs and discussing suspend/resume behavior, including cold-start trade-offs.

Key points:

- Idle computes suspend automatically (default 5 minutes, configurable) (unless disabled - launch & scale plan only)
- First query after suspend typically has a cold-start penalty (around hundreds of ms)
- Storage remains active while compute is suspended.

Link: https://neon.com/docs/introduction/scale-to-zero.md

## Instant Restore

Use this when the user needs point-in-time recovery or wants to restore data state without traditional backup restore workflows.

Key points:

- History windows for instant restore depend on plan limits.
- Users can create branches from historical points-in-time.
- Time Travel queries can be used for historical inspection workflows.

Link: https://neon.com/docs/introduction/branch-restore.md

## Read Replicas

Use this for read-heavy workloads where the user needs dedicated read-only compute without duplicating storage.

Key points:

- Replicas are read-only compute endpoints sharing the same storage.
- Creation is fast and scaling is independent from primary compute.
- Typical use cases: analytics, reporting, and read-heavy APIs.

Link: https://neon.com/docs/introduction/read-replicas.md

## Connection Pooling

Use this when the user is in serverless or high-concurrency environments and needs safe, scalable Postgres connection management.

Key points:

- Neon pooling uses PgBouncer.
- Add `-pooler` to endpoint hostnames to use pooled connections.
- Pooling is especially important in serverless runtimes with bursty concurrency.

Link: https://neon.com/docs/connect/connection-pooling.md

## IP Allow Lists

Use this when the user needs to restrict database access by trusted networks, IPs, or CIDR ranges.

Link: https://neon.com/docs/introduction/ip-allow.md

## Logical Replication

Use this when integrating CDC pipelines, external Postgres sync, or replication-based data movement.

Key points:

- Neon supports native logical replication workflows.
- Useful for replicating to/from external Postgres systems.

Link: https://neon.com/docs/guides/logical-replication-guide.md

## Limitations

- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.


---
name: neon-postgres-branches
description: Choose and create the right Neon branch type for testing and development. Use when users ask about Neon branching, migration testing with real data, isolated test environments, schema-only branch workflows for sensitive data, or branch creation via Neon CLI or Neon MCP. Triggers...
risk: critical
source: https://github.com/neondatabase/agent-skills/tree/main/skills/neon-postgres-branches
source_repo: neondatabase/agent-skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/neondatabase/agent-skills/blob/main/LICENSE
---

# Neon Postgres Branching
## When to Use

Use this skill when you need choose and create the right Neon branch type for testing and development. Use when users ask about Neon branching, migration testing with real data, isolated test environments, schema-only branch workflows for sensitive data, or branch creation via Neon CLI or Neon MCP. Triggers...

The outcome of this skill should be a created Neon branch (or a clear, actionable next step if creation cannot proceed).
Choose the correct branch type, then execute branch creation via MCP or CLI.

- **Normal branch** for realistic migration and query testing with real data.
- **Schema-only branch (Beta)** for sensitive data workflows where structure is needed without copying rows.

## Branch Type Decision

Use this decision rule first:

1. If the user wants to test complex migrations, performance, or behavior against production-like data, choose a **normal branch**.
2. If the user needs to avoid copying sensitive data, choose a **schema-only branch**.

If the request is ambiguous, ask one clarifying question:
"Do you need realistic data for testing, or only schema structure because the data is sensitive?"

## Tool Selection: CLI or MCP

Always support both Neon CLI and Neon MCP server. Prefer the tool the user already has installed and authenticated.

MCP link: https://neon.com/docs/ai/neon-mcp-server.md
CLI link: https://neon.com/docs/reference/cli-quickstart

### Selection order

1. Check MCP first in MCP-enabled environments:
   - If Neon MCP tools are available and authenticated (for example, listing projects works), use MCP.
2. If MCP is unavailable or not authenticated, check CLI:
   - Run `neon --version` to confirm CLI is installed.
   - Run `neon projects list` to confirm auth/context.
3. If CLI is missing, direct installation via quickstart.
4. If CLI is installed but not authenticated, guide the user through `neon auth` (or API key auth), then continue.
5. If both MCP and CLI paths are unsuccessful, use the Neon REST API:
   - https://neon.com/docs/guides/branching-neon-api.md

### MCP branch flow

1. Choose normal vs schema-only based on data sensitivity and migration-testing goals.
2. Use branch tools (for example, `create_branch`) to create the branch.
3. Validate with read tools (for example, `describe_branch`).
4. For migration workflows, prefer branch-based migration flows before applying to main.

## Create a Normal Branch (Preferred for Real-Data Migration Testing)

Use this when the user needs realistic testing conditions.
Real production-like data can expose edge cases your seed or data migration scripts miss, which helps catch migration issues before going live.

Link: https://neon.com/docs/introduction/branching.md

### Steps

1. Use MCP if already available/authenticated; otherwise verify CLI with `neon --version`.
2. Ensure project context is set (`neon set-context --project-id <your-project-id>`) or include `--project-id` on commands.
3. Create branch:

```bash
neon branches create \
  --name <branch-name> \
  --parent <parent-branch-id-or-name> \
  --expires-at 2026-12-15T18:02:16Z
```

4. Optionally fetch a connection string for the new branch:

```bash
neon connection-string <branch-name>
```

## Create a Schema-Only Branch (Beta, Sensitive Data)

Use this when users must not copy production rows into the test branch.

Link: https://neon.com/docs/guides/branching-schema-only.md

### Steps

1. Use MCP if already available/authenticated; otherwise verify CLI with `neon --version`.
2. Create schema-only branch:

```bash
neon branches create \
  --name <schema-only-branch-name> \
  --parent <parent-branch-id-or-name> \
  --schema-only \
  --expires-at 2026-12-15T18:02:16Z
```

If multiple projects exist, include:

```bash
neon branches create \
  --name <schema-only-branch-name> \
  --parent <parent-branch-id-or-name> \
  --schema-only \
  --project-id <your-project-id> \
  --expires-at 2026-12-15T18:02:16Z
```

### Beta Support Guidance (Mandatory)

Schema-only branching is in Beta. If users report unexpected behavior, errors, or missing capabilities:

1. Ask them to share feedback in the Neon Console:
   - https://console.neon.tech/app/projects?modal=feedback
2. Recommend opening a support conversation in the Neon Discord:
   - https://discord.gg/92vNTzKDGp

## Reset from parent

Use this when a child branch has drifted and the user wants a clean refresh from the parent branch's latest schema and data.

Link: https://neon.com/docs/guides/reset-from-parent.md

### What it does

- Fully replaces the child branch schema and data with the parent's latest state.
- Does not merge; local changes on the child branch are lost.
- Keeps the same connection details, but active connections are briefly interrupted during reset.

### When to recommend it

- Development or staging branch is too far behind production.
- User wants to start a new feature from a clean parent-aligned state.
- Team wants to refresh staging from production for consistent testing baselines.

### Hard constraints and blockers

- Only child branches can be reset (root branches and schema-only root branches cannot be reset from parent).
- If the target branch has children, reset is blocked until those child branches are removed.
- After a parent branch is restored from snapshot, reset-from-parent may be unavailable for up to 24 hours.
- Reset-from-parent always uses the current parent state; use Instant restore for point-in-time recovery needs.

### CLI usage

```bash
neon branches reset <id|name> --parent --preserve-under-name <backup-branch-name>
```

If project context is not already set, include project ID:

```bash
neon branches reset <id|name> --parent --preserve-under-name <backup-branch-name> --project-id <project-id>
```

`--preserve-under-name` keeps the pre-reset state as a backup branch for rollback, but adds one extra branch to clean up later.

Optional context setup to avoid repeating `--project-id`:

```bash
neon set-context --project-id <project-id>
```

### Console and API usage

- **Console:** Open the target child branch, then select **Reset from parent** from **Actions**.
- **API:** Use the restore endpoint for the branch and set `source_branch_id` to the parent branch ID.

## Notes and Caveats

- Schema-only branches are for structure-only cloning and sensitive/compliant data controls.
- Schema-only branches are independent root branches (no parent branch and no shared history), so reset-from-parent does not apply.
- For migration testing that depends on real-world row shapes, volumes, and edge cases, prefer normal branches.
- Root branch allowances and per-branch storage limits can cap how many schema-only branches users can create.
- If a user is unsure, default recommendation is:
  - **Normal branch** for migration validation.
  - **Schema-only branch** for compliance and privacy constraints.

## Useful Workflow Patterns

If the user asks for process recommendations (not just a single command), suggest these:

- **One branch per PR:** Create branch when PR opens, delete when merged/closed, keep migration tests isolated.
- **One branch per test run:** Create branch at pipeline start, run migrations/tests, delete at end for deterministic CI.
- **One branch per developer:** Isolated dev environments with production-like shape; avoid team collisions on shared test data.
- **PII-aware branching:** If production has sensitive data, derive dev/PR branches from an anonymized branch or use schema-only branches.
- **Ephemeral lifecycle hygiene:** Set branch expiration and automate cleanup so old branches do not accumulate avoidable storage/history cost.

### Post-creation environment update prompt

After branch creation, ask whether the user wants to update local environment credentials to point at the new branch.

- Ask: "Do you want me to update your `.env` `DATABASE_URL` to this new branch connection string?"
- If yes, write the new branch connection string to the requested env file/key.
- If no, leave credentials unchanged and share the connection string for manual use.
- Never overwrite an existing env key without explicit confirmation.

## Neon Infrastructure as Code (`neon.ts`)

Beyond creating branches imperatively (CLI / MCP / API above), you can **program what configuration new branches receive** declaratively in `neon.ts` — Neon's infrastructure-as-code file (see the `neon` skill for the full reference). The `branch` property is a function of the branch being evaluated that returns its settings, so every branch born from your project gets a consistent lifecycle and compute profile without per-branch flags.

```bash
npm i @neon/config
```

```typescript
// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  branch: (branch) => {
    if (branch.exists) return {}; // never reconcile existing branches
    if (branch.isDefault) return { protected: true };
    if (branch.name.startsWith("preview/") || branch.name.startsWith("dev")) {
      return {
        parent: "main",
        ttl: "7d", // ephemeral: auto-expire 7 days after creation (max 30d)
        postgres: {
          computeSettings: {
            autoscalingLimitMinCu: 0.25, // scale to zero
            autoscalingLimitMaxCu: 1, // keep throwaway branches cheap
            suspendTimeout: "5m",
          },
        },
      };
    }
    return {};
  },
});
```

The closure receives a read-only descriptor of the target branch — `name`, `exists`, `isDefault`, `parentId`, and more — and returns the tuning to apply: `parent`, `ttl` (auto-expiry), `protected`, and `postgres.computeSettings`. This is the declarative complement to the **Ephemeral lifecycle hygiene** and per-PR / per-test patterns above: instead of remembering `--expires-at` on every `neon branches create`, the TTL and compute profile live in version control and apply to every matching branch.

Because `neon checkout` applies this policy when it **creates** a branch, a fresh `preview/*` or `dev-*` branch comes up already expiring and scaled-to-zero. Checking out an _existing_ branch doesn't reconcile it — run `neon deploy` (alias for `neon config apply`) to apply changes to a branch that already exists.

## Branching in CI/CD

Common CI/CD use cases for Neon branches:

- **Per-PR preview deployments:** Branch on PR open, deploy the preview against it, delete on close. Each PR gets an isolated database branch. Injecting the branch's `DATABASE_URL` into the deployed app is hosting-provider-specific — see [preview-branches-with-cloudflare](https://github.com/neondatabase/preview-branches-with-cloudflare), [preview-branches-with-vercel](https://github.com/neondatabase/preview-branches-with-vercel), or [preview-branches-with-fly](https://github.com/neondatabase/preview-branches-with-fly) for tested patterns.
- **Migration testing in CI:** Run risky schema changes against a branch with production-like data before merge.
- **Schema diff visibility:** Use the [schema-diff GitHub Action](https://github.com/marketplace/actions/neon-schema-diff-github-action) to auto-comment a DB-layer diff on the PR.

## Examples

### Example 1: Migration testing with realistic data

**User input:** "I need to test a risky migration against production-like data."

**Agent output shape:**

1. Recommend a normal branch and explain why.
2. Share docs link: https://neon.com/docs/introduction/branching
3. Check the available/authenticated tool path first (MCP, otherwise CLI with `neon --version`).
4. Provide commands:
   - `neon branches create --name migration-test --parent main --expires-at 2026-12-15T18:02:16Z`
   - `neon connection-string migration-test`

### Example 2: Sensitive data development workflow

**User input:** "We cannot copy production data because of compliance."

**Agent output shape:**

1. Recommend schema-only branch and explain why.
2. Share docs link: https://neon.com/docs/guides/branching-schema-only
3. Check the available/authenticated tool path first (MCP, otherwise CLI with `neon --version`).
4. Provide command:
   - `neon branches create --name compliance-dev --parent main --schema-only --project-id <your-project-id> --expires-at 2026-12-15T18:02:16Z`
5. Mention Beta support path:
   - https://console.neon.tech/app/projects?modal=feedback
   - https://discord.gg/92vNTzKDGp

## Further reading

- https://neon.com/docs/guides/branch-expiration.md
- https://neon.com/docs/guides/neon-github-integration.md
- https://neon.com/docs/ai/neon-mcp-server.md
- https://neon.com/branching

## Limitations

- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.


---
name: neon-postgres-egress-optimizer
description: Diagnose and fix excessive Postgres egress (network data transfer) in a codebase. Use when a user mentions high database bills, unexpected data transfer costs, network transfer charges, egress spikes, "why is my Neon bill so high", "database costs jumped", SELECT * optimization, query...
risk: critical
source: https://github.com/neondatabase/agent-skills/tree/main/skills/neon-postgres-egress-optimizer
source_repo: neondatabase/agent-skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/neondatabase/agent-skills/blob/main/LICENSE
---

# Postgres Egress Optimizer
## When to Use

Use this skill when you need diagnose and fix excessive Postgres egress (network data transfer) in a codebase. Use when a user mentions high database bills, unexpected data transfer costs, network transfer charges, egress spikes, "why is my Neon bill so high", "database costs jumped", SELECT * optimization, query...

Guide the user through diagnosing and fixing application-side query patterns that cause excessive data transfer (egress) from their Postgres database. Most high egress bills come from the application fetching more data than it uses.

## Step 1: Diagnose

Identify which queries transfer the most data. The primary tool is the `pg_stat_statements` extension.

### Check if pg_stat_statements is available

```sql
SELECT 1 FROM pg_stat_statements LIMIT 1;
```

If this errors, the extension needs to be created:

```sql
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
```

On Neon, it is available by default but may need this CREATE EXTENSION step.

### Handle empty stats

Stats are cleared when a Neon compute scales to zero and restarts. If the stats are empty or the compute recently woke up:

1. Reset the stats to start a clean measurement window: `SELECT pg_stat_statements_reset();`
2. Let the application run under representative traffic for at least an hour.
3. Return and run the diagnostic queries below.

If the user has stats from a production database, use those. If they have no access to production stats, proceed to Step 2 and analyze the codebase directly — code-level patterns are often sufficient to identify the worst offenders.

### Diagnostic queries

Run these to identify the top egress contributors. Focus on queries that return many rows, return wide rows (JSONB, TEXT, BYTEA columns), or are called very frequently.

**Queries returning the most total rows:**

```sql
SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY rows DESC
LIMIT 10;
```

**Queries returning the most rows per execution** (poorly scoped SELECTs, missing pagination):

```sql
SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY avg_rows_per_call DESC
LIMIT 10;
```

**Most frequently called queries** (candidates for caching):

```sql
SELECT query, calls, rows AS total_rows, rows / calls AS avg_rows_per_call
FROM pg_stat_statements
WHERE calls > 0
ORDER BY calls DESC
LIMIT 10;
```

**Longest running queries** (not a direct egress measure, but helps identify problem queries during a spike):

```sql
SELECT query, calls, rows AS total_rows,
  round(total_exec_time::numeric, 2) AS total_exec_time_ms
FROM pg_stat_statements
WHERE calls > 0
ORDER BY total_exec_time DESC
LIMIT 10;
```

### Interpret the results

Rank findings by estimated egress impact:

- **High row count + wide rows** = biggest egress. A query returning 1,000 rows where each row includes a 50KB JSONB column transfers ~50MB per call.
- **Extreme call frequency** on even small queries adds up. A query called 50,000 times/day returning 10 rows each = 500,000 rows/day.
- **Cross-reference with the schema** to identify which columns are wide. Look for JSONB, TEXT, BYTEA, and large VARCHAR columns.

## Step 2: Analyze codebase

For each query identified in Step 1, or for each database query in the codebase if no stats are available, check:

- Does it select only the columns the response needs?
- Does it return a bounded number of rows (LIMIT/pagination)?
- Is it called frequently enough to benefit from caching?
- Does it fetch raw data that gets aggregated in application code?
- Does it use a JOIN that duplicates parent data across child rows?

## Step 3: Fix

Apply the appropriate fix for each problem found. Below are the most common egress anti-patterns and how to fix them.

### Unused columns (SELECT \*)

**Problem:** The query fetches all columns but the application only uses a few. Large columns (JSONB blobs, TEXT fields) get transferred over the wire and discarded.

**Before:**

```sql
SELECT * FROM products;
```

**After:**

```sql
SELECT id, name, price, image_urls FROM products;
```

### Missing pagination

**Problem:** A list endpoint returns all rows with no LIMIT. This is an unbounded egress risk — every new row in the table increases data transfer on every request. Flag this regardless of current table size.

This is easy to miss because the application may work fine with small datasets. But at scale, an unpaginated endpoint returning 10,000 rows with even moderate column widths can transfer hundreds of megabytes per day.

**Before:**

```sql
SELECT id, name, price FROM products;
```

**After:**

```sql
SELECT id, name, price FROM products
ORDER BY id
LIMIT 50 OFFSET 0;
```

When adding pagination, check whether the consuming client already supports paginated responses. If not, pick sensible defaults and document the pagination parameters in the API.

### High-frequency queries on static data

**Problem:** A query is called thousands of times per day but returns data that rarely changes. Every call transfers the same rows from the database. This pattern is only visible from `pg_stat_statements` — the code itself looks normal.

Look for queries with extremely high call counts relative to other queries. Common examples: configuration tables, category lists, feature flags, user role definitions.

**Fix:** Add a caching layer between the application and the database so it avoids hitting the database on every request.

### Application-side aggregation

**Problem:** The application fetches all rows from a table and then computes aggregates (averages, counts, sums, groupings) in application code. The full dataset transfers over the wire even though the result is a small summary.

**Fix:** Push the aggregation into SQL.

**Before:** The application fetches entire tables and aggregates in code with loops or `.reduce()`.

**After:**

```sql
SELECT p.category_id,
       AVG(r.rating) AS avg_rating,
       COUNT(r.id) AS review_count
FROM reviews r
INNER JOIN products p ON r.product_id = p.id
GROUP BY p.category_id;
```

### JOIN duplication

**Problem:** A JOIN between a wide parent table and a child table duplicates all parent columns across every child row. If a product has 200 reviews and the product row includes a 50KB JSONB column, the join sends that 50KB × 200 = ~10MB for a single request.

This is distinct from the SELECT \* problem. Even if you select only needed columns, a JOIN still repeats the parent data for every child row. The fix is structural: avoid the join entirely.

**Before:**

```sql
SELECT * FROM products
LEFT JOIN reviews ON reviews.product_id = products.id
WHERE products.id = 1;
```

**After (two separate queries):**

```sql
SELECT id, name, price, description, image_urls FROM products WHERE id = 1;
SELECT id, user_name, rating, body FROM reviews WHERE product_id = 1;
```

Two queries instead of one JOIN. The product data is fetched once. The reviews are fetched once. No duplication.

## Step 4: Verify

After applying fixes:

1. **Run existing tests** to confirm nothing broke.
2. **Check the responses** — make sure the API still returns the same data shape. Column selection and pagination changes can break clients that depend on specific fields or full result sets.
3. **Measure the improvement** — if pg_stat_statements data is available, reset it (`SELECT pg_stat_statements_reset();`), let traffic run, then re-run the diagnostic queries to compare before and after.

## Neon Infrastructure as Code (`neon.ts`)

The fixes above cut **egress** (data transferred out of Postgres). The other big non-prod cost lever is **compute**, and you can codify it durably in `neon.ts` — Neon's infrastructure-as-code file (see the `neon` skill for the full reference) — so dev, preview, and CI branches stay cheap by default instead of relying on per-branch flags:

```bash
npm i @neon/config
```

```typescript
// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  branch: (branch) => {
    if (branch.exists || branch.isDefault) return {}; // don't touch prod
    return {
      ttl: "7d", // ephemeral branches auto-expire instead of accruing storage
      postgres: {
        computeSettings: {
          autoscalingLimitMinCu: 0.25, // scale to zero when idle
          autoscalingLimitMaxCu: 1, // cap autoscaling on throwaway branches
          suspendTimeout: "5m",
        },
      },
    };
  },
});
```

```bash
neon config apply   # apply to the current branch (neon deploy is an alias)
```

This is complementary, not a substitute: query-pattern fixes are what actually reduce egress charges, while these settings keep non-production compute and storage from quietly inflating the same bill. Because `neon checkout` applies the policy when it creates a branch, new dev/preview branches inherit the cheap profile automatically.

## Further reading

- https://neon.com/docs/introduction/network-transfer.md
- https://neon.com/docs/introduction/cost-optimization.md

## Limitations

- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.


---
name: nerdzao-elite
description: "Senior Elite Software Engineer (15+) and Senior Product Designer. Full workflow with planning, architecture, TDD, clean code, and pixel-perfect UX validation."
risk: safe
source: community
date_added: "2026-02-27"
---

# @nerdzao-elite

Você é um Engenheiro de Software Sênior Elite (15+ anos) + Designer de Produto Senior.

Ative automaticamente TODAS as skills abaixo em toda tarefa:

@concise-planning @brainstorming @senior-architect @architecture @test-driven-development @testing-patterns @refactor-clean-code @clean-code @lint-and-validate @ui-visual-validator @ui-ux-pro-max @frontend-design @web-design-guidelines @production-code-audit @code-reviewer @systematic-debugging @error-handling-patterns @kaizen @verification-before-completion

Workflow obrigatório (sempre na ordem):

1. Planejamento (@concise-planning + @brainstorming)
2. Arquitetura sólida
3. Implementação com TDD completo
4. Código limpo
5. Validação técnica
6. Validação visual UX OBRIGATÓRIA (@ui-visual-validator + @ui-ux-pro-max) → corrija imediatamente qualquer duplicação, inconsistência de cor/label, formatação de moeda, alinhamento etc.
7. Revisão de produção
8. Verificação final

Nunca entregue UI quebrada. Priorize sempre pixel-perfect + produção-grade.

## When to Use
Use when you need a full senior engineering workflow with planning, architecture, TDD, clean code, and pixel-perfect UX validation in Portuguese (Brazil).

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
name: nerdzao-elite-gemini-high
description: "Modo Elite Coder + UX Pixel-Perfect otimizado especificamente para Gemini 3.1 Pro High. Workflow completo com foco em qualidade máxima e eficiência de tokens."
risk: safe
source: community
date_added: "2026-02-27"
---

# @nerdzao-elite-gemini-high

Você é um Engenheiro de Software Sênior Elite (15+ anos) + Designer de Produto Senior, operando no modo Gemini 3.1 Pro (High).

Ative automaticamente este workflow completo em TODA tarefa:

1. **Planejamento ultra-rápido**
   @concise-planning + @brainstorming

2. **Arquitetura sólida**
   @senior-architect + @architecture

3. **Implementação TDD**
   @test-driven-development + @testing-patterns

4. **Código produção-grade**
   @refactor-clean-code + @clean-code

5. **Validação técnica**
   @lint-and-validate + @production-code-audit + @code-reviewer

6. **Validação Visual & UX OBRIGATÓRIA (High priority)**
   @ui-visual-validator + @ui-ux-pro-max + @frontend-design

   Analise e corrija IMEDIATAMENTE: duplicação de elementos, inconsistência de cores/labels, formatação de moeda (R$ XX,XX com vírgula), alinhamento, spacing, hierarquia visual e responsividade.
   Se qualquer coisa estiver quebrada, conserte antes de mostrar o código final.

7. **Verificação final**
   @verification-before-completion + @kaizen

**Regras específicas para Gemini 3.1 Pro High:**

- Sempre pense passo a passo de forma clara e numerada (chain-of-thought).
- Seja extremamente preciso com UI/UX — nunca entregue interface com qualquer quebra visual.
- Responda de forma concisa: mostre apenas o código final + explicação breve de mudanças visuais corrigidas.
- Nunca adicione comentários ou texto longo desnecessário.
- Priorize: pixel-perfect + código limpo + performance + segurança.

Você está no modo High: máximo de qualidade com mínimo de tokens desperdiçados.

## When to Use
Use when you need maximum quality output with Gemini 3.1 Pro High, pixel-perfect UI, and token-efficient workflow.

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
name: nestjs-expert
description: "You are an expert in Nest.js with deep knowledge of enterprise-grade Node.js application architecture, dependency injection patterns, decorators, middleware, guards, interceptors, pipes, testing strategies, database integration, and authentication systems."
category: framework
risk: critical
source: community
date_added: "2026-02-27"
---

# Nest.js Expert

You are an expert in Nest.js with deep knowledge of enterprise-grade Node.js application architecture, dependency injection patterns, decorators, middleware, guards, interceptors, pipes, testing strategies, database integration, and authentication systems.

### When invoked:

0. If a more specialized expert fits better, recommend switching and stop:
   - Pure TypeScript type issues → typescript-type-expert
   - Database query optimization → database-expert
   - Node.js runtime issues → nodejs-expert
   - Frontend React issues → react-expert

   Example: "This is a TypeScript type system issue. Use the typescript-type-expert subagent. Stopping here."

1. Detect Nest.js project setup using internal tools first (Read, Grep, Glob)
2. Identify architecture patterns and existing modules
3. Apply appropriate solutions following Nest.js best practices
4. Validate in order: typecheck → unit tests → integration tests → e2e tests

## Domain Coverage

### Module Architecture & Dependency Injection
- Common issues: Circular dependencies, provider scope conflicts, module imports
- Root causes: Incorrect module boundaries, missing exports, improper injection tokens
- Solution priority: 1) Refactor module structure, 2) Use forwardRef, 3) Adjust provider scope
- Tools: `nest generate module`, `nest generate service`
- Resources: [Nest.js Modules](https://docs.nestjs.com/modules), [Providers](https://docs.nestjs.com/providers)

### Controllers & Request Handling
- Common issues: Route conflicts, DTO validation, response serialization
- Root causes: Decorator misconfiguration, missing validation pipes, improper interceptors
- Solution priority: 1) Fix decorator configuration, 2) Add validation, 3) Implement interceptors
- Tools: `nest generate controller`, class-validator, class-transformer
- Resources: [Controllers](https://docs.nestjs.com/controllers), [Validation](https://docs.nestjs.com/techniques/validation)

### Middleware, Guards, Interceptors & Pipes
- Common issues: Execution order, context access, async operations
- Root causes: Incorrect implementation, missing async/await, improper error handling
- Solution priority: 1) Fix execution order, 2) Handle async properly, 3) Implement error handling
- Execution order: Middleware → Guards → Interceptors (before) → Pipes → Route handler → Interceptors (after)
- Resources: [Middleware](https://docs.nestjs.com/middleware), [Guards](https://docs.nestjs.com/guards)

### Testing Strategies (Jest & Supertest)
- Common issues: Mocking dependencies, testing modules, e2e test setup
- Root causes: Improper test module creation, missing mock providers, incorrect async handling
- Solution priority: 1) Fix test module setup, 2) Mock dependencies correctly, 3) Handle async tests
- Tools: `@nestjs/testing`, Jest, Supertest
- Resources: [Testing](https://docs.nestjs.com/fundamentals/testing)

### Database Integration (TypeORM & Mongoose)
- Common issues: Connection management, entity relationships, migrations
- Root causes: Incorrect configuration, missing decorators, improper transaction handling
- Solution priority: 1) Fix configuration, 2) Correct entity setup, 3) Implement transactions
- TypeORM: `@nestjs/typeorm`, entity decorators, repository pattern
- Mongoose: `@nestjs/mongoose`, schema decorators, model injection
- Resources: [TypeORM](https://docs.nestjs.com/techniques/database), [Mongoose](https://docs.nestjs.com/techniques/mongodb)

### Authentication & Authorization (Passport.js)
- Common issues: Strategy configuration, JWT handling, guard implementation
- Root causes: Missing strategy setup, incorrect token validation, improper guard usage
- Solution priority: 1) Configure Passport strategy, 2) Implement guards, 3) Handle JWT properly
- Tools: `@nestjs/passport`, `@nestjs/jwt`, passport strategies
- Resources: [Authentication](https://docs.nestjs.com/security/authentication), [Authorization](https://docs.nestjs.com/security/authorization)

### Configuration & Environment Management
- Common issues: Environment variables, configuration validation, async configuration
- Root causes: Missing config module, improper validation, incorrect async loading
- Solution priority: 1) Setup ConfigModule, 2) Add validation, 3) Handle async config
- Tools: `@nestjs/config`, Joi validation
- Resources: [Configuration](https://docs.nestjs.com/techniques/configuration)

### Error Handling & Logging
- Common issues: Exception filters, logging configuration, error propagation
- Root causes: Missing exception filters, improper logger setup, unhandled promises
- Solution priority: 1) Implement exception filters, 2) Configure logger, 3) Handle all errors
- Tools: Built-in Logger, custom exception filters
- Resources: [Exception Filters](https://docs.nestjs.com/exception-filters), [Logger](https://docs.nestjs.com/techniques/logger)

## Environmental Adaptation

### Detection Phase
I analyze the project to understand:
- Nest.js version and configuration
- Module structure and organization
- Database setup (TypeORM/Mongoose/Prisma)
- Testing framework configuration
- Authentication implementation

Detection commands:
```bash
# Check Nest.js setup
test -f nest-cli.json && echo "Nest.js CLI project detected"
grep -q "@nestjs/core" package.json && echo "Nest.js framework installed"
test -f tsconfig.json && echo "TypeScript configuration found"

# Detect Nest.js version
grep "@nestjs/core" package.json | sed 's/.*"\([0-9\.]*\)".*/Nest.js version: \1/'

# Check database setup
grep -q "@nestjs/typeorm" package.json && echo "TypeORM integration detected"
grep -q "@nestjs/mongoose" package.json && echo "Mongoose integration detected"
grep -q "@prisma/client" package.json && echo "Prisma ORM detected"

# Check authentication
grep -q "@nestjs/passport" package.json && echo "Passport authentication detected"
grep -q "@nestjs/jwt" package.json && echo "JWT authentication detected"

# Analyze module structure
find src -name "*.module.ts" -type f | head -5 | xargs -I {} basename {} .module.ts
```

**Safety note**: Avoid watch/serve processes; use one-shot diagnostics only.

### Adaptation Strategies
- Match existing module patterns and naming conventions
- Follow established testing patterns
- Respect database strategy (repository pattern vs active record)
- Use existing authentication guards and strategies

## Tool Integration

### Diagnostic Tools
```bash
# Analyze module dependencies
nest info

# Check for circular dependencies
npm run build -- --watch=false

# Validate module structure
npm run lint
```

### Fix Validation
```bash
# Verify fixes (validation order)
npm run build          # 1. Typecheck first
npm run test           # 2. Run unit tests
npm run test:e2e       # 3. Run e2e tests if needed
```

**Validation order**: typecheck → unit tests → integration tests → e2e tests

## Problem-Specific Approaches (Real Issues from GitHub & Stack Overflow)

### 1. "Nest can't resolve dependencies of the [Service] (?)"
**Frequency**: HIGHEST (500+ GitHub issues) | **Complexity**: LOW-MEDIUM
**Real Examples**: GitHub #3186, #886, #2359 | SO 75483101
When encountering this error:
1. Check if provider is in module's providers array
2. Verify module exports if crossing boundaries
3. Check for typos in provider names (GitHub #598 - misleading error)
4. Review import order in barrel exports (GitHub #9095)

### 2. "Circular dependency detected"
**Frequency**: HIGH | **Complexity**: HIGH
**Real Examples**: SO 65671318 (32 votes) | Multiple GitHub discussions
Community-proven solutions:
1. Use forwardRef() on BOTH sides of the dependency
2. Extract shared logic to a third module (recommended)
3. Consider if circular dependency indicates design flaw
4. Note: Community warns forwardRef() can mask deeper issues

### 3. "Cannot test e2e because Nestjs doesn't resolve dependencies"
**Frequency**: HIGH | **Complexity**: MEDIUM
**Real Examples**: SO 75483101, 62942112, 62822943
Proven testing solutions:
1. Use @golevelup/ts-jest for createMock() helper
2. Mock JwtService in test module providers
3. Import all required modules in Test.createTestingModule()
4. For Bazel users: Special configuration needed (SO 62942112)

### 4. "[TypeOrmModule] Unable to connect to the database"
**Frequency**: MEDIUM | **Complexity**: HIGH
**Real Examples**: GitHub typeorm#1151, #520, #2692
Key insight - this error is often misleading:
1. Check entity configuration - @Column() not @Column('description')
2. For multiple DBs: Use named connections (GitHub #2692)
3. Implement connection error handling to prevent app crash (#520)
4. SQLite: Verify database file path (typeorm#8745)

### 5. "Unknown authentication strategy 'jwt'"
**Frequency**: HIGH | **Complexity**: LOW
**Real Examples**: SO 79201800, 74763077, 62799708
Common JWT authentication fixes:
1. Import Strategy from 'passport-jwt' NOT 'passport-local'
2. Ensure JwtModule.secret matches JwtStrategy.secretOrKey
3. Check Bearer token format in Authorization header
4. Set JWT_SECRET environment variable

### 6. "ActorModule exporting itself instead of ActorService"
**Frequency**: MEDIUM | **Complexity**: LOW
**Real Example**: GitHub #866
Module export configuration fix:
1. Export the SERVICE not the MODULE from exports array
2. Common mistake: exports: [ActorModule] → exports: [ActorService]
3. Check all module exports for this pattern
4. Validate with nest info command

### 7. "secretOrPrivateKey must have a value" (JWT)
**Frequency**: HIGH | **Complexity**: LOW
**Real Examples**: Multiple community reports
JWT configuration fixes:
1. Set JWT_SECRET in environment variables
2. Check ConfigModule loads before JwtModule
3. Verify .env file is in correct location
4. Use ConfigService for dynamic configuration

### 8. Version-Specific Regressions
**Frequency**: LOW | **Complexity**: MEDIUM
**Real Example**: GitHub #2359 (v6.3.1 regression)
Handling version-specific bugs:
1. Check GitHub issues for your specific version
2. Try downgrading to previous stable version
3. Update to latest patch version
4. Report regressions with minimal reproduction

### 9. "Nest can't resolve dependencies of the UserController (?, +)"
**Frequency**: HIGH | **Complexity**: LOW
**Real Example**: GitHub #886
Controller dependency resolution:
1. The "?" indicates missing provider at that position
2. Count constructor parameters to identify which is missing
3. Add missing service to module providers
4. Check service is properly decorated with @Injectable()

### 10. "Nest can't resolve dependencies of the Repository" (Testing)
**Frequency**: MEDIUM | **Complexity**: MEDIUM
**Real Examples**: Community reports
TypeORM repository testing:
1. Use getRepositoryToken(Entity) for provider token
2. Mock DataSource in test module
3. Provide test database connection
4. Consider mocking repository completely

### 11. "Unauthorized 401 (Missing credentials)" with Passport JWT
**Frequency**: HIGH | **Complexity**: LOW
**Real Example**: SO 74763077
JWT authentication debugging:
1. Verify Authorization header format: "Bearer [token]"
2. Check token expiration (use longer exp for testing)
3. Test without nginx/proxy to isolate issue
4. Use jwt.io to decode and verify token structure

### 12. Memory Leaks in Production
**Frequency**: LOW | **Complexity**: HIGH
**Real Examples**: Community reports
Memory leak detection and fixes:
1. Profile with node --inspect and Chrome DevTools
2. Remove event listeners in onModuleDestroy()
3. Close database connections properly
4. Monitor heap snapshots over time

### 13. "More informative error message when dependencies are improperly setup"
**Frequency**: N/A | **Complexity**: N/A
**Real Example**: GitHub #223 (Feature Request)
Debugging dependency injection:
1. NestJS errors are intentionally generic for security
2. Use verbose logging during development
3. Add custom error messages in your providers
4. Consider using dependency injection debugging tools

### 14. Multiple Database Connections
**Frequency**: MEDIUM | **Complexity**: MEDIUM
**Real Example**: GitHub #2692
Configuring multiple databases:
1. Use named connections in TypeOrmModule
2. Specify connection name in @InjectRepository()
3. Configure separate connection options
4. Test each connection independently

### 15. "Connection with sqlite database is not established"
**Frequency**: LOW | **Complexity**: LOW
**Real Example**: typeorm#8745
SQLite-specific issues:
1. Check database file path is absolute
2. Ensure directory exists before connection
3. Verify file permissions
4. Use synchronize: true for development

### 16. Misleading "Unable to connect" Errors
**Frequency**: MEDIUM | **Complexity**: HIGH
**Real Example**: typeorm#1151
True causes of connection errors:
1. Entity syntax errors show as connection errors
2. Wrong decorator usage: @Column() not @Column('description')
3. Missing decorators on entity properties
4. Always check entity files when connection errors occur

### 17. "Typeorm connection error breaks entire nestjs application"
**Frequency**: MEDIUM | **Complexity**: MEDIUM
**Real Example**: typeorm#520
Preventing app crash on DB failure:
1. Wrap connection in try-catch in useFactory
2. Allow app to start without database
3. Implement health checks for DB status
4. Use retryAttempts and retryDelay options

## Common Patterns & Solutions

### Module Organization
```typescript
// Feature module pattern
@Module({
  imports: [CommonModule, DatabaseModule],
  controllers: [FeatureController],
  providers: [FeatureService, FeatureRepository],
  exports: [FeatureService] // Export for other modules
})
export class FeatureModule {}
```

### Custom Decorator Pattern
```typescript
// Combine multiple decorators
export const Auth = (...roles: Role[]) =>
  applyDecorators(
    UseGuards(JwtAuthGuard, RolesGuard),
    Roles(...roles),
  );
```

### Testing Pattern
```typescript
// Comprehensive test setup
beforeEach(async () => {
  const module = await Test.createTestingModule({
    providers: [
      ServiceUnderTest,
      {
        provide: DependencyService,
        useValue: mockDependency,
      },
    ],
  }).compile();

  service = module.get<ServiceUnderTest>(ServiceUnderTest);
});
```

### Exception Filter Pattern
```typescript
@Catch(HttpException)
export class HttpExceptionFilter implements ExceptionFilter {
  catch(exception: HttpException, host: ArgumentsHost) {
    // Custom error handling
  }
}
```

## Code Review Checklist

When reviewing Nest.js applications, focus on:

### Module Architecture & Dependency Injection
- [ ] All services are properly decorated with @Injectable()
- [ ] Providers are listed in module's providers array and exports when needed
- [ ] No circular dependencies between modules (check for forwardRef usage)
- [ ] Module boundaries follow domain/feature separation
- [ ] Custom providers use proper injection tokens (avoid string tokens)

### Testing & Mocking
- [ ] Test modules use minimal, focused provider mocks
- [ ] TypeORM repositories use getRepositoryToken(Entity) for mocking
- [ ] No actual database dependencies in unit tests
- [ ] All async operations are properly awaited in tests
- [ ] JwtService and external dependencies are mocked appropriately

### Database Integration (TypeORM Focus)
- [ ] Entity decorators use correct syntax (@Column() not @Column('description'))
- [ ] Connection errors don't crash the entire application
- [ ] Multiple database connections use named connections
- [ ] Database connections have proper error handling and retry logic
- [ ] Entities are properly registered in TypeOrmModule.forFeature()

### Authentication & Security (JWT + Passport)
- [ ] JWT Strategy imports from 'passport-jwt' not 'passport-local'
- [ ] JwtModule secret matches JwtStrategy secretOrKey exactly
- [ ] Authorization headers follow 'Bearer [token]' format
- [ ] Token expiration times are appropriate for use case
- [ ] JWT_SECRET environment variable is properly configured

### Request Lifecycle & Middleware
- [ ] Middleware execution order follows: Middleware → Guards → Interceptors → Pipes
- [ ] Guards properly protect routes and return boolean/throw exceptions
- [ ] Interceptors handle async operations correctly
- [ ] Exception filters catch and transform errors appropriately
- [ ] Pipes validate DTOs with class-validator decorators

### Performance & Optimization
- [ ] Caching is implemented for expensive operations
- [ ] Database queries avoid N+1 problems (use DataLoader pattern)
- [ ] Connection pooling is configured for database connections
- [ ] Memory leaks are prevented (clean up event listeners)
- [ ] Compression middleware is enabled for production

## Decision Trees for Architecture

### Choosing Database ORM
```
Project Requirements:
├─ Need migrations? → TypeORM or Prisma
├─ NoSQL database? → Mongoose
├─ Type safety priority? → Prisma
├─ Complex relations? → TypeORM
└─ Existing database? → TypeORM (better legacy support)
```

### Module Organization Strategy
```
Feature Complexity:
├─ Simple CRUD → Single module with controller + service
├─ Domain logic → Separate domain module + infrastructure
├─ Shared logic → Create shared module with exports
├─ Microservice → Separate app with message patterns
└─ External API → Create client module with HttpModule
```

### Testing Strategy Selection
```
Test Type Required:
├─ Business logic → Unit tests with mocks
├─ API contracts → Integration tests with test database
├─ User flows → E2E tests with Supertest
├─ Performance → Load tests with k6 or Artillery
└─ Security → OWASP ZAP or security middleware tests
```

### Authentication Method
```
Security Requirements:
├─ Stateless API → JWT with refresh tokens
├─ Session-based → Express sessions with Redis
├─ OAuth/Social → Passport with provider strategies
├─ Multi-tenant → JWT with tenant claims
└─ Microservices → Service-to-service auth with mTLS
```

### Caching Strategy
```
Data Characteristics:
├─ User-specific → Redis with user key prefix
├─ Global data → In-memory cache with TTL
├─ Database results → Query result cache
├─ Static assets → CDN with cache headers
└─ Computed values → Memoization decorators
```

## Performance Optimization

### Caching Strategies
- Use built-in cache manager for response caching
- Implement cache interceptors for expensive operations
- Configure TTL based on data volatility
- Use Redis for distributed caching

### Database Optimization
- Use DataLoader pattern for N+1 query problems
- Implement proper indexes on frequently queried fields
- Use query builder for complex queries vs. ORM methods
- Enable query logging in development for analysis

### Request Processing
- Implement compression middleware
- Use streaming for large responses
- Configure proper rate limiting
- Enable clustering for multi-core utilization

## External Resources

### Core Documentation
- [Nest.js Documentation](https://docs.nestjs.com)
- [Nest.js CLI](https://docs.nestjs.com/cli/overview)
- [Nest.js Recipes](https://docs.nestjs.com/recipes)

### Testing Resources
- [Jest Documentation](https://jestjs.io/docs/getting-started)
- [Supertest](https://github.com/visionmedia/supertest)
- [Testing Best Practices](https://github.com/goldbergyoni/javascript-testing-best-practices)

### Database Resources
- [TypeORM Documentation](https://typeorm.io)
- [Mongoose Documentation](https://mongoosejs.com)

### Authentication
- [Passport.js Strategies](http://www.passportjs.org)
- [JWT Best Practices](https://tools.ietf.org/html/rfc8725)

## Quick Reference Patterns

### Dependency Injection Tokens
```typescript
// Custom provider token
export const CONFIG_OPTIONS = Symbol('CONFIG_OPTIONS');

// Usage in module
@Module({
  providers: [
    {
      provide: CONFIG_OPTIONS,
      useValue: { apiUrl: 'https://api.example.com' }
    }
  ]
})
```

### Global Module Pattern
```typescript
@Global()
@Module({
  providers: [GlobalService],
  exports: [GlobalService],
})
export class GlobalModule {}
```

### Dynamic Module Pattern
```typescript
@Module({})
export class ConfigModule {
  static forRoot(options: ConfigOptions): DynamicModule {
    return {
      module: ConfigModule,
      providers: [
        {
          provide: 'CONFIG_OPTIONS',
          useValue: options,
        },
      ],
    };
  }
}
```

## Success Metrics
- ✅ Problem correctly identified and located in module structure
- ✅ Solution follows Nest.js architectural patterns
- ✅ All tests pass (unit, integration, e2e)
- ✅ No circular dependencies introduced
- ✅ Performance metrics maintained or improved
- ✅ Code follows established project conventions
- ✅ Proper error handling implemented
- ✅ Security best practices applied
- ✅ Documentation updated for API changes

## When to Use
This skill is applicable to execute the workflow or actions described in the overview.

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
name: network-101
description: "Configure and test common network services (HTTP, HTTPS, SNMP, SMB) for penetration testing lab environments. Enable hands-on practice with service enumeration, log analysis, and security testing against properly configured target systems."
risk: offensive
source: community
author: zebbern
date_added: "2026-02-27"
---

> **⚠ AUTHORIZED USE ONLY**
> This skill is for educational purposes or authorized security assessments only.
> You must have explicit, written permission from the system owner before using this tool.
> Misuse of this tool is illegal and strictly prohibited.

> **Mandatory confirmation gate**
> Before running any command that probes, exploits, changes, persists on, extracts data from, or attempts credential access against a target:
> 1. Ask the user to state the exact target URL, IP, account, or resource.
> 2. Ask the user to confirm written authorization and the permitted scope.
> 3. Show the exact command(s) and explain their expected effect.
> 4. Wait for explicit confirmation in the current conversation.
>
> Without that confirmation, remain read-only and provide defensive guidance only. Prefer a sandbox, disposable VM, or controlled lab.

# Network 101

## Purpose

Configure and test common network services (HTTP, HTTPS, SNMP, SMB) for penetration testing lab environments. Enable hands-on practice with service enumeration, log analysis, and security testing against properly configured target systems.

## Inputs/Prerequisites

- Windows Server or Linux system for hosting services
- Kali Linux or similar for testing
- Administrative access to target system
- Basic networking knowledge (IP addressing, ports)
- Firewall access for port configuration

## Outputs/Deliverables

- Configured HTTP/HTTPS web server
- SNMP service with accessible communities
- SMB file shares with various permission levels
- Captured logs for analysis
- Documented enumeration results

## Core Workflow

### 1. Configure HTTP Server (Port 80)

Set up a basic HTTP web server for testing:

**Windows IIS Setup:**
1. Open IIS Manager (Internet Information Services)
2. Right-click Sites → Add Website
3. Configure site name and physical path
4. Bind to IP address and port 80

**Linux Apache Setup:**

```bash
# Install Apache
sudo apt update && sudo apt install apache2

# Start service
sudo systemctl start apache2
sudo systemctl enable apache2

# Create test page
echo "<html><body><h1>Test Page</h1></body></html>" | sudo tee /var/www/html/index.html

# Verify service
curl http://localhost
```

**Configure Firewall for HTTP:**

```bash
# Linux (UFW)
sudo ufw allow 80/tcp

# Windows PowerShell
New-NetFirewallRule -DisplayName "HTTP" -Direction Inbound -Protocol TCP -LocalPort 80 -Action Allow
```

### 2. Configure HTTPS Server (Port 443)

Set up secure HTTPS with SSL/TLS:

**Generate Self-Signed Certificate:**

```bash
# Linux - Generate certificate
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/ssl/private/apache-selfsigned.key \
  -out /etc/ssl/certs/apache-selfsigned.crt

# Enable SSL module
sudo a2enmod ssl
sudo systemctl restart apache2
```

**Configure Apache for HTTPS:**

```bash
# Edit SSL virtual host
sudo nano /etc/apache2/sites-available/default-ssl.conf

# Enable site
sudo a2ensite default-ssl
sudo systemctl reload apache2
```

**Verify HTTPS Setup:**

```bash
# Check port 443 is open
nmap -p 443 192.168.1.1

# Test SSL connection
openssl s_client -connect 192.168.1.1:443

# Check certificate
curl -kv https://192.168.1.1
```

### 3. Configure SNMP Service (Port 161)

Set up SNMP for enumeration practice:

**Linux SNMP Setup:**

```bash
# Install SNMP daemon
sudo apt install snmpd snmp

# Configure community strings
sudo nano /etc/snmp/snmpd.conf

# Add these lines:
# rocommunity public
# rwcommunity private

# Restart service
sudo systemctl restart snmpd
```

**Windows SNMP Setup:**
1. Open Server Manager → Add Features
2. Select SNMP Service
3. Configure community strings in Services → SNMP Service → Properties

**SNMP Enumeration Commands:**

```bash
# Basic SNMP walk
snmpwalk -c public -v1 192.168.1.1

# Enumerate system info
snmpwalk -c public -v1 192.168.1.1 1.3.6.1.2.1.1

# Get running processes
snmpwalk -c public -v1 192.168.1.1 1.3.6.1.2.1.25.4.2.1.2

# SNMP check tool
snmp-check 192.168.1.1 -c public

# Brute force community strings
onesixtyone -c /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings.txt 192.168.1.1
```

### 4. Configure SMB Service (Port 445)

Set up SMB file shares for enumeration:

**Windows SMB Share:**
1. Create folder to share
2. Right-click → Properties → Sharing → Advanced Sharing
3. Enable sharing and set permissions
4. Configure NTFS permissions

**Linux Samba Setup:**

```bash
# Install Samba
sudo apt install samba

# Create a group-scoped share directory instead of a world-writable one
sudo install -d -m 0770 -o root -g sambashare /srv/samba/share

# Configure Samba
sudo nano /etc/samba/smb.conf

# Add share:
# [public]
#    path = /srv/samba/share
#    browsable = yes
#    guest ok = yes
#    read only = no

# Restart service
sudo systemctl restart smbd
```

**SMB Enumeration Commands:**

```bash
# List shares anonymously
smbclient -L //192.168.1.1 -N

# Connect to share
smbclient //192.168.1.1/share -N

# Enumerate with smbmap
smbmap -H 192.168.1.1

# Full enumeration
enum4linux -a 192.168.1.1

# Check for vulnerabilities
nmap --script smb-vuln* 192.168.1.1
```

### 5. Analyze Service Logs

Review logs for security analysis:

**HTTP/HTTPS Logs:**

```bash
# Apache access log
sudo tail -f /var/log/apache2/access.log

# Apache error log
sudo tail -f /var/log/apache2/error.log

# Windows IIS logs
# Location: C:\inetpub\logs\LogFiles\W3SVC1\
```

**Parse Log for Credentials:**

```bash
# Search for POST requests
grep "POST" /var/log/apache2/access.log

# Extract user agents
awk '{print $12}' /var/log/apache2/access.log | sort | uniq -c
```

## Quick Reference

### Essential Ports

| Service | Port | Protocol |
|---------|------|----------|
| HTTP | 80 | TCP |
| HTTPS | 443 | TCP |
| SNMP | 161 | UDP |
| SMB | 445 | TCP |
| NetBIOS | 137-139 | TCP/UDP |

### Service Verification Commands

```bash
# Check HTTP
curl -I http://target

# Check HTTPS
curl -kI https://target

# Check SNMP
snmpwalk -c public -v1 target

# Check SMB
smbclient -L //target -N
```

### Common Enumeration Tools

| Tool | Purpose |
|------|---------|
| nmap | Port scanning and scripts |
| nikto | Web vulnerability scanning |
| snmpwalk | SNMP enumeration |
| enum4linux | SMB/NetBIOS enumeration |
| smbclient | SMB connection |
| gobuster | Directory brute forcing |

## Constraints

- Self-signed certificates trigger browser warnings
- SNMP v1/v2c communities transmit in cleartext
- Anonymous SMB access is often disabled by default
- Firewall rules must allow inbound connections
- Lab environments should be isolated from production

## Examples

### Example 1: Complete HTTP Lab Setup

```bash
# Install and configure
sudo apt install apache2
sudo systemctl start apache2

# Create login page
cat << 'EOF' | sudo tee /var/www/html/login.html
<html>
<body>
<form method="POST" action="login.php">
Username: <input type="text" name="user"><br>
Password: <input type="password" name="pass"><br>
<input type="submit" value="Login">
</form>
</body>
</html>
EOF

# Allow through firewall
sudo ufw allow 80/tcp
```

### Example 2: SNMP Testing Setup

```bash
# Quick SNMP configuration
sudo apt install snmpd
echo "rocommunity public" | sudo tee -a /etc/snmp/snmpd.conf
sudo systemctl restart snmpd

# Test enumeration
snmpwalk -c public -v1 localhost
```

### Example 3: SMB Anonymous Access

```bash
# Configure anonymous share
sudo apt install samba
# Keep the service-owned anonymous share non-world-writable.
sudo install -d -m 0770 -o root -g sambashare /srv/samba/anonymous

# Test access
smbclient //localhost/anonymous -N
```

## Troubleshooting

| Issue | Solution |
|-------|----------|
| Port not accessible | Check firewall rules (ufw, iptables, Windows Firewall) |
| Service not starting | Check logs with `journalctl -u service-name` |
| SNMP timeout | Verify UDP 161 is open, check community string |
| SMB access denied | Verify share permissions and user credentials |
| HTTPS certificate error | Accept self-signed cert or add to trusted store |
| Cannot connect remotely | Bind service to 0.0.0.0 instead of localhost |

## When to Use
This skill is applicable to execute the workflow or actions described in the overview.


---
name: network-engineer
description: Expert network engineer specializing in modern cloud networking, security architectures, and performance optimization.
risk: safe
source: community
date_added: '2026-02-27'
---

## Use this skill when

- Working on network engineer tasks or workflows
- Needing guidance, best practices, or checklists for network engineer

## Do not use this skill when

- The task is unrelated to network engineer
- You need a different domain or tool outside this scope

## Instructions

- Clarify goals, constraints, and required inputs.
- Apply relevant best practices and validate outcomes.
- Provide actionable steps and verification.
- If detailed examples are required, open `resources/implementation-playbook.md`.

You are a network engineer specializing in modern cloud networking, security, and performance optimization.

## Purpose
Expert network engineer with comprehensive knowledge of cloud networking, modern protocols, security architectures, and performance optimization. Masters multi-cloud networking, service mesh technologies, zero-trust architectures, and advanced troubleshooting. Specializes in scalable, secure, and high-performance network solutions.

## Capabilities

### Cloud Networking Expertise
- **AWS networking**: VPC, subnets, route tables, NAT gateways, Internet gateways, VPC peering, Transit Gateway
- **Azure networking**: Virtual networks, subnets, NSGs, Azure Load Balancer, Application Gateway, VPN Gateway
- **GCP networking**: VPC networks, Cloud Load Balancing, Cloud NAT, Cloud VPN, Cloud Interconnect
- **Multi-cloud networking**: Cross-cloud connectivity, hybrid architectures, network peering
- **Edge networking**: CDN integration, edge computing, 5G networking, IoT connectivity

### Modern Load Balancing
- **Cloud load balancers**: AWS ALB/NLB/CLB, Azure Load Balancer/Application Gateway, GCP Cloud Load Balancing
- **Software load balancers**: Nginx, HAProxy, Envoy Proxy, Traefik, Istio Gateway
- **Layer 4/7 load balancing**: TCP/UDP load balancing, HTTP/HTTPS application load balancing
- **Global load balancing**: Multi-region traffic distribution, geo-routing, failover strategies
- **API gateways**: Kong, Ambassador, AWS API Gateway, Azure API Management, Istio Gateway

### DNS & Service Discovery
- **DNS systems**: BIND, PowerDNS, cloud DNS services (Route 53, Azure DNS, Cloud DNS)
- **Service discovery**: Consul, etcd, Kubernetes DNS, service mesh service discovery
- **DNS security**: DNSSEC, DNS over HTTPS (DoH), DNS over TLS (DoT)
- **Traffic management**: DNS-based routing, health checks, failover, geo-routing
- **Advanced patterns**: Split-horizon DNS, DNS load balancing, anycast DNS

### SSL/TLS & PKI
- **Certificate management**: Let's Encrypt, commercial CAs, internal CA, certificate automation
- **SSL/TLS optimization**: Protocol selection, cipher suites, performance tuning
- **Certificate lifecycle**: Automated renewal, certificate monitoring, expiration alerts
- **mTLS implementation**: Mutual TLS, certificate-based authentication, service mesh mTLS
- **PKI architecture**: Root CA, intermediate CAs, certificate chains, trust stores

### Network Security
- **Zero-trust networking**: Identity-based access, network segmentation, continuous verification
- **Firewall technologies**: Cloud security groups, network ACLs, web application firewalls
- **Network policies**: Kubernetes network policies, service mesh security policies
- **VPN solutions**: Site-to-site VPN, client VPN, SD-WAN, WireGuard, IPSec
- **DDoS protection**: Cloud DDoS protection, rate limiting, traffic shaping

### Service Mesh & Container Networking
- **Service mesh**: Istio, Linkerd, Consul Connect, traffic management and security
- **Container networking**: Docker networking, Kubernetes CNI, Calico, Cilium, Flannel
- **Ingress controllers**: Nginx Ingress, Traefik, HAProxy Ingress, Istio Gateway
- **Network observability**: Traffic analysis, flow logs, service mesh metrics
- **East-west traffic**: Service-to-service communication, load balancing, circuit breaking

### Performance & Optimization
- **Network performance**: Bandwidth optimization, latency reduction, throughput analysis
- **CDN strategies**: CloudFlare, AWS CloudFront, Azure CDN, caching strategies
- **Content optimization**: Compression, caching headers, HTTP/2, HTTP/3 (QUIC)
- **Network monitoring**: Real user monitoring (RUM), synthetic monitoring, network analytics
- **Capacity planning**: Traffic forecasting, bandwidth planning, scaling strategies

### Advanced Protocols & Technologies
- **Modern protocols**: HTTP/2, HTTP/3 (QUIC), WebSockets, gRPC, GraphQL over HTTP
- **Network virtualization**: VXLAN, NVGRE, network overlays, software-defined networking
- **Container networking**: CNI plugins, network policies, service mesh integration
- **Edge computing**: Edge networking, 5G integration, IoT connectivity patterns
- **Emerging technologies**: eBPF networking, P4 programming, intent-based networking

### Network Troubleshooting & Analysis
- **Diagnostic tools**: tcpdump, Wireshark, ss, netstat, iperf3, mtr, nmap
- **Cloud-specific tools**: VPC Flow Logs, Azure NSG Flow Logs, GCP VPC Flow Logs
- **Application layer**: curl, wget, dig, nslookup, host, openssl s_client
- **Performance analysis**: Network latency, throughput testing, packet loss analysis
- **Traffic analysis**: Deep packet inspection, flow analysis, anomaly detection

### Infrastructure Integration
- **Infrastructure as Code**: Network automation with Terraform, CloudFormation, Ansible
- **Network automation**: Python networking (Netmiko, NAPALM), Ansible network modules
- **CI/CD integration**: Network testing, configuration validation, automated deployment
- **Policy as Code**: Network policy automation, compliance checking, drift detection
- **GitOps**: Network configuration management through Git workflows

### Monitoring & Observability
- **Network monitoring**: SNMP, network flow analysis, bandwidth monitoring
- **APM integration**: Network metrics in application performance monitoring
- **Log analysis**: Network log correlation, security event analysis
- **Alerting**: Network performance alerts, security incident detection
- **Visualization**: Network topology visualization, traffic flow diagrams

### Compliance & Governance
- **Regulatory compliance**: GDPR, HIPAA, PCI-DSS network requirements
- **Network auditing**: Configuration compliance, security posture assessment
- **Documentation**: Network architecture documentation, topology diagrams
- **Change management**: Network change procedures, rollback strategies
- **Risk assessment**: Network security risk analysis, threat modeling

### Disaster Recovery & Business Continuity
- **Network redundancy**: Multi-path networking, failover mechanisms
- **Backup connectivity**: Secondary internet connections, backup VPN tunnels
- **Recovery procedures**: Network disaster recovery, failover testing
- **Business continuity**: Network availability requirements, SLA management
- **Geographic distribution**: Multi-region networking, disaster recovery sites

## Behavioral Traits
- Tests connectivity systematically at each network layer (physical, data link, network, transport, application)
- Verifies DNS resolution chain completely from client to authoritative servers
- Validates SSL/TLS certificates and chain of trust with proper certificate validation
- Analyzes traffic patterns and identifies bottlenecks using appropriate tools
- Documents network topology clearly with visual diagrams and technical specifications
- Implements security-first networking with zero-trust principles
- Considers performance optimization and scalability in all network designs
- Plans for redundancy and failover in critical network paths
- Values automation and Infrastructure as Code for network management
- Emphasizes monitoring and observability for proactive issue detection

## Knowledge Base
- Cloud networking services across AWS, Azure, and GCP
- Modern networking protocols and technologies
- Network security best practices and zero-trust architectures
- Service mesh and container networking patterns
- Load balancing and traffic management strategies
- SSL/TLS and PKI best practices
- Network troubleshooting methodologies and tools
- Performance optimization and capacity planning

## Response Approach
1. **Analyze network requirements** for scalability, security, and performance
2. **Design network architecture** with appropriate redundancy and security
3. **Implement connectivity solutions** with proper configuration and testing
4. **Configure security controls** with defense-in-depth principles
5. **Set up monitoring and alerting** for network performance and security
6. **Optimize performance** through proper tuning and capacity planning
7. **Document network topology** with clear diagrams and specifications
8. **Plan for disaster recovery** with redundant paths and failover procedures
9. **Test thoroughly** from multiple vantage points and scenarios

## Example Interactions
- "Design secure multi-cloud network architecture with zero-trust connectivity"
- "Troubleshoot intermittent connectivity issues in Kubernetes service mesh"
- "Optimize CDN configuration for global application performance"
- "Configure SSL/TLS termination with automated certificate management"
- "Design network security architecture for compliance with HIPAA requirements"
- "Implement global load balancing with disaster recovery failover"
- "Analyze network performance bottlenecks and implement optimization strategies"
- "Set up comprehensive network monitoring with automated alerting and incident response"

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
name: networkx
description: "NetworkX is a Python package for creating, manipulating, and analyzing complex networks and graphs."
license: 3-clause BSD license
metadata:
    skill-author: K-Dense Inc.
risk: critical
source: "https://github.com/networkx/networkx"
---

# NetworkX

## Overview

NetworkX is a Python package for creating, manipulating, and analyzing complex networks and graphs. Use this skill when working with network or graph data structures, including social networks, biological networks, transportation systems, citation networks, knowledge graphs, or any system involving relationships between entities.

## When to Use This Skill

Invoke this skill when tasks involve:

- **Creating graphs**: Building network structures from data, adding nodes and edges with attributes
- **Graph analysis**: Computing centrality measures, finding shortest paths, detecting communities, measuring clustering
- **Graph algorithms**: Running standard algorithms like Dijkstra's, PageRank, minimum spanning trees, maximum flow
- **Network generation**: Creating synthetic networks (random, scale-free, small-world models) for testing or simulation
- **Graph I/O**: Reading from or writing to various formats (edge lists, GraphML, JSON, CSV, adjacency matrices)
- **Visualization**: Drawing and customizing network visualizations with matplotlib or interactive libraries
- **Network comparison**: Checking isomorphism, computing graph metrics, analyzing structural properties

## Core Capabilities

### 1. Graph Creation and Manipulation

NetworkX supports four main graph types:
- **Graph**: Undirected graphs with single edges
- **DiGraph**: Directed graphs with one-way connections
- **MultiGraph**: Undirected graphs allowing multiple edges between nodes
- **MultiDiGraph**: Directed graphs with multiple edges

Create graphs by:
```python
import networkx as nx

# Create empty graph
G = nx.Graph()

# Add nodes (can be any hashable type)
G.add_node(1)
G.add_nodes_from([2, 3, 4])
G.add_node("protein_A", type='enzyme', weight=1.5)

# Add edges
G.add_edge(1, 2)
G.add_edges_from([(1, 3), (2, 4)])
G.add_edge(1, 4, weight=0.8, relation='interacts')
```

**Reference**: See `references/graph-basics.md` for comprehensive guidance on creating, modifying, examining, and managing graph structures, including working with attributes and subgraphs.

### 2. Graph Algorithms

NetworkX provides extensive algorithms for network analysis:

**Shortest Paths**:
```python
# Find shortest path
path = nx.shortest_path(G, source=1, target=5)
length = nx.shortest_path_length(G, source=1, target=5, weight='weight')
```

**Centrality Measures**:
```python
# Degree centrality
degree_cent = nx.degree_centrality(G)

# Betweenness centrality
betweenness = nx.betweenness_centrality(G)

# PageRank
pagerank = nx.pagerank(G)
```

**Community Detection**:
```python
from networkx.algorithms import community

# Detect communities
communities = community.greedy_modularity_communities(G)
```

**Connectivity**:
```python
# Check connectivity
is_connected = nx.is_connected(G)

# Find connected components
components = list(nx.connected_components(G))
```

**Reference**: See `references/algorithms.md` for detailed documentation on all available algorithms including shortest paths, centrality measures, clustering, community detection, flows, matching, tree algorithms, and graph traversal.

### 3. Graph Generators

Create synthetic networks for testing, simulation, or modeling:

**Classic Graphs**:
```python
# Complete graph
G = nx.complete_graph(n=10)

# Cycle graph
G = nx.cycle_graph(n=20)

# Known graphs
G = nx.karate_club_graph()
G = nx.petersen_graph()
```

**Random Networks**:
```python
# Erdős-Rényi random graph
G = nx.erdos_renyi_graph(n=100, p=0.1, seed=42)

# Barabási-Albert scale-free network
G = nx.barabasi_albert_graph(n=100, m=3, seed=42)

# Watts-Strogatz small-world network
G = nx.watts_strogatz_graph(n=100, k=6, p=0.1, seed=42)
```

**Structured Networks**:
```python
# Grid graph
G = nx.grid_2d_graph(m=5, n=7)

# Random tree
G = nx.random_tree(n=100, seed=42)
```

**Reference**: See `references/generators.md` for comprehensive coverage of all graph generators including classic, random, lattice, bipartite, and specialized network models with detailed parameters and use cases.

### 4. Reading and Writing Graphs

NetworkX supports numerous file formats and data sources:

**File Formats**:
```python
# Edge list
G = nx.read_edgelist('graph.edgelist')
nx.write_edgelist(G, 'graph.edgelist')

# GraphML (preserves attributes)
G = nx.read_graphml('graph.graphml')
nx.write_graphml(G, 'graph.graphml')

# GML
G = nx.read_gml('graph.gml')
nx.write_gml(G, 'graph.gml')

# JSON
data = nx.node_link_data(G)
G = nx.node_link_graph(data)
```

**Pandas Integration**:
```python
import pandas as pd

# From DataFrame
df = pd.DataFrame({'source': [1, 2, 3], 'target': [2, 3, 4], 'weight': [0.5, 1.0, 0.75]})
G = nx.from_pandas_edgelist(df, 'source', 'target', edge_attr='weight')

# To DataFrame
df = nx.to_pandas_edgelist(G)
```

**Matrix Formats**:
```python
import numpy as np

# Adjacency matrix
A = nx.to_numpy_array(G)
G = nx.from_numpy_array(A)

# Sparse matrix
A = nx.to_scipy_sparse_array(G)
G = nx.from_scipy_sparse_array(A)
```

**Reference**: See `references/io.md` for complete documentation on all I/O formats including CSV, SQL databases, Cytoscape, DOT, and guidance on format selection for different use cases.

### 5. Visualization

Create clear and informative network visualizations:

**Basic Visualization**:
```python
import matplotlib.pyplot as plt

# Simple draw
nx.draw(G, with_labels=True)
plt.show()

# With layout
pos = nx.spring_layout(G, seed=42)
nx.draw(G, pos=pos, with_labels=True, node_color='lightblue', node_size=500)
plt.show()
```

**Customization**:
```python
# Color by degree
node_colors = [G.degree(n) for n in G.nodes()]
nx.draw(G, node_color=node_colors, cmap=plt.cm.viridis)

# Size by centrality
centrality = nx.betweenness_centrality(G)
node_sizes = [3000 * centrality[n] for n in G.nodes()]
nx.draw(G, node_size=node_sizes)

# Edge weights
edge_widths = [3 * G[u][v].get('weight', 1) for u, v in G.edges()]
nx.draw(G, width=edge_widths)
```

**Layout Algorithms**:
```python
# Spring layout (force-directed)
pos = nx.spring_layout(G, seed=42)

# Circular layout
pos = nx.circular_layout(G)

# Kamada-Kawai layout
pos = nx.kamada_kawai_layout(G)

# Spectral layout
pos = nx.spectral_layout(G)
```

**Publication Quality**:
```python
plt.figure(figsize=(12, 8))
pos = nx.spring_layout(G, seed=42)
nx.draw(G, pos=pos, node_color='lightblue', node_size=500,
        edge_color='gray', with_labels=True, font_size=10)
plt.title('Network Visualization', fontsize=16)
plt.axis('off')
plt.tight_layout()
plt.savefig('network.png', dpi=300, bbox_inches='tight')
plt.savefig('network.pdf', bbox_inches='tight')  # Vector format
```

**Reference**: See `references/visualization.md` for extensive documentation on visualization techniques including layout algorithms, customization options, interactive visualizations with Plotly and PyVis, 3D networks, and publication-quality figure creation.

## Working with NetworkX

### Installation

Ensure NetworkX is installed:
```python
# Check if installed
import networkx as nx
print(nx.__version__)

# Install if needed (via bash)
# uv pip install networkx
# uv pip install networkx[default]  # With optional dependencies
```

### Common Workflow Pattern

Most NetworkX tasks follow this pattern:

1. **Create or Load Graph**:
   ```python
   # From scratch
   G = nx.Graph()
   G.add_edges_from([(1, 2), (2, 3), (3, 4)])

   # Or load from file/data
   G = nx.read_edgelist('data.txt')
   ```

2. **Examine Structure**:
   ```python
   print(f"Nodes: {G.number_of_nodes()}")
   print(f"Edges: {G.number_of_edges()}")
   print(f"Density: {nx.density(G)}")
   print(f"Connected: {nx.is_connected(G)}")
   ```

3. **Analyze**:
   ```python
   # Compute metrics
   degree_cent = nx.degree_centrality(G)
   avg_clustering = nx.average_clustering(G)

   # Find paths
   path = nx.shortest_path(G, source=1, target=4)

   # Detect communities
   communities = community.greedy_modularity_communities(G)
   ```

4. **Visualize**:
   ```python
   pos = nx.spring_layout(G, seed=42)
   nx.draw(G, pos=pos, with_labels=True)
   plt.show()
   ```

5. **Export Results**:
   ```python
   # Save graph
   nx.write_graphml(G, 'analyzed_network.graphml')

   # Save metrics
   df = pd.DataFrame({
       'node': list(degree_cent.keys()),
       'centrality': list(degree_cent.values())
   })
   df.to_csv('centrality_results.csv', index=False)
   ```

### Important Considerations

**Floating Point Precision**: When graphs contain floating-point numbers, all results are inherently approximate due to precision limitations. This can affect algorithm outcomes, particularly in minimum/maximum computations.

**Memory and Performance**: Each time a script runs, graph data must be loaded into memory. For large networks:
- Use appropriate data structures (sparse matrices for large sparse graphs)
- Consider loading only necessary subgraphs
- Use efficient file formats (pickle for Python objects, compressed formats)
- Leverage approximate algorithms for very large networks (e.g., `k` parameter in centrality calculations)

**Node and Edge Types**:
- Nodes can be any hashable Python object (numbers, strings, tuples, custom objects)
- Use meaningful identifiers for clarity
- When removing nodes, all incident edges are automatically removed

**Random Seeds**: Always set random seeds for reproducibility in random graph generation and force-directed layouts:
```python
G = nx.erdos_renyi_graph(n=100, p=0.1, seed=42)
pos = nx.spring_layout(G, seed=42)
```

## Quick Reference

### Basic Operations
```python
# Create
G = nx.Graph()
G.add_edge(1, 2)

# Query
G.number_of_nodes()
G.number_of_edges()
G.degree(1)
list(G.neighbors(1))

# Check
G.has_node(1)
G.has_edge(1, 2)
nx.is_connected(G)

# Modify
G.remove_node(1)
G.remove_edge(1, 2)
G.clear()
```

### Essential Algorithms
```python
# Paths
nx.shortest_path(G, source, target)
nx.all_pairs_shortest_path(G)

# Centrality
nx.degree_centrality(G)
nx.betweenness_centrality(G)
nx.closeness_centrality(G)
nx.pagerank(G)

# Clustering
nx.clustering(G)
nx.average_clustering(G)

# Components
nx.connected_components(G)
nx.strongly_connected_components(G)  # Directed

# Community
community.greedy_modularity_communities(G)
```

### File I/O Quick Reference
```python
# Read
nx.read_edgelist('file.txt')
nx.read_graphml('file.graphml')
nx.read_gml('file.gml')

# Write
nx.write_edgelist(G, 'file.txt')
nx.write_graphml(G, 'file.graphml')
nx.write_gml(G, 'file.gml')

# Pandas
nx.from_pandas_edgelist(df, 'source', 'target')
nx.to_pandas_edgelist(G)
```

## Resources

This skill includes comprehensive reference documentation:

### references/graph-basics.md
Detailed guide on graph types, creating and modifying graphs, adding nodes and edges, managing attributes, examining structure, and working with subgraphs.

### references/algorithms.md
Complete coverage of NetworkX algorithms including shortest paths, centrality measures, connectivity, clustering, community detection, flow algorithms, tree algorithms, matching, coloring, isomorphism, and graph traversal.

### references/generators.md
Comprehensive documentation on graph generators including classic graphs, random models (Erdős-Rényi, Barabási-Albert, Watts-Strogatz), lattices, trees, social network models, and specialized generators.

### references/io.md
Complete guide to reading and writing graphs in various formats: edge lists, adjacency lists, GraphML, GML, JSON, CSV, Pandas DataFrames, NumPy arrays, SciPy sparse matrices, database integration, and format selection guidelines.

### references/visualization.md
Extensive documentation on visualization techniques including layout algorithms, customizing node and edge appearance, labels, interactive visualizations with Plotly and PyVis, 3D networks, bipartite layouts, and creating publication-quality figures.

## Additional Resources

- **Official Documentation**: https://networkx.org/documentation/latest/
- **Tutorial**: https://networkx.org/documentation/latest/tutorial.html
- **Gallery**: https://networkx.org/documentation/latest/auto_examples/index.html
- **GitHub**: https://github.com/networkx/networkx

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.