# RepoBlackbox by Catalayer

**AI coding agents move fast. Your repo needs a blackbox.**

[![CI](https://github.com/stephenywilson/RepoBlackbox/actions/workflows/ci.yml/badge.svg)](https://github.com/stephenywilson/RepoBlackbox/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Node >=18](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg)](package.json)

RepoBlackbox is a lightweight **safety, evaluation, and workflow layer** for Claude Code, Codex,
Cursor, Copilot, and other AI coding agents.

It wraps each AI coding session with scope definition, a repo snapshot, a post-run audit, a review
report, local benchmark tasks, and structured workflow prompts — so you always know exactly what the
agent changed and whether it stayed within bounds.

v0.2 adds **Agent Task Bench**: local benchmark tasks for testing whether AI coding agents can
complete realistic repo-maintenance tasks without touching forbidden files.

v0.3 adds **Agent Skill Packs**: structured, reusable, copy-paste-ready workflow prompts for Claude
Code, Codex, Cursor, Copilot, and other AI coding agents.

v0.4 adds **Custom User Skill Directories**: project-local skill packs in `.repoblackbox/skills/`.

*By [Catalayer](https://catalayer.com)*

![RepoBlackbox terminal demo](docs/assets/terminal-demo.svg)

---

## What RepoBlackbox catches

| Agent behavior | Detection |
|---|---|
| Edits package.json during a UI-only task | Scope violation and dependency risk |
| Touches .env or .env.* | Sensitive file flag, content never read |
| Modifies files outside --allow patterns | Out-of-scope warning |
| Changes auth, billing, API, or database files | Built-in HIGH risk |
| Deletes many files | Bulk deletion warning |
| Only edits declared allowed files | LOW risk |

---

## Quick Start

```bash
npm install -g repoblackbox
repoblackbox --help
```

### Use in your project

```bash
cd /path/to/your/project

repoblackbox init

repoblackbox scope \
  --task "Refactor homepage hero" \
  --allow "src/components/home/**,src/styles/tokens.css" \
  --forbid ".env,package.json,src/lib/auth/**"

repoblackbox snapshot "before claude task"

# Run Claude Code, Codex, or Cursor

repoblackbox audit
repoblackbox report
```

### Development install

To build from source or contribute:

```bash
git clone https://github.com/stephenywilson/RepoBlackbox
cd RepoBlackbox
npm install
npm run build
npm link
repoblackbox --help
```

---

## Core Workflow

```bash
# Initialize once per project
repoblackbox init

# Before each AI session, define scope
repoblackbox scope \
  --task "Refactor homepage hero" \
  --allow "src/components/home/**,src/styles/tokens.css" \
  --forbid ".env,package.json,src/lib/auth/**"

# Snapshot before the agent starts
repoblackbox snapshot "before claude task"

# Run Claude Code, Codex, or Cursor

# Audit what changed
repoblackbox audit

# Generate a review report
repoblackbox report
```

---

## Agent Task Bench

RepoBlackbox v0.2 added local benchmark tasks for evaluating whether AI coding agents can complete
realistic repo-maintenance tasks safely without touching forbidden files.

```bash
repoblackbox bench list
repoblackbox bench prepare readme-url-fix
# run your AI coding agent inside the prepared workspace
repoblackbox bench score readme-url-fix
repoblackbox bench report readme-url-fix
repoblackbox bench demo
```

Built-in tasks:

| Task | What it tests |
|---|---|
| readme-url-fix | Fix a wrong clone URL without touching forbidden files |
| package-version-sync | Sync CLI version output with package metadata |
| docs-toc-update | Add a missing README Table of Contents entry |
| security-cleanup | Remove unsafe-looking docs examples |
| forbidden-file-guard | Make a docs-only change without touching package, src, or env files |

Key properties:

- RepoBlackbox does not run AI agents automatically.
- Scoring is deterministic and runs on the local filesystem. No API keys required.
- bench demo is fully self-contained and requires no AI agent.

See [docs/agent-task-bench.md](https://github.com/stephenywilson/RepoBlackbox/blob/main/docs/agent-task-bench.md)
for the full reference.

---

## Agent Skill Packs

RepoBlackbox v0.3 added structured, reusable, copy-paste-ready workflow prompts for Claude Code,
Codex, Cursor, Copilot, and other AI coding agents.

```bash
repoblackbox skill list

repoblackbox skill show readme-audit

repoblackbox skill use github-release-polish \
  --var project_path=/path/to/repo \
  --var repo_url=https://github.com/user/repo \
  --var version=0.3.0

repoblackbox skill use readme-audit \
  --var project_path=/path/to/repo \
  --output .repoblackbox/skills/readme-audit.md
```

Built-in skills:

| Skill | Purpose |
|---|---|
| github-release-polish | Prepare an open-source repo for a GitHub release |
| readme-audit | Audit README clarity, install accuracy, and copy-paste correctness |
| repo-url-fix | Fix wrong repository URLs after a rename or ownership change |
| security-privacy-scan | Scan for private paths, API keys, and internal references |
| npm-package-release-check | Prepare a Node or TypeScript CLI for npm publishing without publishing |
| python-package-release-check | Prepare a Python CLI for PyPI release without publishing |
| cli-smoke-test | Add or improve CLI smoke tests |
| changelog-update | Update CHANGELOG for a new version |
| ui-screenshot-audit | Generate targeted polish from UI screenshots |
| agent-safe-refactor | Guide a constrained refactor with explicit boundaries |

Key properties:

- skill use renders a prompt and prints or writes it. It does not execute the prompt.
- No API keys required. No model providers are called.
- Skills are local Markdown templates with variable substitution.
- Paste the rendered output into Claude Code, Codex, or Cursor yourself.

See [docs/agent-skill-packs.md](https://github.com/stephenywilson/RepoBlackbox/blob/main/docs/agent-skill-packs.md)
for the full reference.

---

## Custom User Skill Directories

RepoBlackbox v0.4 supports local skill packs inside your project.

```bash
# Create .repoblackbox/skills/ and an example skill
repoblackbox skill init

# List only local skills
repoblackbox skill list --local

# Show a local skill
repoblackbox skill show example-custom-skill --local

# Use a local skill
repoblackbox skill use example-custom-skill \
  --var project_path=/path/to/repo \
  --var task="Refactor homepage hero" \
  --local
```

Key properties:

- Built-in skills live in the RepoBlackbox package (global).
- Local skills live in `.repoblackbox/skills/` (per-project).
- Local skills use the same Markdown + YAML frontmatter format as built-in skills.
- Local skills are resolved first — they can override a built-in skill with the same id.
- Use `--skills-dir <path>` to point to a custom directory instead of `.repoblackbox/skills/`.
- No API providers are called. RepoBlackbox only renders prompt text.

See [docs/custom-skills.md](https://github.com/stephenywilson/RepoBlackbox/blob/main/docs/custom-skills.md)
for the full reference.

---

## Commands

| Command | What it does |
|---|---|
| repoblackbox init | Create config and agent safety documents |
| repoblackbox scope | Define task boundaries with allow and forbid patterns |
| repoblackbox snapshot | Capture repo state before AI edits |
| repoblackbox audit | Compare current state against snapshot |
| repoblackbox report | Generate Markdown review report |
| repoblackbox bench list | List built-in benchmark tasks |
| repoblackbox bench prepare | Prepare a benchmark workspace |
| repoblackbox bench score | Score a completed benchmark task |
| repoblackbox bench report | Generate a benchmark report |
| repoblackbox bench demo | Run a self-contained bench demo |
| repoblackbox skill init | Create local skill directory and example skill |
| repoblackbox skill list | List built-in and local skills |
| repoblackbox skill list --local | List only local skills |
| repoblackbox skill list --built-in | List only built-in skills |
| repoblackbox skill list --all | List built-in and local skills explicitly |
| repoblackbox skill show | Show skill metadata and preview |
| repoblackbox skill use | Render a skill prompt with variables |

---

## Important Clarifications

**RepoBlackbox does not run AI agents automatically.**
It prepares workspaces, renders prompts, audits diffs, and generates reports.
You run the AI agent yourself.

**RepoBlackbox does not call external model providers.**
No API keys are required anywhere.

**RepoBlackbox does not read .env file contents.**
For .env and .env.* files, RepoBlackbox records only whether the file exists, its size, and its
modification time. It never reads or hashes the file content. This is by design.

**RepoBlackbox is not a security scanner.**
It does not scan for secrets, vulnerabilities, or malicious code. It tells you what the agent
changed, what was forbidden, and what needs human review.

**RepoBlackbox does not perform rollback yet.**
Safe rollback is planned for a future release. Automated rollback done poorly can cause data loss,
so current versions focus on scope, snapshot, audit, report, bench, and skill workflows.
Use git checkout -- path/to/file or git reset manually after reviewing the audit.

---

## What Gets Protected

By default, RepoBlackbox flags changes to:

| Category | Files |
|---|---|
| Secrets | .env, .env.* |
| Dependencies | package.json, lock files |
| Deployment | Dockerfile, vercel.json, netlify.toml |
| CI/CD | .github/workflows/**, .travis.yml |
| Auth | src/lib/auth/** |
| Billing | src/lib/billing/**, src/lib/stripe/** |
| API routes | app/api/**, pages/api/**, src/api/** |
| Database | prisma/**, migrations/**, database/** |
| Config | src/config/** |

You can customize the protected list in .repoblackbox/protected-files.json.

---

## Risk Levels

| Level | Meaning |
|---|---|
| LOW | Only allowed, normal files changed |
| MEDIUM | Dependency, package, config, or out-of-scope files changed |
| HIGH | Env, auth, billing, API, deployment, database, bulk deletion, or forbid-pattern match |

---

## RepoBlackbox Version Line

- v0.1: Safety workflow with scope, snapshot, audit, and report
- v0.2: Agent Task Bench
- v0.3: Agent Skill Packs
- v0.4: Custom user skill directories

Current version: v0.4.0

---

## Roadmap

Planned for future releases:

- Safe rollback workflow
- GitHub PR comment support
- CI gate mode
- Team policy files
- Optional HTML reports
- MCP integration exploration

---

## Documentation

| Doc | Description |
|---|---|
| docs/example-workflow.md | Full step-by-step workflow |
| docs/risk-model.md | Risk model and LOW / MEDIUM / HIGH rules |
| docs/ai-agent-rules.md | Agent rule files and task scope workflow |
| docs/agent-task-bench.md | Agent Task Bench task format and scoring |
| docs/agent-skill-packs.md | Skill Pack format, variables, and built-in skills |
| docs/custom-skills.md | Custom user skill directories, local override, --skills-dir |
| CHANGELOG.md | Version history |
| CONTRIBUTING.md | Setup, build, smoke test, and contribution guidelines |
| SECURITY.md | What RepoBlackbox does and does not do with files |

---

## License

Apache 2.0 — free to use, modify, and build upon.

© 2024-2026 Catalayer AI

---

*RepoBlackbox was built from real AI coding experience at Catalayer.*
*It is not just a template — it is a workflow.*