---
name: test-connector
description: |
  Trigger this skill when the user wants to test a connector:

  **Triggers:**
  - "test the connector"
  - "test [provider] connector"
  - "run tests for [provider]"
  - "test all actions"
  - "verify connector works"
  - "test connector with [auth type]"
  - Any request to test/verify connector functionality

  **What it does:**
  1. Creates action tracking file (`.connector-dev/<provider>/testing/actions_tracking.json`)
  2. Tests every action with every auth type systematically
  3. Updates tracking file after each test
  4. Handles failures and retries
  5. Verifies 100% completion before finishing
  6. Provides real-time progress updates

  **Key principle:** Follows the "Leave no trace" testing ideology - only tests with read-only actions OR cleans up all test data created.
sdkVersion: 3.18.1
features:
  authTypes: [ oauth2 ]
---

# Connector Testing Skill

## Overview

This skill automates the complete testing workflow for connectors: every action is tested with every auth type, in dependency order (LIST → GET → CREATE → UPDATE → DELETE), with progress tracked to 100% completion and no test data left behind.

## Setup (REQUIRED - Collect ALL upfront)

| Required | Description |
|----------|-------------|
| **Provider** | Provider name (e.g., `bamboohr`, `datadog`) |
| **Connector Path** | Path to connector YAML file (e.g., `src/configs/bamboohr/bamboohr.connector.s1.yaml`) |
| **Auth Types** | List of auth types to test (e.g., `["oauth2", "api_key"]`) |
| **Credentials** | JSON files for each auth type (e.g., `oauth2_creds.json`, `api_key_creds.json`) |
| **Account JSON** | Account configuration: `{"environment": "production", "provider": "provider_name"}` |
| **Testing Mode** | `read-only` (list/get only) OR `full` (includes create/update/delete) |

### ⚠️ Testing Mode Decision

**CRITICAL:** First, ask the user if testing should be:
- **Read-only**: Only test list/get actions (safe, no data modification)
- **Full**: Test all actions including create/update/delete (requires cleanup)

**Testing ideology - Leave no trace:** The state and data in the system should be as before testing began. Any sample data generated should be cleaned up.

## Phase 1: Action Tracking Setup

### Step 1: Read Connector Configuration

```bash
# Read main connector file
cat src/configs/<provider>/<provider>.connector.s1.yaml

# Read all partial files
find src/configs/<provider> -name "*.partial.yaml" -exec cat {} \;
```

### Step 2: Extract All Actions

For each action found:
- Extract `actionId` (operation name)
- Extract `actionType` (LIST, GET, CREATE, UPDATE, DELETE)
- Extract endpoint path from steps
- Extract HTTP method
- Extract category

### Step 3: Create Action Tracking File

Create `.connector-dev/<provider>/testing/actions_tracking.json` with this structure:

```json
{
  "provider": "provider_name",
  "timestamp": "2025-01-21T10:30:00Z",
  "auth_types": ["oauth2", "api_key"],
  "total_actions": 15,
  "actions": [
    {
      "authKey": "oauth2",
      "operation_name": "list_employees",
      "operation_path": "/employees",
      "method": "GET",
      "actionType": "LIST",
      "category": "hris",
      "tested": false,
      "test_results": {},
      "comments": "",
      "auxiliaryFor": [],
      "capturedData": {}
    }
  ],
  "testing_progress": {
    "total_required_tests": 30,
    "tests_completed": 0,
    "percentage_complete": 0
  }
}
```

**Key points:**
- Create one entry per action per auth type (if 15 actions × 2 auth types = 30 entries)
- Order by auth type first, then by action order in connector
- Calculate `total_required_tests = total_actions × auth_types.length`
- Initialize all `tested: false` and `test_results: {}`

### Step 4: Verify Tracking File

```bash
# Verify structure
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '.'

# Verify counts
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '{total: .total_actions, auth_types: .auth_types, required_tests: .testing_progress.total_required_tests}'
```

## Phase 2: Systematic Testing Execution

### Testing Order (Dependency-Based Cycles)

**Cycle 1: LIST Actions (No Dependencies)**
- Test all LIST actions first
- Capture IDs, tokens, or other data needed by other actions
- Mark as auxiliary for dependent actions
- Example: `list_employees` → captures `employeeIds`

**Cycle 2: GET Actions (Use IDs from Cycle 1)**
- Test GET actions using IDs from LIST actions
- Verify individual resource retrieval
- Example: `get_employee` using `employeeId` from `list_employees`

**Cycle 3: CREATE Actions (Generate New Resources) - ONLY if full testing mode**
- Test CREATE actions to generate new resources
- Capture new IDs for UPDATE/DELETE actions
- Example: `create_room` → captures `newRoomId`

**Cycle 4: UPDATE Actions (Use IDs from Cycle 3) - ONLY if full testing mode**
- Test UPDATE actions using IDs from CREATE actions
- Verify modifications work correctly
- Example: `update_room` using `roomId` from `create_room`

**Cycle 5: DELETE Actions (Clean Up from Cycle 3) - ONLY if full testing mode**
- Test DELETE actions using IDs from CREATE actions
- Clean up test data
- Example: `delete_room` using `roomId` from `create_room`

### For Each Action in Tracking File

#### A. Read Input Configuration

```bash
# Find and read action inputs
grep -A 30 "actionId: <action_name>" src/configs/<provider>/*.partial.yaml
```

**Extract:**
- Required vs optional parameters (`required: true/false`)
- Parameter locations (`in` is REQUIRED on every input; valid: `path`, `query`, `body`, `headers`)
- Parameter types (`type: string/number/boolean`, `array: true`)

#### B. Prepare Input Parameters

Build params JSON based on input configuration:

```json
// Path params → {"path": {"id": "..."}}
// Query params → {"queryParams": {"max": 10, "filter": "active"}}
// Body params → {"body": {"title": "...", "description": "..."}}
// Mixed params → {"path": {"id": "..."}, "body": {"title": "..."}}
```

**Use captured data from auxiliary actions:**
- If testing `get_employee`, use `employeeId` from `list_employees.capturedData`
- If testing `update_room`, use `roomId` from `create_room.capturedData`

#### C. Test the Action Using CLI

```bash
stackone run --connector <connector_path> \
  --account <account_json_file> \
  --credentials <credentials_json_file> \
  --action-id <action_name> \
  --params '<params_json>' \
  [--debug]
```

#### D. Handle Test Results

**✅ If Action PASSES:**

1. Update tracking file:
   ```json
   {
     "test_results": {
       "oauth2": {
         "status": "success",
         "tested_at": "2025-01-21T10:35:00Z",
         "response_code": 200
       }
     }
   }
   ```

2. Capture useful data in `capturedData`:
   ```json
   {
     "capturedData": {
       "employeeIds": ["123", "456", "789"],
       "roomId": "abc123"
     }
   }
   ```

3. Add test scenarios to `comments` field

4. If this action provides data for other actions, mark it as auxiliary:
   ```json
   {
     "auxiliaryFor": ["get_employee", "update_employee", "delete_employee"]
   }
   ```

5. After testing with ALL auth types, mark `tested: true`

6. Update `testing_progress.tests_completed++` and `percentage_complete`

**❌ If Action FAILS:**

1. Analyze the error (400, 401, 404, 405, 500, etc.)

2. Determine fix strategy:
   | Error | Root Cause | Fix Action |
   |-------|-----------|------------|
   | **400 Bad Request** | Missing/wrong params | Read inputs config, fix parameter structure/type/location |
   | **401/403 Forbidden** | Missing auth scope | Add scope to credentials or document as admin-only |
   | **404 Not Found** | Invalid endpoint | Check provider API docs, fix URL or REMOVE action |
   | **405 Method Not Allowed** | Wrong HTTP method | Check provider docs, fix method or REMOVE action |
   | **500 Server Error** | Invalid request format | Check request body/headers, compare with working actions |

3. Apply fix to YAML configuration

4. Re-test immediately

5. Repeat until action passes OR is removed

6. Update tracking file only after resolution:
   ```json
   {
     "test_results": {
       "oauth2": {
         "status": "failed",
         "tested_at": "2025-01-21T10:35:00Z",
         "response_code": 404,
         "error": "Endpoint not found"
       }
     },
     "comments": "REMOVED - Endpoint deprecated by provider, returns 404"
   }
   ```

### Progress Monitoring

**After EACH test, verify:**

```bash
# Read tracking file
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '.testing_progress'

# Expected output:
{
  "total_required_tests": 30,
  "tests_completed": 15,  # Increases after each test
  "percentage_complete": 50
}
```

**Provide real-time updates to user:**
```
🧪 Testing Progress: 15/30 (50%)
✅ Passed: 14 | ❌ Failed: 1 | ⏭️ Skipped: 0
Current: Testing list_employees with api_key...
```

## Phase 3: Testing Completion Verification

**BEFORE finishing, verify:**

```bash
# Check completion percentage
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '.testing_progress.percentage_complete'
# MUST return: 100

# Check for untested actions
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '.actions[] | select(.tested == false)'
# MUST return: empty (no results)

# Check for failed tests
cat .connector-dev/<provider>/testing/actions_tracking.json | jq '.actions[].test_results[][] | select(.status == "failed")'
# MUST return: empty (no results)
```

**If ANY check fails, DO NOT PROCEED. Fix and re-test.**

## Phase 4: Output Report

Generate a comprehensive test report:

```markdown
## Test Results: <provider> connector

### Summary
- Total actions: 15
- Auth types tested: 2 (oauth2, api_key)
- Total required tests: 30
- Tests completed: 30 (100%)
- Tests passed: 28 (93%)
- Tests failed: 2 (7%)
- Actions removed: 0

### Testing Mode
- Mode: read-only / full
- Cleanup performed: Yes/No

### Test Results by Auth Type
| Auth Type | Tests | Passed | Failed |
|-----------|-------|--------|--------|
| oauth2    | 15    | 14     | 1      |
| api_key   | 15    | 14     | 1      |

### Failed Actions
| Action | Auth Type | Error | Resolution |
|--------|-----------|-------|------------|
| list_workspaces | oauth2 | 404 Not Found | REMOVED - Endpoint deprecated |
| get_workspace | api_key | 401 Unauthorized | Fixed - Added required scope |

### Actions Removed
| Action | Reason |
|--------|--------|
| list_workspaces | Endpoint deprecated by provider |

### Captured Data
- Employee IDs: 3 captured from list_employees
- Room IDs: 2 captured from create_room
- User IDs: 5 captured from list_users

### Testing Cycles Completed
- ✅ Cycle 1: LIST Actions (5 actions)
- ✅ Cycle 2: GET Actions (5 actions)
- ✅ Cycle 3: CREATE Actions (3 actions) - Full mode only
- ✅ Cycle 4: UPDATE Actions (1 action) - Full mode only
- ✅ Cycle 5: DELETE Actions (1 action) - Full mode only

### Files Modified
- src/configs/<provider>/<provider>.connector.s1.yaml
- src/configs/<provider>/<provider>.employees.s1.partial.yaml

### Tracking File
- Location: .connector-dev/<provider>/testing/actions_tracking.json
- Status: Complete (100%)
```

## Key Principles

1. **Leave no trace**: Clean up all test data created during full testing mode
2. **100% coverage**: Every action tested with every auth type
3. **Real-time updates**: Provide progress updates after each test
4. **Failure handling**: Fix and retry failed actions immediately
5. **Verification**: Verify 100% completion before finishing
6. **Documentation**: Document all failures, fixes, and removals

## Anti-Patterns (NEVER DO THESE)

- ❌ Testing operations with only one auth type when multiple are supported
- ❌ Skipping action tracking file creation
- ❌ Not updating tracking file during testing
- ❌ Proceeding without 100% test coverage
- ❌ Assuming success without verification against tracking file
- ❌ Testing DELETE actions without creating test data first
- ❌ Leaving test data in the system after testing