# 🚀 Scout Autonomous Agent  Features & Architecture Guide

Scout is an advanced, multi-modal autonomous agent platform built directly into FT-CLI. It bridges traditional developer terminal workflows with modern web automation and full operating-system desktop control (GUI).

---

## 📑 Table of Contents

1. [Architectural Overview](#architectural-overview)
2. [Desktop & Computer-Use (GUI Control)](#1-desktop--computer-use-gui-control)
3. [Browser Automation & Web Scraping](#2-browser-automation--web-scraping)
4. [Multi-Step Planning & Cognitive Scratchpad](#3-multi-step-planning--cognitive-scratchpad)
5. [Action Verification & Self-Healing Engine](#4-action-verification--self-healing-engine)
6. [Safety & Human-in-the-Loop Risk Classifier](#5-safety--human-in-the-loop-risk-classifier)
7. [Developer & Workspace Baseline Operations](#6-developer--workspace-baseline-operations)
8. [Subscription Tier Capabilities (Free vs Pro / Enterprise)](#subscription-tier-capabilities)

---

## Architectural Overview

Scout executes a continuous **Perceive ➔ Plan ➔ Act ➔ Verify ➔ Recover** operational loop:

```mermaid
flowchart TD
    Goal[User High-Level Objective] --> Planner[TaskPlanner & DAG Decomposition]
    Planner --> Scratchpad[Cognitive Scratchpad & State Checkpoint]
    Scratchpad --> SelectTool[Tool Selection Hierarchy]
    SelectTool --> RiskCheck{RiskClassifier & Safety Gate}
    RiskCheck -- High/Critical Risk --> Approval[ApprovalManager: Human Confirmation]
    Approval -- Approved --> Dispatcher[Agent Runtime Dispatcher]
    RiskCheck -- Low/Medium Risk --> Dispatcher
    Dispatcher --> Execute[Execution: Native Computer / Browser / CLI Tools]
    Execute --> Verify[ActionVerifier: Post-Execution Verification]
    Verify -- Success --> Checkpoint[Save Task State & Next Step]
    Verify -- Failure --> Recovery[Error Recovery Strategy: Retry / Alternative]
    Recovery --> Dispatcher
    Checkpoint --> Done{All Steps Complete?}
    Done -- No --> Scratchpad
    Done -- Yes --> Complete[Task Complete Notification]
```

---

## 1. 🖥️ Desktop & Computer-Use (GUI Control)

Scout provides deep operating system control powered by the high-performance native `scout_driver.exe` engine (Win32 API interop) and Node.js controller integration (`SystemComputerController`).

### Core Features

- **Multi-Monitor Screen Perception:**
  - `computer_screenshot` / `screenshot`: Captures high-resolution screen frames to perceive visible windows, buttons, and visual state.
- **Precision Mouse Automation:**
  - `computer_click`: Sends left, right, or middle mouse clicks to exact screen coordinates `(x, y)`.
  - `computer_double_click`: Opens files, selects words, or triggers desktop shortcuts via double-click.
  - `computer_mouse_move`: Translates the cursor smoothly or instantly across display bounds.
  - `computer_mouse_down` & `computer_mouse_up`: Granular mouse button holds and releases for complex interactions.
  - `computer_drag`: Drags items from `(fromX, fromY)` to destination coordinates `(toX, toY)`.
  - `computer_scroll`: Dispatches vertical and horizontal mouse-wheel scroll deltas.
- **Keyboard & Shortcut Dispatch:**
  - `computer_type`: Emits text strings directly into focused input controls or editor windows.
  - `computer_key_press`: Presses single functional keys (Enter, Escape, Tab, Backspace, Arrow keys, etc.).
  - `computer_hotkey`: Emits multi-key chords (e.g. `["Control", "s"]` to save, `["Alt", "F4"]` to close, `["Alt", "Tab"]` to switch tasks).
- **Application & Window Orchestration:**
  - `computer_open_app`: Launches desktop applications (VS Code, Notepad, File Explorer, Terminal, etc.).
  - `computer_close_app`: Gracefully terminates or closes targeted desktop processes and windows.
  - `computer_switch_window`: Brings specific applications into foreground focus by matching title or process.
  - `computer_inspect_ui`: Enumerates active open desktop windows and highlights the current foreground window.
  - `computer_wait`: Pauses execution (100ms - 30,000ms) to allow GUI windows, animations, and transitions to settle.
  - `computer_diagnostic`: Collects display resolution, DPI scaling metrics, and driver connection status.

---

## 2. 🌐 Browser Automation & Web Scraping

Implemented in [`browserController.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/browser/browserController.ts), Scout includes an intelligent browser automation controller capable of navigating websites, analyzing semantic DOM trees, and extracting data.

### Core Features

- **Page Navigation & State:**
  - `browser_navigate`: Loads HTTP/HTTPS URLs, handles navigation history, and constructs interactive DOM trees.
  - `browser_inspect`: Discovers interactive elements (links, buttons, form fields) and assigns lightweight reference IDs (e.g., `[ref=1]`).
- **Web Form Interaction:**
  - `browser_click`: Clicks web elements identified by CSS selector, visible label text, or ref ID.
  - `browser_type`: Enters query strings or credentials into active form fields and textareas.
  - `browser_select`: Selects options from dropdown menus (`<select>`) by visible text or value.
  - `browser_scroll`: Scrolls viewport up or down to trigger infinite scroll or lazy-loaded assets.
- **Data Scraping & File Downloads:**
  - `browser_content`: Strips out HTML markup and extracts clean structured text, tables, and article content.
  - `browser_screenshot`: Captures a visual snapshot of the current web page viewport.
  - `browser_download`: Automatically downloads invoices, reports, PDFs, or files from web links and saves them directly to the workspace `downloads/` directory.

#### Example Implementation (`browserController.ts`):
```typescript
// Content Extraction & Document Download
public async extract_content(selector?: string): Promise<BrowserActionResult> {
    const page = this.mockPages.get(this.currentUrl);
    const text = page
        ? page.html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim()
        : `Content of ${this.currentUrl}`;
    return { success: true, output: text, url: this.currentUrl, title: this.currentTitle };
}

public async download_file(urlOrSelector: string, destPath?: string): Promise<BrowserActionResult> {
    // Discovers links, downloads files, and writes to workspace disk
    const fileName = path.basename(targetUrl).includes('.') ? path.basename(targetUrl) : 'downloaded_invoice.pdf';
    const finalPath = destPath ? path.resolve(this.workspaceDir, destPath) : path.resolve(this.workspaceDir, 'downloads', fileName);
    fs.mkdirSync(path.dirname(finalPath), { recursive: true });
    // Stream data and persist to disk...
}
```

---

## 3. 🧠 Multi-Step Planning & Cognitive Scratchpad

Complex tasks require structured reasoning, plan tracking, and persistent state across agent steps.

- **Dynamic Task Planning (`TaskPlanner`):**
  - Parses open-ended goals into structured steps with explicit dependencies.
  - Dynamically adapts when intermediate steps succeed, fail, or reveal new context.
- **Cognitive Scratchpad Memory (`scratchpad`):**
  - Exclusive to **Pro & Enterprise** tiers.
  - Provides working memory via `write`, `read`, `append`, and `clear` operations.
  - Persisted locally to `.ft/scratchpad.md` to prevent LLM context overflow during long executions.
- **Task State Checkpoints (`TaskStateManager`):**
  - Saves full task execution histories and artifact logs to `.ft/tasks/<task_id>.json`.
  - Enables clean task resumption, history reviews, and rollback capabilities.

---

## 4. 🔍 Action Verification & Self-Healing Engine

Scout doesn't simply fire tools and assume success. Every action is verified through [`actionVerifier.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/actionVerifier.ts).

### Verification Types
1. `file_exists`: Verifies target file is present on the filesystem and contains non-empty contents.
2. `app_active`: Verifies that an opened application is running and holds foreground focus.
3. `url_matches`: Verifies the browser navigated to the expected target domain/path.
4. `element_exists`: Confirms targeted DOM elements rendered on the page.

### Autonomous Recovery Strategy
When verification fails:
- **`RETRY`**: Re-attempts the action if caused by transient loading latency.
- **`ALTERNATIVE`**: Selects a secondary tool or selector if the primary approach is blocked.
- **`ABORT`**: Gracefully yields to the user if irreversible roadblocks occur.

### Intent Preservation
If a user requests GUI action (e.g. *"Open Notepad and type my notes"*), Scout strictly honors the intent and will **not** cheat by performing silent background file writes unless explicitly permitted.

---

## 5. 🛡️ Safety & Human-in-the-Loop Risk Classifier

Implemented in [`safety.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/safety.ts), Scout protects user systems from accidental disruption or destructive operations.

| Risk Level | Triggering Tools / Patterns | Behavior |
| :--- | :--- | :--- |
| **LOW** | `read_file`, `browser_inspect`, `computer_screenshot`, `file_info` | Auto-executed silently. |
| **MEDIUM** | `write_file`, `edit_file`, `browser_type`, `browser_click` | Executed with standard logging. |
| **HIGH** | `computer_close_app`, destructive hotkeys (`Alt+F4`), process termination | **Execution pauses.** Modal/CLI prompt requires user approval. |
| **CRITICAL** | System-wide deletion (`rmdir /s /q`, formatting), registry edits | **Blocked / Strict explicit consent required.** |

---

## 6. 💼 Developer & Workspace Baseline Operations

In addition to autonomous GUI and web tools, Scout provides a full suite of developer tools:

- **File Operations:** `read_file`, `write_file`, `edit_file`, `multi_edit_file`, `file_info`.
- **Search & Inspection:** `grep_search`, `glob_search`, `tree_view`, `list_dir`.
- **Command & Version Control:** `run_command`, `git_diff`.
- **Interaction:** `ask_user`, `task_completed`.

---

## Subscription Tier Capabilities

| Feature Capability | Free Tier | Pro Plan | Enterprise Plan |
| :--- | :---: | :---: | :---: |
| **Code Reading & File Editing** | ✅ Full | ✅ Full | ✅ Full |
| **File / Directory Creation** | ✅ Full | ✅ Full | ✅ Full |
| **Terminal & Build Commands** | ✅ Full | ✅ Full | ✅ Full |
| **AI Cognitive Scratchpad** | ❌ Blocked | ✅ Full | ✅ Full |
| **Autonomous Browser Automation** | ❌ Blocked | ✅ Full | ✅ Full |
| **Autonomous Computer & GUI Control** | ❌ Blocked | ✅ Full | ✅ Full |
| **Multi-App System Automation** | ❌ Blocked | ✅ Full | ✅ Full |
| **Dedicated Concurrency & Support** | Community | Priority | 24/7 Dedicated |

---

*Authored by Front Terrain Engineering. For upgrades or enterprise deployment, visit [https://scout.frontterrain.com](https://scout.frontterrain.com).*