---
title: Fallback Memory Manager Algorithm
summary: Primary Redis + fallback search architecture with circuit breaker integration and cache eviction strategies
---

# Fallback Memory Manager Algorithm

## Overview

The Fallback Memory Manager provides resilient memory search by automatically failing over from a primary QMD (Quadro Memory Database) backend to a local SQLite-based index when the primary becomes unavailable.

**Location:** `src/memory/search-manager.ts`

### Key Features

- **Dual-backend architecture** - Primary QMD + fallback SQLite index
- **Automatic failover** - Seamless switch on primary failure
- **Circuit breaker pattern** - Prevents cascade failures
- **Cache eviction** - Removes failed managers from cache
- **Transparent recovery** - Retries primary on next request

---

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│  getMemorySearchManager()                               │
│                                                         │
│  ┌───────────────────────────────────────────────────┐ │
│  │  Config Resolution                                │ │
│  │  - Resolve backend: "qmd" | "sqlite" | "none"     │ │
│  │  - Validate QMD config if selected                │ │
│  └─────────────────┬─────────────────────────────────┘ │
│                  │                                     │
│          Backend = "qmd"?                             │
│             │                                          │
│           YES ─────────────────────────────┐          │
│             │                              │          │
│             ▼                              │          │
│  ┌─────────────────────────┐              │          │
│  │ QmdMemoryManager        │              │          │
│  │ .create()              │              │          │
│  └───────────┬─────────────┘              │          │
│              │                             │          │
│        Success?                           │          │
│          │                                │          │
│      ┌───┴───┐                           │          │
│     YES     NO                          │          │
│      │       │                          │          │
│      │       ▼                          │          │
│      │  ┌─────────────────┐            │          │
│      │  │ Log warning     │            │          │
│      │  │ Fall through    │            │          │
│      │  └────────┬────────┘            │          │
│      │           │                     │          │
│      ▼           ▼                     │          │
│  ┌─────────────────────────┐           │          │
│  │ FallbackMemoryManager   │◄──────────┘          │
│  │ - primary: QMD          │                      │
│  │ - fallback: SQLite      │                      │
│  └───────────┬─────────────┘                      │
│              │                                    │
│              ▼                                    │
│  ┌─────────────────────────┐                     │
│  │ MemoryIndexManager      │◄────────────────────┘
│  │ (SQLite builtin)        │
│  └─────────────────────────┘
└─────────────────────────────────────────────────────────┘
```

---

## Fallback Memory Manager Implementation

### Data Structures

```typescript
class FallbackMemoryManager implements MemorySearchManager {
  private fallback: MemorySearchManager | null = null;
  private primaryFailed = false; // Circuit breaker flag
  private lastError?: string; // Last error from primary
  private cacheEvicted = false; // Eviction tracking

  constructor(
    private readonly deps: {
      primary: MemorySearchManager;
      fallbackFactory: () => Promise<MemorySearchManager | null>;
    },
    private readonly onClose?: () => void, // Cache eviction callback
  ) {}
}
```

### Circuit Breaker Pattern

```typescript
// Circuit breaker states:
// CLOSED (primaryFailed=false) → Normal operation, use primary
// OPEN (primaryFailed=true)    → Primary failed, use fallback

class FallbackMemoryManager {
  async search(
    query: string,
    opts?: { maxResults?: number; minScore?: number; sessionKey?: string },
  ) {
    // Circuit CLOSED: Try primary first
    if (!this.primaryFailed) {
      try {
        return await this.deps.primary.search(query, opts);
      } catch (err) {
        // Transition to OPEN state
        this.primaryFailed = true;
        this.lastError = err instanceof Error ? err.message : String(err);

        log.warn(`qmd memory failed; switching to builtin index: ${this.lastError}`);

        // Cleanup failed primary
        await this.deps.primary.close?.().catch(() => {});

        // Evict from cache to allow fresh retry on next request
        this.evictCacheEntry();
      }
    }

    // Circuit OPEN: Use fallback
    const fallback = await this.ensureFallback();
    if (fallback) {
      return await fallback.search(query, opts);
    }

    throw new Error(this.lastError ?? "memory search unavailable");
  }
}
```

### State Machine

```
┌─────────────────────────────────────────────────────────┐
│  FallbackMemoryManager State Machine                    │
└─────────────────────────────────────────────────────────┘

         ┌─────────────────────────────────────────┐
         │                                         │
         ▼                                         │
    ┌─────────┐                                   │
    │ CLOSED  │ ── primary.search() succeeds ──►  │
    │ (normal)│                                    │
    └────┬────┘                                    │
         │                                         │
         │ primary.search() throws                 │
         │                                         │
         ▼                                         │
    ┌─────────┐                                    │
    │  OPEN   │                                    │
    │ (failed)│                                    │
    └────┬────┘                                    │
         │                                         │
         │ 1. Log error                            │
         │ 2. Close primary                        │
         │ 3. Evict cache                          │
         │ 4. Initialize fallback                  │
         │                                         │
         ▼                                         │
    ┌─────────┐                                    │
    │FALLBACK │ ◄─────────────────────────────────┘
    │(using   │
    │ SQLite) │
    └─────────┘

Next request creates fresh manager, retrying QMD
```

### Method Implementations

#### Search Operation

```typescript
async search(
  query: string,
  opts?: { maxResults?: number; minScore?: number; sessionKey?: string },
) {
  // Try primary (QMD) if circuit is closed
  if (!this.primaryFailed) {
    try {
      return await this.deps.primary.search(query, opts);
    } catch (err) {
      // Transition to open state
      this.primaryFailed = true;
      this.lastError = err instanceof Error ? err.message : String(err);

      log.warn(`qmd memory failed; switching to builtin: ${this.lastError}`);

      // Cleanup
      await this.deps.primary.close?.().catch(() => {});
      this.evictCacheEntry();
    }
  }

  // Use fallback (SQLite)
  const fallback = await this.ensureFallback();
  if (fallback) {
    return await fallback.search(query, opts);
  }

  throw new Error(this.lastError ?? "memory search unavailable");
}
```

#### Read File Operation

```typescript
async readFile(params: {
  relPath: string;
  from?: number;
  lines?: number;
}) {
  if (!this.primaryFailed) {
    return await this.deps.primary.readFile(params);
  }

  const fallback = await this.ensureFallback();
  if (fallback) {
    return await fallback.readFile(params);
  }

  throw new Error(this.lastError ?? "memory read unavailable");
}
```

#### Status Operation

```typescript
status() {
  if (!this.primaryFailed) {
    return this.deps.primary.status();
  }

  // Primary failed - report fallback state
  const fallbackStatus = this.fallback?.status();
  const fallbackInfo = {
    from: "qmd",
    reason: this.lastError ?? "unknown"
  };

  if (fallbackStatus) {
    const custom = fallbackStatus.custom ?? {};
    return {
      ...fallbackStatus,
      fallback: fallbackInfo,
      custom: {
        ...custom,
        fallback: {
          disabled: true,
          reason: this.lastError ?? "unknown"
        },
      },
    };
  }

  // Fallback not initialized yet
  const primaryStatus = this.deps.primary.status();
  const custom = primaryStatus.custom ?? {};
  return {
    ...primaryStatus,
    fallback: fallbackInfo,
    custom: {
      ...custom,
      fallback: {
        disabled: true,
        reason: this.lastError ?? "unknown"
      },
    },
  };
}
```

#### Sync Operation

```typescript
async sync(params?: {
  reason?: string;
  force?: boolean;
  progress?: (update: MemorySyncProgressUpdate) => void;
}) {
  if (!this.primaryFailed) {
    await this.deps.primary.sync?.(params);
    return;
  }

  const fallback = await this.ensureFallback();
  await fallback?.sync?.(params);
}
```

#### Probe Operations

```typescript
async probeEmbeddingAvailability(): Promise<MemoryEmbeddingProbeResult> {
  if (!this.primaryFailed) {
    return await this.deps.primary.probeEmbeddingAvailability();
  }

  const fallback = await this.ensureFallback();
  if (fallback) {
    return await fallback.probeEmbeddingAvailability();
  }

  return {
    ok: false,
    error: this.lastError ?? "memory embeddings unavailable"
  };
}

async probeVectorAvailability() {
  if (!this.primaryFailed) {
    return await this.deps.primary.probeVectorAvailability();
  }

  const fallback = await this.ensureFallback();
  return (await fallback?.probeVectorAvailability()) ?? false;
}
```

### Fallback Initialization

```typescript
private async ensureFallback(): Promise<MemorySearchManager | null> {
  // Return cached fallback if available
  if (this.fallback) {
    return this.fallback;
  }

  let fallback: MemorySearchManager | null;
  try {
    // Lazy initialization via factory
    fallback = await this.deps.fallbackFactory();

    if (!fallback) {
      log.warn("memory fallback requested but builtin index is unavailable");
      return null;
    }
  } catch (err) {
    const message = err instanceof Error ? err.message : String(err);
    log.warn(`memory fallback unavailable: ${message}`);
    return null;
  }

  this.fallback = fallback;
  return this.fallback;
}
```

### Cache Eviction

```typescript
private evictCacheEntry(): void {
  if (this.cacheEvicted) {
    return;  // Idempotent
  }

  this.cacheEvicted = true;
  this.onClose?.();  // Remove from QMD_MANAGER_CACHE
}
```

---

## Cache Management

### QMD Manager Cache

```typescript
// Global cache for QMD managers
const QMD_MANAGER_CACHE = new Map<string, MemorySearchManager>();

function buildQmdCacheKey(agentId: string, config: ResolvedQmdConfig): string {
  return `${agentId}:${stableSerialize(config)}`;
}

// Stable serialization ensures consistent keys
function stableSerialize(value: unknown): string {
  return JSON.stringify(sortValue(value));
}

function sortValue(value: unknown): unknown {
  if (Array.isArray(value)) {
    return value.map((entry) => sortValue(entry));
  }
  if (value && typeof value === "object") {
    const sortedEntries = Object.keys(value as Record<string, unknown>)
      .toSorted((a, b) => a.localeCompare(b))
      .map((key) => [key, sortValue((value as Record<string, unknown>)[key])]);
    return Object.fromEntries(sortedEntries);
  }
  return value;
}
```

### Cache Lifecycle

```
┌─────────────────────────────────────────────────────────┐
│  Cache Lifecycle                                        │
└─────────────────────────────────────────────────────────┘

Request 1: getMemorySearchManager(agentId="main")
  │
  ├─► Cache miss
  ├─► Create QmdMemoryManager
  ├─► Wrap in FallbackMemoryManager
  └─► Cache.set(key, manager)

Request 2: getMemorySearchManager(agentId="main")
  │
  ├─► Cache hit
  └─► Return cached manager (QMD primary)

Request 3: search() throws error
  │
  ├─► FallbackMemoryManager.primaryFailed = true
  ├─► FallbackMemoryManager.evictCacheEntry()
  └─► Cache.delete(key)

Request 4: getMemorySearchManager(agentId="main")
  │
  ├─► Cache miss (entry was evicted)
  ├─► Create fresh QmdMemoryManager (retry QMD)
  └─► Cache.set(key, newManager)
```

---

## Manager Creation Flow

### Main Entry Point

```typescript
export async function getMemorySearchManager(params: {
  cfg: SophiaClawConfig;
  agentId: string;
  purpose?: "default" | "status";
}): Promise<MemorySearchManagerResult> {
  const resolved = resolveMemoryBackendConfig(params);

  // Try QMD backend first
  if (resolved.backend === "qmd" && resolved.qmd) {
    const statusOnly = params.purpose === "status";
    const cacheKey = buildQmdCacheKey(params.agentId, resolved.qmd);

    // Check cache (skip for status checks)
    if (!statusOnly) {
      const cached = QMD_MANAGER_CACHE.get(cacheKey);
      if (cached) {
        return { manager: cached };
      }
    }

    try {
      const { QmdMemoryManager } = await import("./qmd-manager.js");
      const primary = await QmdMemoryManager.create({
        cfg: params.cfg,
        agentId: params.agentId,
        resolved,
        mode: statusOnly ? "status" : "full",
      });

      if (primary) {
        if (statusOnly) {
          return { manager: primary };
        }

        // Wrap with fallback
        const wrapper = new FallbackMemoryManager(
          {
            primary,
            fallbackFactory: async () => {
              const { MemoryIndexManager } = await import("./manager.js");
              return await MemoryIndexManager.get(params);
            },
          },
          () => QMD_MANAGER_CACHE.delete(cacheKey),
        );

        QMD_MANAGER_CACHE.set(cacheKey, wrapper);
        return { manager: wrapper };
      }
    } catch (err) {
      const message = err instanceof Error ? err.message : String(err);
      log.warn(`qmd memory unavailable; falling back to builtin: ${message}`);
      // Fall through to builtin index
    }
  }

  // Use builtin SQLite index
  try {
    const { MemoryIndexManager } = await import("./manager.js");
    const manager = await MemoryIndexManager.get(params);
    return { manager };
  } catch (err) {
    const message = err instanceof Error ? err.message : String(err);
    return { manager: null, error: message };
  }
}
```

---

## Error Handling

### Error Categories

```typescript
// QMD-specific errors that trigger fallback
const QMD_ERROR_PATTERNS = [
  /connection refused/i,
  /timeout/i,
  /unavailable/i,
  /not found/i,
  /permission denied/i,
];

function isQmdError(err: Error): boolean {
  return QMD_ERROR_PATTERNS.some((pattern) => pattern.test(err.message));
}
```

### Logging Strategy

```typescript
// Log at appropriate levels
log.warn(`qmd memory failed; switching to builtin: ${errorMessage}`);
log.warn(`memory fallback unavailable: ${errorMessage}`);

// Include context
log.warn({
  agentId: params.agentId,
  backend: "qmd",
  error: errorMessage,
  timestamp: Date.now(),
});
```

---

## Testing Strategies

### Unit Tests

```typescript
describe("FallbackMemoryManager", () => {
  it("uses primary when available", async () => {
    const primary = createMockManager();
    const manager = new FallbackMemoryManager({
      primary,
      fallbackFactory: async () => createMockManager(),
    });

    await manager.search("test query");

    expect(primary.search).toHaveBeenCalledWith("test query");
  });

  it("falls back when primary fails", async () => {
    const primary = createMockManager();
    primary.search.mockRejectedValue(new Error("connection refused"));

    const fallback = createMockManager();
    const manager = new FallbackMemoryManager({
      primary,
      fallbackFactory: async () => fallback,
    });

    await manager.search("test query");

    expect(fallback.search).toHaveBeenCalledWith("test query");
  });

  it("evicts cache on primary failure", async () => {
    const evictCallback = jest.fn();
    const primary = createMockManager();
    primary.search.mockRejectedValue(new Error("timeout"));

    const manager = new FallbackMemoryManager(
      {
        primary,
        fallbackFactory: async () => createMockManager(),
      },
      evictCallback,
    );

    await manager.search("test query");

    expect(evictCallback).toHaveBeenCalled();
  });

  it("reports fallback in status", () => {
    const primary = createMockManager();
    primary.search.mockRejectedValue(new Error("unavailable"));

    const manager = new FallbackMemoryManager({
      primary,
      fallbackFactory: async () => createMockManager(),
    });

    // Trigger failure
    manager.search("test").catch(() => {});

    const status = manager.status();
    expect(status.fallback).toBeDefined();
    expect(status.fallback.from).toBe("qmd");
  });
});
```

### Integration Tests

```typescript
describe("getMemorySearchManager integration", () => {
  it("creates fallback wrapper for QMD config", async () => {
    const result = await getMemorySearchManager({
      cfg: {
        memory: {
          backend: "qmd",
          qmd: {
            /* config */
          },
        },
      },
      agentId: "test",
    });

    expect(result.manager).toBeInstanceOf(FallbackMemoryManager);
  });

  it("uses builtin index when QMD unavailable", async () => {
    const result = await getMemorySearchManager({
      cfg: { memory: { backend: "none" } },
      agentId: "test",
    });

    expect(result.manager).toBeInstanceOf(MemoryIndexManager);
  });
});
```

---

## Performance Considerations

### Lazy Fallback Initialization

```typescript
// Fallback is only created when primary fails
// This avoids unnecessary SQLite initialization overhead
private async ensureFallback(): Promise<MemorySearchManager | null> {
  if (this.fallback) {
    return this.fallback;  // Return cached
  }

  // Create on-demand
  this.fallback = await this.deps.fallbackFactory();
  return this.fallback;
}
```

### Cache Key Stability

```typescript
// Stable serialization prevents cache fragmentation
function stableSerialize(value: unknown): string {
  return JSON.stringify(sortValue(value));
}

// Ensures these configs produce same cache key:
// { host: "localhost", port: 8080 }
// { port: 8080, host: "localhost" }
```

### Memory Footprint

- **Cache entry:** ~1-5 MB per agent (QMD connection + state)
- **Fallback entry:** ~10-50 MB per agent (SQLite DB + indexes)
- **Eviction:** Immediate on primary failure

---

## Related Documentation

- [Index Manager](/algorithms/memory-search/index-manager)
- [QMD Backend](/algorithms/memory-search/qmd-backend)
- [Memory Configuration](/memory/configuration)
- [Memory Search](/memory/search)