# 🔍 pi-search-multi — Backend Benchmark Report

**Date:** 2026-05-04
**Test queries:** "latest AI frameworks 2025", "climate change solutions", "python async programming best practices"
**Results requested per query:** 5

## 📊 Overall Summary

| #   | Backend           | Avg Time | Success | Results | Avg Score |
| --- | ----------------- | -------- | ------- | ------- | --------- |
| 1   | Tavily            | 356ms    | ✅ 100% | 15      | 3.7/10    |
| 2   | DuckDuckGo        | 1158ms   | ✅ 100% | 15      | 3.5/10    |
| 3   | Serper            | 667ms    | ✅ 100% | 15      | 3.5/10    |
| 4   | Firecrawl         | 644ms    | ✅ 100% | 15      | 3.5/10    |
| 5   | WebSearchAPI.ai   | 1323ms   | ✅ 100% | 15      | 3.5/10    |
| 6   | Exa               | 137ms    | ✅ 100% | 15      | 3.2/10    |
| 7   | LangSearch        | 1816ms   | ✅ 100% | 15      | 3.2/10    |
| 8   | Marginalia Search | 354ms    | ✅ 100% | 15      | 3.0/10    |
| 9   | Brave Search      | 460ms    | ✅ 100% | 15      | 3.3/10    |

## 📋 Detailed Per-Backend Results

**How Quality is scored:** Each result is evaluated for keyword relevance (query words matched in title/snippet), source diversity (penalty for generic search engines), and snippet completeness. The average per-result score is then normalized to a 0–10 scale.

### DuckDuckGo

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 1129ms | **Results:** 5 | **Score:** 3.5/10
- 🏷 **Sample:** "Top 10 AI Development Frameworks to Watch in 2025"

**Query:** "climate change solutions"

- ✅ **Time:** 972ms | **Results:** 5 | **Score:** 3.9/10
- 🏷 **Sample:** "Climate Change Solutions"

**Query:** "python async programming best practices"

- ✅ **Time:** 1374ms | **Results:** 5 | **Score:** 3.1/10
- 🏷 **Sample:** "Python's asyncio: A Hands-On Walkthrough – Real Python"

---

### Marginalia Search

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 530ms | **Results:** 5 | **Score:** 2.7/10
- 🏷 **Sample:** "NVIDIA AI Enterprise - NVIDIA Docs"

**Query:** "climate change solutions"

- ✅ **Time:** 263ms | **Results:** 5 | **Score:** 4/10
- 🏷 **Sample:** "Episode 80: Unleashing Our Imaginations for Climate Change Solutions  | Climate Change Podcast"

**Query:** "python async programming best practices"

- ✅ **Time:** 269ms | **Results:** 5 | **Score:** 2.3/10
- 🏷 **Sample:** "CUDA Best Practices Guide — CUDA C++ Best Practices Guide 13.2 documentation"

---

### Serper

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 699ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "Top AI Agent Frameworks in 2025: Honest Reviews and Real-World ..."

**Query:** "climate change solutions"

- ✅ **Time:** 702ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "What are the solutions to climate change? - Greenpeace UK"

**Query:** "python async programming best practices"

- ✅ **Time:** 601ms | **Results:** 5 | **Score:** 3.4/10
- 🏷 **Sample:** "Asyncio best practices - Async-SIG - Discussions on Python.org"

---

### Tavily

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 950ms | **Results:** 5 | **Score:** 3.5/10
- 🏷 **Sample:** "Top 10 Open-Source AI Frameworks You Should Try (2025 Edition)"

**Query:** "climate change solutions"

- ✅ **Time:** 83ms | **Results:** 5 | **Score:** 3.9/10
- 🏷 **Sample:** "10 Solutions for Climate Change | Scientific American"

**Query:** "python async programming best practices"

- ✅ **Time:** 34ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "Unlocking the Power of Asynchronous Programming in Python"

---

### Exa

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 128ms | **Results:** 5 | **Score:** 2.9/10
- 🏷 **Sample:** "Monarch: an API to your supercomputer – PyTorch"

**Query:** "climate change solutions"

- ✅ **Time:** 199ms | **Results:** 5 | **Score:** 3.5/10
- 🏷 **Sample:** "A Primer on Science‑Based Climate Solutions | Project Drawdown®"

**Query:** "python async programming best practices"

- ✅ **Time:** 83ms | **Results:** 5 | **Score:** 3.1/10
- 🏷 **Sample:** "Python Async/Await Playbook: Learn From Years of Production Failures"

---

### Brave Search

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 289ms | **Results:** 5 | **Score:** 3.5/10
- 🏷 **Sample:** "Top AI Frameworks in 2025: A Review"

**Query:** "climate change solutions"

- ✅ **Time:** 418ms | **Results:** 5 | **Score:** 3.5/10
- 🏷 **Sample:** "What are the solutions to climate change? - Greenpeace UK"

**Query:** "python async programming best practices"

- ✅ **Time:** 673ms | **Results:** 5 | **Score:** 3.0/10
- 🏷 **Sample:** "The Python Async/Await Guide: Best Practices for 2025"

---

### LangSearch

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 2359ms | **Results:** 5 | **Score:** 3.2/10
- 🏷 **Sample:** "ai 领域 开发 top 开源 产品 - github - 2025 \_ git 上 比较 火 的 ai 框架 - csdn 博客"

**Query:** "climate change solutions"

- ✅ **Time:** 1521ms | **Results:** 5 | **Score:** 3.2/10
- 🏷 **Sample:** "Climate Action Initiatives | United Nations"

**Query:** "python async programming best practices"

- ✅ **Time:** 1569ms | **Results:** 5 | **Score:** 3.3/10
- 🏷 **Sample:** "Mastering Asynchronous Programming in Python: Unlock the Secrets of Expert-Level Skills"

---

### Firecrawl

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 891ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "Top AI Agent Frameworks in 2025: Honest Reviews and Real-World ..."

**Query:** "climate change solutions"

- ✅ **Time:** 662ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "What are the solutions to climate change? - Greenpeace UK"

**Query:** "python async programming best practices"

- ✅ **Time:** 379ms | **Results:** 5 | **Score:** 3.2/10
- 🏷 **Sample:** "Asyncio best practices - Async-SIG - Discussions on Python.org"

---

### WebSearchAPI.ai

**Query:** "latest AI frameworks 2025"

- ✅ **Time:** 1518ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "Top AI Agent Frameworks in 2025: Honest Reviews and Real-World ..."

**Query:** "climate change solutions"

- ✅ **Time:** 1148ms | **Results:** 5 | **Score:** 3.6/10
- 🏷 **Sample:** "What are the solutions to climate change? - Greenpeace UK"

**Query:** "python async programming best practices"

- ✅ **Time:** 1302ms | **Results:** 5 | **Score:** 3.4/10
- 🏷 **Sample:** "Asyncio best practices - Async-SIG - Discussions on Python.org"

---

## 🏆 Top Performers

**Best quality:** Tavily (**3.7/10**)
**Fastest:** Exa (**137ms**)
**Best value (free):** Marginalia Search — 354ms avg, 3.0/10 quality, no key needed

> Time is shown for reference only — it is not a factor in the quality score.

## 💡 Recommendations

### Best no-key backends

1. **DuckDuckGo** — 1158ms avg, 3.5/10 quality, reliable
2. **Marginalia Search** — 354ms avg, 3.0/10 quality, fastest no-key

### Best backends (with API key)

1. **Tavily** — 356ms avg, 3.7/10 quality, rich content extraction
2. **Serper** — 667ms avg, 3.5/10 quality, Google results
3. **Firecrawl** — 644ms avg, 3.5/10 quality, search+crawl+extract
4. **WebSearchAPI.ai** — 1323ms avg, 3.5/10 quality, Google-powered
5. **Brave Search** — 460ms avg, 3.3/10 quality (fast, ~1 req/s limit)
6. **Exa** — 137ms avg, 3.2/10 quality, AI-native search
7. **LangSearch** — 1816ms avg, 3.2/10 quality, 10 results/query

---

_Generated by pi-search-multi benchmark on 2026-05-04T18:34:41.598Z_