LeaderboardEvalsAboutSubmit
Join Discord

Evals

Additional benchmarks and evaluations

Search API Eval v2

Brave, Exa, Parallel, Firecrawl and Tavily as search tools for a PR reviewer and a coding agent: retrieval accuracy, bug finding, hidden tests, latency and cost.

Vector DB Benchmark

Compares LanceDB, DuckDB+VSS+FTS, and SQLite+FTS5+sqlite-vec across code search and fantasy book RAG workloads.

Web Search API Benchmark

Benchmarks Brave, Exa, Firecrawl, Mixedbread, and Tavily across search, crawl, and extract scenarios.