Reproducibility protocolVersion 1.0.0

Search API retrieval benchmark

The exact queries, environment, commands, normalization fields, pass criteria, and public artifacts used for the Perplexity Search, Exa, and Parallel Search reports.

Frozen Aug 23, 2026 Three providers · 27 requests Credentials excluded

Protocol JSON Artifact index

01 · Frozen inputs

Exact queries

Every provider received these words unchanged. We did not tune a query after seeing a result.

Retrieve the official release schedule and current LTS status.

Find the current Node.js release lines and identify which releases are LTS. Prefer current primary sources from the Node.js project.

Expected: nodejs.org

Retrieve the maintained runner-image inventory from GitHub.

Find the current GitHub-hosted Actions runner images and their included software. Prefer current primary sources from GitHub.

Expected: github.com or docs.github.com

Retrieve the official supported-version and release-policy pages.

Find the PostgreSQL versions currently supported by the PostgreSQL project and the official support policy. Prefer current primary sources.

Expected: postgresql.org

02 · Fixed conditions

Shared test conditions

Provider-native endpoints were normalized to a shared result shape. Native retrieval modes were documented rather than treated as perfectly equivalent.

Queries: English, provider-default geography
Result limit: 10 requested
Repetitions: 3 per query
Timeout: 30 seconds
Delay: 1.1 seconds between requests
Normalized fields: Title, URL, snippet, published date, updated date

03 · Invocation

Commands used

Credentials are loaded from a local ignored environment file. They are never placed in commands, prompts, artifacts, or page source.

Perplexity Searchnode scripts/search-agent-bench/run-perplexity.mjs --label launch-v1Environment: PERPLEXITY_API_KEY

Exanode scripts/search-agent-bench/run-exa.mjs --label launch-v1Environment: EXA_API_KEY

Parallel Searchnode scripts/search-agent-bench/run-parallel.mjs --label launch-v1Environment: PARALLEL_API_KEY

04 · Evaluation

A request passes only when retrieval is usable

HTTP success alone does not pass. The normalized output must include enough valid results, sufficient snippet coverage, and a primary source near the top.

Attempt criteria

At least 5 valid URLs

Expected source at rank 5 or better

At least 80% snippet coverage

Case and report criteria

At least 2 of 3 repetitions pass

All three cases pass for an overall pass

Failed and blocked attempts remain in artifacts

05 · Public artifacts

Audit every repetition

Each bundle includes the protocol, exact queries, runner environment, timing, evaluation, and normalized ranked results. Raw responses and provider identifiers stay private.

Perplexity Search9 requests · 9 passed Exa9 requests · 9 passed Parallel Search9 requests · 9 passed

Public by default, secrets excluded by design.

Published snippets are capped at 280 characters. API keys, account IDs, search IDs, session IDs, raw provider payloads, and rate-limit headers are excluded from the public bundle.