Every Search API passed. That was the useful result.

We ran 27 Search API requests and got 27 passes. Every expected primary source ranked first. If the goal had been a tidy leaderboard, this would have been an annoying result.

It was also the honest one.

The test compared Exa, Perplexity Search, and Parallel Search across three technical research jobs, repeated three times per provider. A pass required a successful response, at least five valid URLs, 80% snippet coverage, and the expected first-party source within the top five.

All three providers cleared that bar every time. The pass rate told us the minimum bar was reasonable. It did not tell us the products were equivalent.

Binary scores remove the information you probably need

Pass/fail is useful for rejecting an API that cannot complete the task. Once several providers pass, collapsing their outputs back to one bit destroys most of the decision data.

The actual run looked like this:

Provider	Attempts	Expected source	Median latency range	Date coverage
Exa	9/9	Rank 1 in 9/9	130–171 ms	33%
Perplexity Search	9/9	Rank 1 in 9/9	427–1,045 ms	100%
Parallel Search	9/9	Rank 1 in 9/9	1,732–2,044 ms	49%

There is no defensible universal weighting for those columns.

A documentation lookup can tolerate missing dates and care about response time. A release-monitoring system may prefer complete date metadata. A research agent may accept a slower first call if richer excerpts reduce the number of pages it must fetch later. The benchmark observes inputs to those decisions; it does not know the utility function of your application.

“Fastest” was true only inside a narrow sentence

Exa recorded the lowest median latency in all three cases on our machine. That is a useful observation. It is not the same claim as “Exa is the fastest Search API.”

The sample contained nine requests, used provider-default geography, ran from one environment, and tested one endpoint configuration. The high observed values also varied considerably: Exa's GitHub and PostgreSQL cases reached 1,575 ms and 1,868 ms even though their medians stayed below 200 ms.

The honest sentence needs all of its qualifiers: Exa had the lowest median in each of three cases in this run. Removing those qualifiers makes the sentence more shareable and less true.

Complete dates did not mean better search

Perplexity returned date metadata on every normalized result. Exa returned dates on 33%; Parallel returned them on 49%.

Nothing in that result shows that Perplexity found more relevant pages. All three providers put the expected source first in every attempt. Date coverage is a response-shape capability, and its importance depends on whether freshness is part of the task.

This distinction matters because benchmark tables encourage readers to scan for the largest number. Sometimes the largest number is not a score.

A tie should change the next test, not the current result

When every system passes, there are two tempting responses:

1. tighten the threshold until somebody loses;
2. invent weights for secondary metrics and call the total a score.

Both can be legitimate if defined before the run and justified by a real workload. Doing either after seeing the results is just moving the finish line around the data.

We left the published pass result alone. The next protocol can ask harder questions: ambiguous queries, stale pages, date filters, duplicate results, or a downstream synthesis task. Those tests may separate the providers. They may produce another tie.

The point of a benchmark is not to guarantee separation. It is to make a claim that survives contact with the artifacts.

The useful output is a decision surface

For our three providers, the first run established a common floor:

• all requests succeeded;
• every task passed all three repetitions;
• snippets were present on every normalized result;
• the expected primary source ranked first in every attempt.

Above that floor, the APIs exposed different latency and metadata behavior. A buyer can now ask a better question than “which provider won?”: which observed tradeoff matters in my workflow, and what test would falsify my choice?

That is less satisfying than a gold medal. It is more useful when you have to ship the integration.

The exact queries, attempt-level results, and runner details are available in the three provider reports and the versioned protocol.