A research agent read the developer docs for 100 real apps and scored each one on auth, self-serve access, and API surface. Then a second, adversarial pass fact-checked a sample of the results against live docs — and caught a real mistake. Every dot below is one app.
The headline isn't the 100 rows — it's what clusters inside them. Four things stood out.
Category, one-liner, auth, access, API surface, verdict, and the evidence URL behind every call.
| App | Category | Auth | Access | Surface | Verdict | Evidence |
|---|
Four steps, run per app. Source in
agent/research_agent.py.
Composio's SEARCH toolkit (or any web-search MCP) finds the app's real developer docs — not the marketing homepage.
Top result gets pulled with a browser/fetch tool, handling JS-rendered doc sites (Mintlify, Docusaurus, Notion-hosted docs).
Claude reads the fetched text and returns the 7 fields as strict JSON — nothing invented, blockers stated explicitly when evidence is thin.
A second, adversarial Claude call re-reads the same source and is told to find what's wrong, not confirm it. Disagreements get written back, not averaged away.
# the extension point in research_agent.py — where Composio's own tools plug in from composio import Composio client = Composio(api_key=os.environ["COMPOSIO_API_KEY"]) session = client.create(user_id="research-agent") results = session.execute_tool("COMPOSIO_SEARCH_SEARCH", {"query": f"{app_name} API auth docs"}) page = session.execute_tool("COMPOSIO_BROWSER_FETCH", {"url": results[0]["url"]}) # → fed into extract() and verify(), see the full file
When a name matches more than one real product (see iPayX in the matrix below), the agent is instructed to say so instead of guessing. A human picks the right one — this is not a case where more scraping helps.
PitchBook, Gladly: no further public docs exist to scrape past "contact sales." The finding is "gated, no public path" — the assignment counts that as a correct answer, not a miss.
The verify step catches disagreements automatically, but a human should skim the flagged rows before trusting them at scale — exactly what the verification pass below does on a 5-app sample.
Free-trial-but-time-limited, paid-tier-but-cheap, approval-that-takes-5-minutes — the agent defaults to the stricter label and a human can loosen it case by case.
Per the brief: sample the 100, cross-check by hand, show hits and misses honestly. These 5 were picked adversarially — the least-familiar, most-likely-to-be-wrong apps — not at random, because a random sample of 100 mostly-standard SaaS APIs would have looked artificially clean.
This page reads data/apps.json and
data/verification_sample.json directly — no build step.