Principles
- Never fake a pass. If a check cannot run safely, it is recorded as skipped with the reason, not as passed.
- No real credentials. We never give a server real API keys or account access.
- No business actions. We list a server's tools but never call them.
- Isolation. Third-party code only runs in a separate CI job that has no access to our database keys.
Capability discovery
Once a week, we pick servers that can be started safely — currently Node.js packages run with npx, with a clean package name and no required credentials. Each one is started in a throwaway environment and sent the standard MCP handshake: initialize, then tools/list, resources/list, and prompts/list. We store the names and descriptions it returns. Python, Docker, and git-based servers are recorded as skipped for now.
Current outcomes across 453 servers:
- Not run: no usable start command in the docs: 276
- Not run: needs credentials (API keys or tokens): 77
- Not run: runtime not yet supported, or command unsafe to run: 58
- Started and answered the MCP handshake: 26
- Started, but no handshake before the timeout: 16
A skipped server is not a broken server: either we found no start command we can run safely, or it needs credentials we deliberately do not provide.
Documentation, configuration, and safety checks
These checks read the README and metadata; they do not run any code. The docs score adds points for a clear description, install instructions, a client configuration example, documented environment variables, usage examples, a license, and a security note. The config score checks for a valid mcpServers JSON block. The safety score looks for sensitive environment variables (such as tokens) and destructive operations (such as delete or deploy), and gives credit for good documentation.
Benchmark score
Each part is scored 0–100 and combined with these weights:
| Part | Weight | What it measures |
|---|---|---|
| Install | 20% | Whether the documented install method resolves (for example, the npm package exists). |
| Startup | 20% | Whether the server process starts. |
| Capability discovery | 25% | Whether the server answers the MCP handshake and lists tools, resources, or prompts. |
| Documentation | 15% | README covers description, install, client config, environment variables, usage, license, and security. |
| Configuration | 10% | README has a valid mcpServers JSON example with command, args, and env. |
| Safety | 10% | Credential and destructive-operation signals, balanced by documentation quality. |
A check that has not run yet counts as "unknown" and scores in the middle-low range — neither a pass nor a failure. Skipped checks score higher than failures but lower than passes. Results are on each server's benchmark page and in the benchmark overview.
Risk profiles
Separately from benchmarks, each server gets a risk profile from keyword signals in its README, description, install command, and category: API tokens, filesystem access, writes to external systems, database connections, and network access. A signal is only marked "Yes" when we find evidence; otherwise it stays Unknown. See MCP server security for how to use them.
Ranking scores
Rankings use a separate score from daily GitHub data: popularity (stars, forks, watchers) 35%, freshness (recent commits) 25%, growth (7-day star change) 25%, and maintenance 15%. Nobody can pay for a better position. Browse the rankings.
Known limitations
- Only Node.js servers are started today; other runtimes are skipped.
- Servers that need credentials cannot be discovered without them.
- README-based checks reward good documentation, which is not the same as good code.
- Risk signals come from text and can miss undocumented behaviour.
Spotted a wrong result? Tell us and we will correct it.