Enterprise MCP gateway evaluation kit

Prepare a reproducible evaluation

Use a disposable workspace, separate requester and approver identities, and synthetic resources. Record gateway release, client/version, protocol, policy revision, schema digest and UTC date. Count upstream dispatches; HTTP status alone is insufficient.

1. Allow engineering resources; require independent write approval. Establish a successful baseline read.
2. Example read: sandbox__read_document with {"resource":"engineering/handbook"}. Example write: sandbox__create_ticket with {"resource":"engineering/roadmap","title":"Evaluation"}. Map equivalent fixtures for another gateway.

Try the synthetic scenarios

Estimate Durin costs

Run the six control tests

Reset the successful baseline before each test. Record decisions, dispatch counts and audit outcomes.

• Revocation: read, revoke membership, repeat using the existing session. Expect denial, zero additional dispatches; reconnecting must not restore access.
• Schema changes: approve a write, change its reviewed schema/version, retry. Expect fresh review; no stale-authority execution.
• Resources: change engineering/handbook to finance/payroll, then an unmapped resource. Both must deny before dispatch.
• Independent approval: reject self-approval. A separate approver grants the write; an identical retry executes once. Changed arguments and consumed grants must not authorize another write.
• Expiry: grant a new write, advance the fixture clock past expiry, retry unchanged. Expect no dispatch and fresh approval.
• Uncertainty: commit a fixture write, drop its response. Expect uncertain outcome and no automatic repeat. Reconcile against the fixture before retrying.

Record evidence and decide

Copy per test: ID; UTC date; operator; versions; fixture/policy revision; setup; synthetic request; expected/observed decision; dispatch count before/after; audit ID/outcome; pass/fail/unverified; evidence location; limitations; cleanup.

Compare gateway approaches

Review Durin scope