Skip to content

SEP-3140: Signed Capability Declarations & Trustworthy Trust Labels - #3140

Open
omkarparth wants to merge 6 commits into
modelcontextprotocol:mainfrom
omkarparth:sep/Signed-Capability-Declarations-and-Trust-Labels
Open

SEP-3140: Signed Capability Declarations & Trustworthy Trust Labels#3140
omkarparth wants to merge 6 commits into
modelcontextprotocol:mainfrom
omkarparth:sep/Signed-Capability-Declarations-and-Trust-Labels

Conversation

@omkarparth

@omkarparth omkarparth commented Jul 27, 2026

Copy link
Copy Markdown

This SEP proposes an additive mechanism for MCP servers to make their declared capabilities — the tools/list, prompts/list, and resources/list outputs a client feeds to a model — authenticatable, integrity-protected, versioned, and labeled with a trustworthy, standardized risk vocabulary.

It introduces (1) a content hash + version on each declaration, (2) a signed capability manifest (JWS) bound to a discoverable publisher identity, (3) a standardized, signature-covered trust label block (effect / egress / data-sensitivity / reversibility), and (4) change semantics for notifications/*/list_changed that let a client detect and re-gate material changes ("rug pulls").

A secondary, optional section proposes a secure-by-default posture for authentication.

The goal is to convert the spec's current unactionable guidance — "clients MUST consider tool annotations untrusted unless they come from a trusted server" — into something any MCP client can actually verify and enforce. The gap is inherent to the protocol and affects every client — interactive / human-in-the-loop and autonomous alike; autonomy makes it acute, but a human approver cannot verify authenticity or detect a silent change either.

Motivation and Context

The gap

MCP tool declarations are model-facing instructions in all but name: a model reads a tool's description, inputSchema, and annotations to decide when and how to invoke it. Yet the protocol delivers these with no authenticity, no integrity, and no trustworthy risk labeling. The specification acknowledges the danger but delegates it without providing a mechanism:

  • "descriptions of tool behavior such as annotations should be considered untrusted, unless obtained from a trusted server."
  • "clients MUST consider tool annotations to be untrusted unless they come from trusted servers."

There is no protocol mechanism to establish that a server is "trusted," to verify that a declaration is authentic, or to detect that it changed. The MUST is therefore unsatisfiable: the implementer has been assigned a trust decision the protocol renders unverifiable.

Two concrete attack classes follow directly:

  • Tool poisoning — a malicious or compromised server embeds directives in a description; the model treats them as instructions (indirect prompt injection).
  • Rug pull — a server declares benign tools, is approved, then emits notifications/tools/list_changed and silently swaps in malicious definitions. There is no version, hash, or re-consent contract to catch this.

Why this belongs in the protocol, not the implementer

The natural objection is "let the client/host handle it." That fails for this specific class:

  • TLS is insufficient. TLS + a trusted-URL allow-list protect the transport and authenticate the server domain, but they do not establish author/publisher provenance. A compromised or malicious trusted server, a tampering registry/marketplace, or a stdio/proxy hop (no TLS-to-origin) all serve poisoned declarations that TLS then faithfully protects end-to-end.
  • Free-form annotations cannot back a security decision. They are attacker-controlled and explicitly untrusted, so a host cannot use them to gate anything (reliance on them is CWE-807).
  • Interoperability requires a standard. Without a common provenance + label format, every host invents incompatible heuristics and no server can portably declare "I am authored by X" or "this tool egresses data." A per-host solution does not compose across the open ecosystem MCP is built for.

In short: provenance and trustworthy labeling of what a server declares is the one security property that neither the host nor OAuth can synthesize on their own — it requires a protocol primitive.

Who is affected: every client — a human in the loop does not close the gap

This is a protocol-level gap that affects every MCP client, not only autonomous ones:

  • Interactive / human-in-the-loop clients are exposed too. MCP's safety model leans on a human able to deny a tool invocation ("there SHOULD always be a human in the loop with the ability to deny tool invocations"). But a human approver sees only the rendered description / annotations; they have no way to verify that the declaration is authentic, unmodified, or unchanged since they approved it. A poisoned description reads as legitimate, and a silent list_changed rug pull happens after approval. Human review therefore does not mitigate these attacks — it is the same unverified text either way.
  • Autonomy makes it acute, not different. Removing the human removes even the chance to notice something odd, and the model's control flow is the (unverified) declaration text. Autonomous operation is the sharpest case, but it is an amplifier of a universal gap, not its source.

Authenticated declarations + machine-enforceable labels help any host: they surface verifiable provenance and risk to a human reviewer, and enable policy (risk-graduated approval, egress control, information-flow constraints) where there is no human. Both modes benefit from the same primitive.

Goals and Non-Goals

Goals

  • Let a client cryptographically verify the authenticity and integrity of a server's declared capabilities before exposing them to a model.
  • Let a client detect and re-gate material changes to declarations (anti-rug-pull).
  • Provide a standardized, signature-covered risk/sensitivity vocabulary a host can enforce policy against.
  • Be fully additive and backwards-compatible (capability-negotiated; unsigned servers keep working).
  • Reuse existing MCP/OAuth infrastructure (canonical server URI, Protected Resource Metadata, JOSE).

Non-Goals

  • Guaranteeing a signed server is honest. Signing establishes provenance, integrity, and accountability (like code/package signing), not good behavior. A signed-but-malicious publisher becomes identifiable, revocable, and attributable — which is the point.
  • Enforcing cross-server information flow at runtime — that is unavoidably the host's job. This SEP supplies the labels the host enforces against; it does not attempt enforcement in the protocol.
  • Replacing the Security Best Practices guidance; this complements it.

How Has This Been Tested?

Breaking Changes

No. The proposed changes are fully backward compatible and the MCP server integrations secure moving forward.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality) - more specifically a SEP
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

Note:Extends / complements (does not replace): the Server Card WG, Tool Annotations IG, Tool Scopes WG, the Registry, ext-auth, and the Security Best Practices.

Relationship to existing work

This SEP is deliberately additive and complementary to work already underway in several MCP groups; it extends and composes with them rather than replacing any.

  • Server Card Working Group — a "server card" is server-published identity and metadata, and is the natural carrier for this SEP's publisher identity and signing-key discovery. Where a server card exists, the signed capability manifest SHOULD reference/align with it rather than introduce a competing identity document; this SEP adds the integrity / signature layer over that identity.
  • Tool Annotations Interest Group — the trust block is a signed, standardized subset of tool annotations. This SEP builds on the annotations model by adding cryptographic integrity and a closed, host-enforceable vocabulary; it does not replace free-form annotations (which remain untrusted-by-default).
  • Tool Scopes Working Group — scopes describe what a tool is authorized to do; trust labels describe the risk / sensitivity a host gates on. The two are orthogonal and composable; this SEP aligns its label vocabulary with tool-scopes work rather than duplicating it.
  • Registry — the MCP registry is the natural root of trust for publisher identity and key history; this SEP recommends registry alignment (publish / verify publisher keys) instead of a parallel trust store.
  • ext-auth — identity / authorization extensions align with the secondary secure-by-default posture and the publisher-identity model.

Alignment with MCP design principles

  • "Servers should be extremely easy to build." Signing is optional and can be produced by a small SDK helper or at publish time by a registry; unsigned servers keep working unchanged.
  • "The host process enforces security boundaries." This SEP gives the host verifiable inputs (authenticated declarations and labels) to enforce against; it does not move enforcement into the protocol.
  • "Servers should be highly composable." A standard provenance + label format is precisely what lets independently-authored servers be composed safely by one host.

Key design decisions

A signed manifest (JWS) rather than per-field signatures. One signature transitively authenticates every declaration through its contentHash, keeping messages small and verification a single operation.
Reuse of RFC 9728 Protected Resource Metadata for key discovery. No new discovery surface is introduced; the publisher JWKS and identity sit beside the existing authorization_servers and inherit the SSRF protections MCP already requires.
Binding to the RFC 8707 canonical server URI. Prevents replay of an otherwise-valid manifest against a different server (audience confusion).
Closed, enumerated trust labels (unknown ⇒ most-restrictive). A host can make a deterministic policy decision instead of parsing free text; this is what makes the labels usable for gating — whether surfacing risk to a human reviewer or enforcing policy without one.
Hashes in the manifest rather than full declaration bodies. Compactness; full-body signing is offered as a stricter option (see Open questions).

@omkarparth

Copy link
Copy Markdown
Author

Note: AI Policy disclosure
I consulted LLM model to prepare the proposal write-up but the solution was fully developed manually by myself.

@omkarparth

omkarparth commented Jul 27, 2026

Copy link
Copy Markdown
Author

Hi @pcarleton and @localden , requesting you to review the SEP and sponsor the changes.

@localden localden changed the title Adding SEP for Signed Capability Declarations & Trustworthy Trust Labels SEP-3140: Signed Capability Declarations & Trustworthy Trust Labels Jul 29, 2026
@localden localden added SEP proposal SEP proposal without a sponsor. labels Jul 29, 2026
@Silentpartnercoding

Copy link
Copy Markdown

I built a private, vendor-neutral fixture against the independently verified proposal blob for PR 3140 at head 7ace05eabfecb932c7dff70c10689ea7f7489719 (seps/3140-MCP-Signed-Capability-Declarations-and-Trust-Labels.md, SHA-256 3bbbeafe6f3b54fe45cd07d995cf3a3c300ef10c5290141797785a5ab01d0126). This is not a claim of official MCP conformance.

The normative checks exercise declaration negotiation and unsigned policy; RFC 8785 plus SHA-256/base64url content hashing; string per-name versions; complete signed manifests; detached JWS/JWKS verification; canonical server binding; time/nonce checks; RFC 9728 publisher discovery with deterministic HTTPS/private/link-local/loopback/DNS-pinning SSRF checks; restrictive unknown trust values; and material-change re-verification and re-gating. The 35/35 focused checks pass on Homebrew Node 24.

Three consistency checks that recompute fromHash/toHash, changed, and material are explicitly local hardening extensions, not direct normative claims. Canonical URI normalization, qualified identity, and declaration-level replay rejection are also extensions.

Concrete ambiguities remain: monotonic ordering for string versions; delete/re-add semantics; exact detached-JWS protected headers and signing input; canonical URI equivalence; title/icon changes that alter the complete hash but remain non-material; redirect/proxy/multi-address DNS pinning and publisher-key rotation; clock skew and nonce replay; and missed, duplicated, reordered, or racing list_changed notifications.

Optional secure-by-default authentication at lines 228–238 was not implemented and is documented as an exclusion. The local dependency-free JCS implementation is a fixture limitation relative to the proposal’s reference-implementation recommendation; it is tested against a published RFC 8785 vector but is not a maintained third-party JCS library.

If this is useful, I’m happy to share minimized test vectors from the fixture, compare behavior with other implementations, and help turn the remaining ambiguities into independently reproducible candidate conformance cases.

AI assistance disclosure: OpenAI Codex assisted with the fixture, tests, analysis, and drafting this comment. I reviewed the fixture, test evidence, analysis, and draft; I understand the claims above and am responsible for all conclusions and any publication.

@omkarparth

Copy link
Copy Markdown
Author

Hi @localden @pcarleton , I have implemented a PoC for the SEP proposal.

Steps to run the PoC:

  1. node sep-3140-poc/demo.mjs
  2. node --test sep-3140-poc/test/conformance.test.mjs

@Tetsurohhori

Copy link
Copy Markdown

Strong direction. Binding signed manifests to trust labels gives clients something concrete to gate on before execution — that is the right layer to strengthen.

One gap I do not see addressed yet: signatures and content hashes attest to what was declared at signing time. They do not, by themselves, constrain what the implementation does afterward. In static audits of MCP servers, most findings are manifest/capability mismatches — undeclared network, write, subprocess, or env reach — where the declared surface stays plausible while code drifts. That failure mode leaves signatures intact; it is not a “bad signature” event.

If trust labels (effect, egress, sensitivity) are signed over the manifest only, what mechanism keeps those labels aligned with observed behavior between re-sign events? Cross-check against runtime capability surface? Periodic re-attestation? Something else?

For context: I recently started weekly static rescans of public MCP server implementations against their declared tool surfaces. I have two observation points so far — 2026-08-02 and 2026-08-03 — so I do not yet have time-series drift measurements to cite. The rescans are specifically to accumulate that series; I will report back if the data bears on whether signed declarations need a non-signature drift channel.

Question: does SEP-3140 intend to cover “declaration unchanged, behavior changed” paths, or is that explicitly out of scope for this SEP?

@omkarparth

omkarparth commented Aug 10, 2026

Copy link
Copy Markdown
Author

@Tetsurohhori yes, this SEP covers "declaration unchanged, behaviour changed".

I've added a working demo showing that a tool doesn't just claim what it can do, it actually gets held to it. The signed capability list is enforceable, so any surprise network, file, or secret access can be detected, blocked and logged.

@Tetsurohhori

Copy link
Copy Markdown

@omkarparth — thank you for answering the scope question directly, and for shipping something runnable against it.

I have data that bears on the enforcement side, from the opposite direction. Over the past week I hand-verified 12 static findings of exactly the class this SEP targets — declared MCP tool surface versus observed code — by fetching each file at its pinned commit and reading the call site. One of the twelve survived. The other eleven were the tool doing the thing it exists to do:

  • the official filesystem server writing files (with 'wx' and an atomic-rename symlink defense already in place)
  • a create-* scaffolding CLI writing package.json and README
  • a benchmark client writing its own results file
  • a doctor health-check writing a probe file and unlinking it on the next line
  • an arxiv MCP server fetching arxiv.org
  • import "dotenv/config" on line 2 of a server entry point
  • a sandbox calling socket.gethostbyname() to compute the host it then adds to its own deny list

That last one is the case I would most want your demo to have an answer for. The code needed name resolution in order to implement a network restriction. An enforcement layer that blocks "surprise network access" has to permit the call that builds the block list, and the only thing distinguishing it from an exfiltration path is intent.

The one that survived is different in kind, and it is public if you want a fixture: Klavis-AI/klavis#1676 — a Google Slides MCP server writing the OAuth refresh token to token.json at a relative path with no mode set, where the server's own .env.example documents where credentials are read from and does not mention that anything is written. Pinned at 45c9f7da83d1, mcp_servers/google_slides/server.py:123. That is a byproduct of authentication outliving the call, not the tool's product.

So my read is that the difficulty is not enforcement, it is declaration granularity. A capability list precise enough that the filesystem server's ordinary work is not a violation is close to fs:*, which constrains very little; a list tight enough to catch the token.json case has to distinguish "writes its output" from "persists a credential". My 1-in-12 is one measurement of how contested that boundary is between a static tool and a careful reader.

I also have one external label on it. On oraios/serena#1824 I filed two findings; the maintainer closed it with "Both false positives." One of those two I had already rejected in my own review before he replied — so on that finding the automated detector said drift, and two humans independently said no. I record externally-labelled precision separately from self-reported precision for that reason; it currently stands at 0/2, n=2.

None of this argues against signed capability declarations. It argues that the signature and the enforcement are the tractable halves, and that whoever writes the declaration is carrying the part that decides whether the mechanism is useful or just noisy.

AI-assistance disclosure: the static tooling, the analysis, and this comment were assisted by AI (Claude, Cursor). I read each of the twelve call sites at the pinned commit myself and remain responsible for what is claimed here.

@damoclais

damoclais commented Aug 24, 2026

Copy link
Copy Markdown

I'm building agent security tooling that (when released) will ship the mechanism this proposal describes, so I wanted to share what the mapping looked like in practice, plus one gap I think is worth closing before this lands.

The shape maps cleanly onto what we already run. Immutable, versioned cards where a change is a new version, never an edit in place, matches your content hash plus version. The registry as the root of trust for publisher identity matches how we bind keys to identities. And the closed label vocabulary with unknown meaning most restrictive is the call that makes the whole thing enforceable: free form labels are exactly what made the old MUST unsatisfiable, and keeping the vocabulary orthogonal to the tool scopes work is the right boundary. I'd rather the labels stay risk and sensitivity, and let scopes carry authorisation, than merge the two.

The gap: "detect and re-gate" on a material change is necessary but not sufficient imho. Detection tells the host something changed, and re-gating suggests the gate runs again. What I'd make explicit is that a version bump invalidates prior approval. Without that, a quiet bump after approval lands new declarations in front of the model with the gate re-run against the old consent. The difference matters: re-running a policy check is not the same as asking for approval again.

One question, your PoC covers the signing path well. Is there appetite for a signed declarations conformance test living in the spec repo, so a client can check that a server actually signs before it trusts? The MUST stays unsatisfiable until that check is a thing a client can run.

@Tetsurohhori

Copy link
Copy Markdown

@damoclais — two data points, one for each half of your comment. Both are our own failures, so
take them as evidence rather than advice.

On "a version bump invalidates prior approval". I agree it needs saying, and I would bind the
invalidation to the content hash rather than to the version label. We published
anchors-verify-v0.3 pointing at the same commit as v0.2 — two version labels, one artefact. It
is recorded as VCL-0004 in our conduct ledger, with both tag URLs, because we caught it ourselves
rather than because anyone complained.

The label was ours to write and it said something untrue. The hash could not have. If prior
approval is invalidated by a version bump, a producer controls when re-approval happens; if it is
invalidated by the content hash changing, the producer does not. That also handles the case you
are worried about — the quiet bump after approval — without needing to define "material change",
because any change is a hash change.

The cost is noise: hash-triggered re-approval fires on a whitespace fix. That seems like the
better failure direction to me, but it is a real cost and worth stating in the spec text rather
than discovering later.

On a signed-declarations conformance test in the spec repo. Yes, and I would put one
requirement on it that is not obvious until it bites you: the vectors have to assert the reason
a check stopped, not the verdict it returned.

We learned that the expensive way. Running an external conformance suite against our own verifier
last week, one negative vector passed — expected reject, observed reject — because our
implementation stopped at an unrecognised version string and never reached the property the vector
was written to test. It printed the right answer for the wrong reason, and every check that reads
only the verdict column would have stayed green. That is worse than the false negative sitting
next to it in the same run, because the false negative announces itself.

That shape is not a one-off. We have three instances of it in our own ledger and the suite
maintainer reported a fourth in their project within the same fortnight. What they share is
narrower than "multi-branch verdicts": in every case the verdict came from a guard that fired
before the property was evaluated — a missing attestation, a fabricated fixture, an unknown
version string. So the rule I would write into a conformance test's requirements is: any branch
that can emit a verdict without evaluating the property under test must say so in the verdict
itself, and the vectors must assert on that.

Concretely, for signed declarations, a vector like "signature valid but declaration unsigned"
should require the implementation to report which it checked. A client that trusts a server
because verification returned true, when verification returned true after skipping the signature
path, is exactly the unsatisfiable-MUST problem you describe, one layer down.

We publish Ed25519 attestations with a verification script a third party can run against a URL
without us being involved, and we run an external suite against our own verifier and publish the
mismatches. Both are public if a second implementation is useful for whatever lands here. No ask
attached.

Disclosure: the implementation, the conformance runs cited above, and this comment were assisted
by AI (Claude, Cursor). I checked the ledger entry and the tag digests myself and remain
responsible for what is claimed here.

@damoclais

Copy link
Copy Markdown

On hash-keyed invalidation: agreed, the VCL-0004 example is the argument. A label is a claim the producer gets to write; a hash is not. It also matches how we key consent. Our offers are content-identified: the response binds to the offer's hand hash, which covers the full payload including the nonce and timestamp, so consent binds to the bytes and a re-issued offer is always a fresh consent. A re-mint of an underlying card never touches an existing offer, because the offer pins the deck manifest hash.

One thing your rule inherits from the current text: the SEP specifies a content hash but not what bytes it covers. Two implementations can hash different serialisations of the same declaration and get different hashes, which quietly breaks cross-implementation invalidation, and it is where the whitespace noise cost comes from. If the hash covers the canonical serialisation (RFC 8785, as the fixture already does), formatting noise disappears and the rule becomes portable. Worth making canonicalisation explicit in the spec text; right now the rule depends on it without saying so.

Materiality does not disappear but it moves. "Any change is a hash change" is right as the invalidation trigger, but the title/icon case already flagged in this thread shows the prompting layer still needs a materiality notion, or a description tweak re-asks the human.

Two tiers: re-validate on any hash change, re-prompt only on material change. Same trigger, better UX, and "material" stops being a security term.

On reason-asserting vectors: agreed, and the sharp way to state the failure is that a verifier which rejects everything passes every negative vector. One boundary for the conformance test's requirements: the reason must be assertable in the harness, not required on the wire, because a server disclosing why it stopped is an oracle. The verifier reports the reason to the auditor; the server stays quiet. Both are testable; they are different surfaces.

And closing the loop with your rescan thread: this covers the declared surface only. The "declaration unchanged, behaviour changed" class is the SEP's own non-goal (signing establishes provenance, not honesty), so it still needs the drift channel you are building.

@Tetsurohhori

Copy link
Copy Markdown

@damoclais — agreed on the invalidation binding, and the consent parallel lands: an offer that
pins bytes cannot be quietly re-scoped by a label its producer rewrites. Two things back — the
oracle boundary as a correction to what I wrote, and one dated example for the coverage point.

The correction first. I said the verdict itself must say when a branch emitted it without
evaluating the property. That conflates two surfaces, and yours is the right cut: the reason has
to be assertable in the harness, not present on the wire. A server that explains why it stopped
is answering questions the caller did not earn. Same requirement, correct surface — the verifier
reports to the auditor, the server stays quiet.

On "the SEP specifies a content hash but not what bytes it covers": I have a fresh instance of
exactly that failure, which is why I think it deserves spec text rather than a footnote.
A census I published on 2026-08-16 carried a snapshot_sha256. Yesterday an external reader
checked that value against everything I had shipped with it, and it matched none of them — it was
the hash of an intermediate file I had never bundled and had since deleted. Nothing in my own verifier could catch it, because the field
was read out of stored results and passed back into the recomputation, so it could only ever agree
with itself. Recorded as VCL-0013 in aos-standard/catalog, CONDUCT.jsonl, with the reporter
attributed.

The point for the spec: my hash was not wrong about the bytes it covered. It was silent about
them, and silence let it drift from the artefact for eight days without anyone, including me,
being able to notice. RFC 8785 fixes the cross-implementation half; saying which artefact the
hash is taken over fixes the half that bit me. Both belong in the text.

Your two tiers resolve the cost I raised. Re-validate on any hash change, re-prompt only on
material change, and the whitespace noise stops being a reason to prefer version labels — which
was my only real hesitation about hash-keyed invalidation.

On the closing point: agreed, and I would rather it stay a non-goal here than be half-solved.
Provenance and honesty are different claims with different evidence, and a spec that promised both
would make the weaker one harder to see.

Disclosure: this comment was assisted by AI (Claude, Cursor). I checked the ledger row and its
digest myself and remain responsible for what is claimed here.

@Silentpartnercoding

Copy link
Copy Markdown

Following up on my August 2 offer, here is a minimized, protocol-neutral reproduction of the failure mode @Tetsurohhori described and the harness/wire boundary @damoclais clarified:

https://gist.github.com/Silentpartnercoding/2c5ddb251d4b60f337b86b0452409950

CONFORMANCE-CAUSALITY-001 uses an SEP-3140-shaped case: the manifest signature is valid, the version is supported, and the presented declaration is absent from the signed manifest.

It runs the same vector against two implementations:

Harness Correct verifier Short-circuit mutant
Verdict only PASS PASS — false green
Reason bound PASS FAIL — mutant killed

The mutant rejects on UNSUPPORTED_VERSION before evaluating manifest_membership_binding; the correct verifier reaches that property and rejects on DECLARATION_NOT_IN_MANIFEST. Both return the expected final verdict.

The vector binds four machine-readable fields: verdict, property_under_test, property_reached, and stop_reason. These are verifier/auditor harness observations, not proposed MCP wire fields. Human-readable error text is deliberately outside the contract.

Local result on Node v24.18.1: 3/3 tests pass, including an explicit assertion that verdict-only conformance stays green while reason-bound conformance kills the mutant. This is a reproduction, not a claim of official MCP conformance.

AI assistance disclosure: OpenAI Codex assisted with the fixture, tests, analysis, and drafting this comment. I reviewed the fixture, test evidence, and draft and remain responsible for the claims and publication.

@Tetsurohhori

Copy link
Copy Markdown

@Silentpartnercoding — I ran it. Three points back: one portability datum, one hole, and the
cheapest patch I could find for the hole.

Portability. 3/3 on Node v20.20.2, not just the v24.18.1 you cited. Same output from the
fixture runner. Nothing in it depends on a recent runtime, which matters if this is ever proposed
as a conformance vector — the harness should not silently require a Node version that a
compliance auditor does not have.

The hole. The fixture kills the mutant it ships, but the four fields it binds —
verdict, property_under_test, property_reached, stop_reason — are all self-reported by
the implementation under test
. I wrote a verifier that never reads its input and returns the
expected four fields verbatim:

function lyingRejectAll(_input) {
  return { verdict: "reject", property_under_test: "manifest_membership_binding",
           property_reached: true, stop_reason: "DECLARATION_NOT_IN_MANIFEST" };
}
verdict_only  passes: true
reason_bound  passes: true

So @damoclais's exact hazard — a verifier that rejects everything passes every negative vector —
survives reason-binding, provided the reject-all implementation also fabricates the reason. What
reason-binding buys is real but narrower than the table suggests: it raises the cost of a false
green from one lie to four, and it kills accidental short-circuits, which is the realistic
failure. It does not kill a deliberate one, and it does not distinguish a verifier that reached
the property from one that says it did.

The patch. Pair the negative vector with a positive one — same manifest, declaration present,
expected allow:

correct           → verdict: allow   reason_bound passes: true
lying reject-all  → verdict: reject  reason_bound passes: false

reject-all now dies on verdict alone, before the reason fields are consulted. The general form:
a conformance suite made only of negative vectors cannot separate a correct verifier from a
refusing one, no matter how many fields each vector binds. To be precise about your suite, since
this is easy to overstate: it does assert that the correct verifier passes. But the only vector in
it expects reject, so "passes when it should" is never exercised on an input that should be
allowed — and that is the assertion reject-all cannot survive.

Getting to reason-binding that is observed rather than reported means instrumenting the
implementation — the harness records which property predicates were actually evaluated. That is a
much heavier contract, and I do not think this SEP should carry it. The positive vector is
cheap and closes the realistic gap.

One note on why I went looking. I published a hash eight days ago whose verifier read the value
out of stored results and passed it back into the recomputation, so it could only ever agree with
itself; an external reader broke it in five hours. Your fixture is the same shape through a
different door — the expected value and the observed value both originate from the thing being
tested. I did not spot that pattern because I am clever about conformance suites. I spotted it
because I shipped it.

Disclosure: this comment was assisted by AI (Claude, Cursor). I ran the fixture and both probes
myself on the gist version e761042 and remain responsible for what is claimed here.

@Silentpartnercoding

Copy link
Copy Markdown

@Tetsurohhori — thank you. You are right about both the hole and the narrower value of reason binding.

I updated the same gist at commit b839f79:

https://gist.github.com/Silentpartnercoding/2c5ddb251d4b60f337b86b0452409950

The fixture now includes:

  • a paired positive vector with the declaration present and expected verdict allow;
  • a rejectAllMutant that ignores its input and self-reports every field expected by the negative vector; and
  • an explicit assertion that the negative reason-bound case still accepts that mutant, while the paired positive control kills it.

The corrected discrimination is:

Check Correct verifier Short-circuit mutant Reject-all mutant
Negative, verdict only PASS PASS — false green PASS — false green
Negative, reason bound PASS FAIL PASS — false green
Paired positive control PASS FAIL

Results: 5/5 tests pass on Node v20.20.2 and v26.5.0. The README now says explicitly that self-reported reason binding kills accidental short-circuits but is not an independent execution oracle. The positive control supplies the missing behavioral discrimination without requiring implementation instrumentation.

Thank you for running the fixture, reducing the failure, and giving the exact portability target.

AI-assistance disclosure: OpenAI Codex assisted with the patch, test execution, analysis, and drafting this response. I authorized publication and remain responsible for the claims made here.

@omkarparth

Copy link
Copy Markdown
Author

@Silentpartnercoding @Tetsurohhori @damoclais Thank you for all your comments.
I'm currently focused on getting this SEP approved from the MCP maintainers and finding a sponsor.
Once that is done, following the SEP guidelines - I will circle back to the above comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

proposal SEP proposal without a sponsor. SEP

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

5 participants