A 200 is not the same as access

The verdict was FAIL, with seventeen leaks, the first time I ran my access probe against production. For a moment that is a bad feeling, because seventeen is a lot of doors to have left open.

The probe is a management command with a narrow job. It walks every route my platform map knows about, on both places where access gets enforced: the Django URL conf and the FastAPI gateway. It logs in as a deliberately low-privilege account and fires requests at each route. A route on the allow list must answer 200. A route off the list must answer 403 on every verb, or serve a redacted body. Any write, whether a POST, a PUT, or a PATCH, must be refused with a 403 or a 405.

In that matrix a leak is the one outcome I cannot accept: a route off the allow list that hands back data, or a write that gets through. A single leak fails the whole run. The probe is also a gate.

So I read the seventeen. Not one of them was under the API prefix. They were paths like repositories, stats, and a dashboard metrics page. Each had returned a 200 with a body, and the probe had read that as data served to someone who should not have it.

The cheapest test I could think of was to ask again with no credentials at all. An anonymous curl of the same paths came back with the identical answer: 200, text/html. The real API path for repositories, asked the same way, answered 401. If a caller with no login gets the same response as my logged-in probe, then nothing behind that response is checking logins. The access rules were sound. My reading of them was wrong.

The mechanism was the web server. In production, nginx hands back the React index.html for any path the API is not mounted at. That is how a single-page app works: the shell loads, and then the app asks the real API for data. My platform map lists routes by their Django pattern, which says what exists in the code. It says nothing about where a request for that path is actually served. The probe trusted the map, hit seventeen paths with no API behind them, and graded the app shell as a breach.

I had split the probe on purpose. The grading lives in a small pure function, classify, that takes whether the route is allowlisted, the method, and the status. The HTTP calls live in a separate shell around it. The split lets CI test every grading rule with no network, and I still like it.

It also explains the bug. The function's inputs are the allow list flag, the method, and the status. There is no body and no content type. A 200 on a GET to a route off the allow list could only ever come out as a leak, because I never handed the function the one fact that tells a shell from a payload.

def _is_spa_html(resp_headers: dict | None) -> bool:
    ct = ""
    for k, v in (resp_headers or {}).items():
        if k.lower() == "content-type":
            ct = str(v).lower()
            break
    return ct.startswith("text/html")

spa_shell = status == 200 and _is_spa_html(resp_headers)
if spa_shell:
    outcome, reason, leak = svc.WARN, "spa_catch_all_html_not_api", False

On the next production run, leaks went from seventeen to zero. Admitted writes stayed at zero. All twenty allowlisted panels still served their 200. The same run then showed two new failures on allowlisted routes that take an id in the path. Both returned 404.

That one took less digging. To call a route that needs an id, the probe substitutes a placeholder UUID. No object has that UUID, so 404 is the correct answer. An allowlisted route with a parameter that returns 404 is now a warning. An allowlisted route with no parameter that returns 404 still fails, because that page should exist.

Six new tests cover all of this, and they are hermetic: a fake session hands back a canned status and content type for every verb, with no network. Rolling the change back is a revert, and since the probe is a read-only tool, nothing in production behaves differently either way.

These false alarms were harmless, and I still could not leave them in place. A gate that is permanently red on false positives cannot gate anything, because the people reading it learn to skim past it, me included. Had I left it red, a real leak would have arrived looking like the other seventeen. A status code tells me the server answered. Whether it answered with private data is a question about the body, and the probe now asks it.