All chapters
  1. Start hereA valid token is not a permission
  2. Foundations · 1Identity and trust primitives: who is calling, and who checks
  3. Foundations · 2People and identity proofing: accounts, passwords and recovery
  4. Foundations · 3Sessions and browser security: cookies, CSRF, XSS and the BFF
  5. Foundations · 4Authenticators and MFA: passwords, passkeys and phishing resistance
  6. Foundations · 5API keys and application credentials: what a key proves
  7. Foundations · 6Tokens: bearer, JWT anatomy, validation and revocation
  8. Foundations · 7OAuth, OpenID Connect and federation: delegation and login
  9. Authorization · 8Authorization from first principles: subject, action, resource, context
  10. Authorization · 9Access control models: RBAC, ABAC, ReBAC and ownership
  11. Authorization · 10The enforcement architecture: PEP, PDP, PIP and PAP
  12. Authorization · 11Policy as code: versions, tests, engines and decision logs
  13. Authorization · 12Tenant and data isolation: one platform, many brokerages
  14. Authorization · 13Identity governance and lifecycle: joiner, mover, leaver
  15. Authorization · 14Privileged access: standing rights, just-in-time elevation and break-glass
  16. Workloads · 15TLS, PKI and certificates: proving a host, and a client
  17. Workloads · 16Workload and service identity: SPIFFE, mTLS and caller allowlists
  18. Workloads · 17API gateways and zero trust: the edge checks, the service decides
  19. Workloads · 18Event-driven identity: Kafka principals, ACLs and replay
  20. Workloads · 19Cloud IAM and secretless deployment: roles, federation and no stored keys
  21. Workloads · 20External SaaS, webhooks and trust: calls that cross company lines
  22. Workloads · 21Private controlled environments: what private networking does and does not secure
  23. AI-native · 22Authenticating to model providers: keys, gateways and cloud IAM
  24. AI-native · 23MCP identity and authorization: discovery is not permission
  25. AI-native · 24RAG and retrieval authorization: only what this caller may read
  26. AI-native · 25Agents as principals and delegates: whose authority is this?
  27. AI-native · 26Agent-to-agent trust: identity, capabilities and forwarded authority
  28. AI-native · 27Coding agents and CI: repository scopes, OIDC and no production keys
  29. Operations · 28Auditing and observability of access: decisions you can prove later
  30. Operations · 29Threat modeling and testing: the negative tests that prove access control
  31. Operations · 30Operational lifecycles: rotation, revocation and outages
  32. Reference · 31Standards, frameworks and products: what each one is, and what it is not
  33. Reference · 32Architecture decision playbooks: choosing an auth strategy
  34. ReferenceGlossary
  35. AboutAbout this site and its sources
Operations · 29

Threat modeling and testing: the negative tests that prove access control

STRIDE over trust boundaries, then the tests that matter: tampered and expired tokens, wrong audience, BOLA, BFLA, cross-tenant reads, token substitution, confused deputy, escalation, races, access after revocation and failing open.

You will map
The trust boundaries of the insurance flow with STRIDE, and give every threat a control and a test.
You will write
Negative tests that keep every input valid except one: the tenant, the function, the audience, the time.
You will show
The definition-of-done evidence: 28 negative tests and 24 cases, run and pasted, mapped to what they prove.

A small policy change ships on a Thursday. The release pipeline runs every test, and everything is green: the assigned underwriter u-uw-w1 can record a decision on SUB-1004, the broker u-broker-n1 can read Q-5531. Every test asked "can the right person do the right thing?", and every answer was yes.

No test asked the opposite question. The new version had lost the assignment check on decision.record, so the peer underwriter u-uw-w2, same unit, not assigned, could now decide SUB-1004. This site's own case UW-A09 records exactly this drift between authz-2026.11.1 and authz-2026.10.3. Positive tests cannot see it: a policy that allows too much still allows everything they ask for.

This page is about the other half of the suite: deciding what must never happen, at which boundary, and writing the test that fails the day it does.

Threat models and negative tests, in plain words#

A threat model is a written list of where a system can be attacked, what could go wrong there, what stops it, and how you know it works. A tiny example: "at the quote API, a token issued for another service could be presented; the API checks the audience; test UW-A05 proves the check."

The places worth listing are the trust boundaries: points where a request leaves one principal's control and enters another's (the vocabulary is on Primitives). The broker's browser to the portal backend is one; the agent application to the tool server is another.

A negative test sends a request that must be refused and passes only when it is refused and nothing changed. The useful kind keeps every input genuine except one. Broker u-broker-s1 gets a real, valid token and asks for Q-5531, which belongs to brk-north; the test expects 404 and no body. With a broken token it would only test authentication.

Before AI. Spring teams have written these for years: a MockMvc test with @WithMockUser(roles = "BROKER") that calls an admin endpoint and expects 403, or a test that reads another customer's order and expects 404. The agent adds new boundaries (the token service, the tool server, the model's output); the discipline is the same.

Guideline (OWASP Authorization Cheat Sheet, read 2026-11-18): automated access-control tests "can help reduce the number of security flaws that make it into production" and should ask "is access being denied by default?"; they catch the "low-hanging fruit" and do not replace a skilled manual tester.

STRIDE: six questions per boundary#

Listing threats from memory finds the ones you already thought of. STRIDE is a checklist of six categories to ask about at each boundary.

Pattern (STRIDE, as described by Microsoft on 2026-11-18): Microsoft uses it in its Threat Modeling Tool to categorize threats; the table quotes or follows that page.

LetterCategoryMicrosoft's description (short)The property it attacks
SSpoofingusing another user's authentication informationauthentication
TTampering"the malicious modification of data", stored or in transitintegrity
RRepudiationusers deny an action and nobody can prove otherwiseaccountability (audit)
IInformation disclosure"the exposure of information to individuals who are not supposed to have access to it"confidentiality
DDenial of serviceattacks that "deny service to valid users"availability
EElevation of privilege"An unprivileged user gains privileged access"authorization

The fourth column is this site's mapping, not Microsoft's wording.

The trust boundaries of the insurance flow#

The diagram keeps one threat per boundary; the table after it is the working threat model. The last column names the test or case that proves each control.

Diagram: Trust boundaries of the insurance flow · the worst STRIDE threat at each, its control, its test

Each row is one trust boundary: a place where a request crosses from one principal's control into another's. Read across: the threat that matters most there, the control that stops it, and the negative test or case that proves the control works. The full table on the page has more rows; this view keeps one per boundary.

  1. Broker browser to portal
    1. S: stolen session or forged post: Spoofing the broker: a script reads a token in the browser, or another site posts a form with the broker's cookie.
    2. HttpOnly cookie, tokens stay server-side: The BFF keeps tokens on the server; the cookie is HttpOnly, Secure, SameSite; state-changing requests need a CSRF defense.
    3. session lab outcome table: The recorded outcomes of XSS and cross-site post against both designs.
  2. Portal BFF to quote API
    1. S, T: substituted or edited token: A genuine token for another audience, an ID token, an edited scope, alg none.
    2. ordered validation: alg, kid, signature, iss, aud, exp: The resource server validates the token in a fixed order and trusts no claim before the signature verifies.
    3. UW-A05 to UW-A08: Wrong audience, expired, tampered and alg none all rejected at the first failing check.
  3. Quote service to database
    1. I: another tenant's row: A valid caller asks for a record id that belongs to another brokerage.
    2. PDP on the record + repository predicate: The service decides on the loaded record; the repository adds the tenant from the verified subject.
    3. cross-tenant matrix: Every principal with a genuine token against every other tenant's or unit's record, by both ids.
  4. Agent application to tool server
    1. E: agent asks for uw.decide: A prompt or a bug makes the agent application ask for a power no agent may hold.
    2. token exchange cap + R-AZ-09: The token service never issues uw.decide to an agent; the PDP refuses decisions by any non-person even if the scope slipped through.
    3. UW-A10, UW-A22: Exchange refused; misgranted scope still refused.
  5. Cockpit to policy decision point
    1. E, D: peer decides; PDP down: A policy version without the assignment check, or a PEP that allows when the PDP does not answer.
    2. assignment check, fail closed (503): Rule R-AZ-09 checks assignment on the record; the PEP refuses when it cannot get a decision.
    3. peer row + outage test: A negative row for the peer underwriter under every policy version shipped; a test with the PDP made unavailable.
  6. Services to audit log
    1. R: edited record; I: token in a log: Repudiation by editing a deny into an allow; disclosure by writing a bearer token into a log line.
    2. hash chain + redaction: Each record carries the hash of the previous one; tokens and keys are never written.
    3. UW-A24: Scan every audit record for token or key material; verify both chains.
BoundarySTRIDEThreat in this flowControlProved by
broker browser to portal BFFSscript steals a token; cross-site form posttokens kept at the BFF, HttpOnly cookie, CSRF defensesession lab outcome table
portal BFF to quote APIStoken for another audience, or an ID token, presentedaudience check before any claim is usedUW-A05; substitution run below
portal BFF to quote APITscope edited, alg set to none, signed with a guessed keyalgorithm allowlist, signature before claimsUW-A07, UW-A08; token lab
portal BFF to quote APISexpired or revoked token replayedexp, deny list or introspectionUW-A06, UW-A16
gateway to quote serviceIvalid token, another tenant's quote (BOLA)PDP decides on the record, not the routeUW-A03; test_service_hop_decides_on_the_record_not_the_token
caller to quote serviceSforged x-app-id: svc-cockpit-bff headerpeer identity from mTLS, not a headerUW-A04; test_x_app_id_header_is_not_identity
quote service to databaseIforgotten check reads another tenant's rowrepository predicate from the verified subjecttest_repository_hop_applies_the_tenant_predicate_even_without_a_pep_check
any caller to decision.recordEbroker, auditor or agent calls the decision function (BFLA)R-AZ-02 scope, R-AZ-09 person and roletest_auditor_cannot_write; matrix below
agent application to token serviceEagent asks for uw.decidetoken exchange caps agent scopestest_agent_cannot_obtain_uw_decide_by_token_exchange
agent application to tool serverEagent acts beyond its delegated task (confused deputy)task names one submission, actions and expiryUW-A11; agent run step 7
cockpit to PDPEpeer underwriter decidesassignment check in R-AZ-09UW-A09
cockpit to PDPTdecision written on an answer that is no longer true (race)re-check at write, or one transactionTOCTOU run below
cockpit to PDPDPDP unreachable and the PEP allowsfail closed: 503, audit, alertUW-A23; test_pdp_outage_fails_closed_for_writes
services to audit logRa deny edited into an allowhash chain over recordsUW-A24; incident lab
services to audit logIa bearer token written to a logredaction; scan for token-shaped stringstest_audit_log_never_contains_a_token_or_key
identity governance to directoryEleaver keeps accessdeactivation, session revoke, short token lifetest_leaver_loses_access_and_session_at_once

Most rows are E or I: authentication is usually a library, authorization is usually your code, and your code is where it breaks. A row whose last column is empty is the next test to write.

The smallest negative test#

The smallest useful negative test changes one fact and keeps everything else valid, even generous. This one, from examples/uw-auth/test_negative.py, gives the underwriting agent everything a decision needs: scope, entitlement, role, assignment, a fresh strong sign-in, the cockpit as client. Only the principal type is wrong.

def test_agent_decision_refused_even_with_misgranted_uw_decide(self) -> None:
    bad = agent_subject(scopes=["uw.read", "uw.recommend", "uw.decide"], entitlements=["uw.decide"], roles=["underwriter"],
                        assignments=["SUB-1004"], acr=ACR_STRONG, auth_time=NOW, client_id="svc-cockpit-bff")
    for v in (pdp.CURRENT_POLICY, pdp.PREVIOUS_POLICY):
        r = pdp.decide(bad, "decision.record", resource_for("SUB-1004", "submission"), CTX, v)
        self.assertEqual((r["allow"], r["rule"]), (False, "R-AZ-09"), v)

It checks both policy versions, because you may roll back. It asserts the rule that refused, so a refusal for the wrong reason does not pass. The model recommends; a person decides, and this test makes that sentence true in code.

The negative tests, attack by attack#

Each part explains an attack only as far as the control needs. The tokens are demo tokens signed with keys that say they are not secrets; use a maintained JWT library and a real identity provider in production. The agent here is a scripted stand-in, so these results show how the access controls work, not how good any model is.

Tampered, expired and revoked tokens#

An attacker edits the payload and keeps the old signature, sets alg to none, signs with a guessed key, or replays an expired or revoked token. The controls are the ordered checks of Tokens: an algorithm allowlist, the signature before any claim is read, then iss, aud, exp, nbf and revocation.

Standard (RFC 8725, read 2026-11-18): "Libraries MUST enable the caller to specify a supported set of algorithms", and RFC 7518 says implementations "MUST NOT accept Unsecured JWSs by default".

Guideline (OWASP API2, read 2026-11-18): an API is vulnerable if it "Accepts unsigned/weakly signed JWT tokens" or does not validate expiry.

The token lab opens on the attack presets: the validator stops at the first failing check, and authorization is never reached.

Attack presetValidation resultFirst failed check saysquote.read Q-5531
wrong-audrejected at audaud is https://assessment-svc.internal.example; this verifier accepts https://api.uw.insurer.examplenot reached
expiredrejected at expexp 1794993300, now 1794993301: expired or missingnot reached
tampered (scope edited, old signature kept)rejected at signatureHMAC-SHA256 with demo key demo-2026a does not matchnot reached
alg-nonerejected at algalg none is not on the allowlist (HS256)not reached
forged-keyrejected at signatureHMAC-SHA256 with demo key demo-2026a does not matchnot reached
revokedrejected at revocationjti jti-1d180a7211eb is revokednot reached
valid (control)valid(all checks pass)200 (R-AZ-03)

The control row matters: a validator that rejects every token passes all six attack rows.

Token substitution: the right token in the wrong place#

A token substitution presents a genuine, unmodified token at a recipient it was not issued for; the signature is perfect. Two forms matter here: an assessment-service token at the quote API, and the portal's ID token (which tells the portal backend who signed in) used as an access token.

Standard (RFC 8725, read 2026-11-18): section 2.7 describes attacks in which a recipient given a JWT "will attempt to use it at a different recipient for which that JWT was not intended", and section 3.9 says that when one issuer serves several relying parties, "the JWT MUST contain an "aud" (audience) claim", which the recipient validates. The snippet, written for this page, demonstrates the quote API's validator on each token; a successful run proves the audience check refuses both substitutions in this example.

import demo_keys
import oauth
import tokens
from clock import START_EPOCH_S as NOW

api = {"keyset": demo_keys.keyset(), "expected_issuer": tokens.ISSUER, "expected_audience": tokens.API_AUD, "now": NOW}
candidates = {
    "access token for the quote API": tokens.issue_for("u-uw-w1", aud=tokens.API_AUD, now=NOW),
    "access token for the assessment service": tokens.issue_for("u-uw-w1", aud=tokens.ASSESSMENT_AUD, now=NOW),
    "ID token for the portal BFF": tokens.sign(oauth.auth_code_trace(NOW)["id_token_claims"]),
}
for name, tok in candidates.items():
    r = tokens.verify(tok, **api)
    detail = next((c["detail"] for c in r["checks"] if c["name"] == r["first_failed"]), "all checks pass")
    print(f"{name:40} valid={r['valid']!s:5} first_failed={r['first_failed']} | {detail}")

Run in a scratch copy of examples/uw-auth on 2026-11-18:

access token for the quote API           valid=True  first_failed=None | all checks pass
access token for the assessment service  valid=False first_failed=aud | aud is https://assessment-svc.internal.example; this verifier accepts https://api.uw.insurer.example
ID token for the portal BFF              valid=False first_failed=aud | aud is svc-portal-bff; this verifier accepts https://api.uw.insurer.example

Diagram: Token substitution · two genuine tokens, the wrong place, refused at the audience check

Time runs down the page. Both tokens are correctly signed by the identity provider. The first was issued for the assessment service; the second is the portal's ID token, issued so the portal backend knows who signed in. Presented at the quote API, both fail the same check: the audience is not this API. Only the third request, with an access token for the quote API, reaches the policy decision point.

  1. Caller with a stolen or reused token to Quote API (resource server): Bearer: assessment-service token. A genuine access token whose aud is https://assessment-svc.internal.example.
  2. Quote API (resource server) to Caller with a stolen or reused token: 401: rejected at aud. Signature, issuer and expiry pass; the audience does not name this API.
  3. Caller with a stolen or reused token to Quote API (resource server): Bearer: portal ID token. An OpenID Connect ID token, aud svc-portal-bff, used as if it were an access token.
  4. Quote API (resource server) to Caller with a stolen or reused token: 401: rejected at aud. Same check, same answer. An RFC 9068 typ check (at+jwt) would refuse it a second way.
  5. Caller with a stolen or reused token to Quote API (resource server): Bearer: access token for the API. The right token for this recipient.
  6. Quote API (resource server) to Policy decision point: decide quote.read. Now, and only now, the record is decided on.

The demo validator relies on the audience alone. Standard (RFC 9068, read 2026-11-18): JWT access tokens carry typ at+jwt, and a resource server rejects a token whose typ is not that value. That second check refuses an ID token even in a deployment where a careless configuration gave it the API's audience. The demo tokens carry typ JWT and the example does not check it; a production validator should.

BOLA and BFLA: the wrong record, and the wrong function#

Two different holes share one cause: the check ran on the token instead of on the request. Guideline (OWASP API Security Top 10 2023, read 2026-11-18) lists them as API1, Broken Object Level Authorization, and API5, Broken Function Level Authorization.

BOLA (also called IDOR) is the right function on the wrong record: broker u-broker-n1, with a valid token and the scope quotes.read, asks for Q-6620 of brk-south. The gateway-only design returns it (UW-A03); the service check answers 404. The full explanation, with the hops lab, lives on Enforcement.

BFLA is the wrong function: "legitimate API calls to an API endpoint that they should not have access to" (OWASP API5). Here, a broker or the auditor calling decision.record, or the agent calling the tool record_underwriter_decision. OWASP's advice is to "deny all access by default, requiring explicit grants to specific roles". pdp.decide refuses each at a named rule: R-AZ-02 for a missing scope, R-AZ-09 for anyone who is not a person with the underwriter role.

Common confusion. Hiding the decision button for brokers is not a BFLA control. The cockpit's UI is a convenience; the check that counts runs on the server, for every caller, including ones that never load the UI.

Cross-tenant access: a matrix you can add to your suite#

Cross-tenant tests are repetitive and easy to leave half-done. Tenants lists the paths to cover; this is the shape that keeps them complete: a table of rows, one test that walks it, and unittest's subTest so one failing row does not hide the others. Standard library only; copy it next to test_negative.py.

"""A cross-tenant and cross-function matrix against pdp.decide (written for the threat-testing page)."""
import unittest

import pdp
from clock import START_EPOCH_S as NOW
from directory import resource_for, subject_for

CTX = {"now": NOW}

# principal, action, record, expected status. Every subject is genuine; only the record or the function is wrong.
DENY = [
    ("u-broker-s1", "quote.read", "Q-5531", 404),         # other tenant, by quote id
    ("u-broker-s1", "quote.read", "SUB-1004", 404),       # the same record, by submission id
    ("u-broker-s1", "quote.submit", "Q-5532", 404),       # other tenant, a write
    ("u-broker-n1", "quote.read", "Q-6620", 404),         # the other direction
    ("u-uw-e1", "submission.read", "SUB-1004", 404),      # other business unit
    ("u-uw-w1", "submission.read", "SUB-2001", 404),
    ("u-broker-n1", "decision.record", "SUB-1004", 403),  # BFLA: a function brokers never call
    ("u-auditor-1", "decision.record", "SUB-1004", 403),  # BFLA: read-only role, write function
]
# Control rows: if these fail, the suite is testing a PDP that denies everything.
ALLOW = [
    ("u-broker-n1", "quote.read", "Q-5531"),
    ("u-broker-s1", "quote.read", "Q-6620"),
    ("u-uw-w1", "submission.read", "SUB-1004"),
]


class CrossTenantMatrix(unittest.TestCase):
    def test_every_cross_boundary_row_is_refused(self) -> None:
        for principal, action, record, expected in DENY:
            with self.subTest(principal=principal, action=action, record=record):
                d = pdp.decide(subject_for(principal, now=NOW), action, resource_for(record), CTX)
                self.assertFalse(d["allow"], d["reason"])
                self.assertEqual(d["status"], expected, d["reason"])

    def test_control_rows_are_allowed(self) -> None:
        for principal, action, record in ALLOW:
            with self.subTest(principal=principal, action=action, record=record):
                self.assertTrue(pdp.decide(subject_for(principal, now=NOW), action, resource_for(record), CTX)["allow"])


if __name__ == "__main__":
    unittest.main()

Run in the scratch copy on 2026-11-18 (/opt/homebrew/bin/python3 -m unittest test_cross_tenant_matrix -v):

test_control_rows_are_allowed (test_cross_tenant_matrix.CrossTenantMatrix.test_control_rows_are_allowed) ... ok
test_every_cross_boundary_row_is_refused (test_cross_tenant_matrix.CrossTenantMatrix.test_every_cross_boundary_row_is_refused) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Three choices carry the value. The expected status is in each row, so a 403 (which confirms the record exists) fails as surely as a 200. The same record appears as Q-5531 and as SUB-1004, because authorization is decided on the record both ids point to. And the control rows catch the cheapest false pass: a policy that denies everything. Guideline (OWASP Authorization Testing Automation Cheat Sheet, read 2026-11-18) describes the same structure at larger scale: "Formalize the authorization matrix" in a file programs can process and people can review, and run one test per point of view.

Confused deputy: the agent's own authority is not the task's#

A deputy is a component with more authority than its caller, acting on the caller's behalf. It is "confused" when a caller gets it to use its own authority for something the caller could not do.

Guideline (MITRE CWE-441, read 2026-11-18) gives "Confused Deputy" as the alternate term for an intermediary that forwards a request without keeping its original source.

The underwriting agent is a deputy. As a workload of unit commercial-west it could reach every west submission; acting for u-uw-w1 on run run-7f3a, its delegated token names one submission, SUB-1004, a list of actions and an expiry. The test asks for something the workload could reach but the task does not cover: a recommendation for SUB-1007. Case UW-A11 expects 403 ("outside the delegated task"), and refusal again after the task expires. The token service applies the same idea to audiences: an exchange targeting the LLM gateway gets invalid_target (UW-A22).

What this is not. The injected passage that asked the agent for such writes (case UW-005 shape) is prompt injection; defenses against it are on Guardrails Engineering. This page tests that the request, once sent, is refused. Delegation design: Agents and MCP.

Privilege escalation: asking for more, and policies that check less#

Privilege escalation is a principal ending up with a power it was never granted. The insurance flow has two shapes of it.

Vertical: the agent asks for uw.decide. Three controls stack: the token service refuses the exchange with invalid_scope (agent run step 2); the tool server refuses record_underwriter_decision at its scope check (step 6); and with uw.decide misgranted, R-AZ-09 still refuses. Each layer has its own negative test, so removing one fails a test.

Horizontal: the peer underwriter decides a colleague's submission. This is the opening's drift, and the case shows both versions:

UW-A09  PASS   Peer underwriter decides (policy drift)              current={"allow": false, "status": 403, "reas...
    ok  current: expected {"allow": false, "status": 403, "reason": "underwriter is not assig..., actual {"allow": false, "status": 403, "reason": "underwriter is not assig...
    ok  previous: expected {"allow": true, "status": 200, "policy_version": "authz-2026.10.3"}, actual {"allow": true, "status": 200, "policy_version": "authz-2026.10.3"}

The previous line is the hole, recorded on purpose. A negative row for u-uw-w2 on SUB-1004, run against every deployable policy version, is the test the Thursday release lacked (versions: Policy as code).

Race conditions: check, then the facts change, then act#

A TOCTOU race (time of check to time of use) happens when access is checked, the facts change, and the action runs on the old answer.

Guideline (MITRE CWE-367, read 2026-11-18): "The product checks the state of a resource before using that resource, but the resource's state can change" in between.

In the cockpit, u-uw-w1 opens SUB-1004 and the backend asks the PDP: allowed. Five seconds later a supervisor reassigns the submission to the peer. If the backend then writes on the answer it got when the page loaded, a decision is recorded by someone no longer assigned. The snippet was written for this page; a successful run demonstrates the stale answer and the re-check, not a real concurrent race.

# Written for the threat-testing page (not part of the example): check-then-act on a reassigned submission.
import pdp
from clock import START_EPOCH_S as NOW
from directory import ACR_STRONG, resource_for, subject_for

assignments = {"SUB-1004": "u-uw-w1"}          # the live directory; a supervisor changes it below
res = resource_for("SUB-1004", "submission")


def cockpit_subject(now):
    s = subject_for("u-uw-w1", now=now, acr=ACR_STRONG, auth_time=NOW, client_id="svc-cockpit-bff")
    s["assignments"] = [sub for sub, uw in assignments.items() if uw == "u-uw-w1"]   # read the facts now
    return s


checked = pdp.decide(cockpit_subject(NOW), "decision.record", res, {"now": NOW})
print("09:00:00 check:", checked["status"], checked["reason"])

assignments["SUB-1004"] = "u-uw-w2"             # 09:00:05 the submission is reassigned to the peer

print("09:00:06 check-then-act writes:", "yes (stale answer)" if checked["allow"] else "no")
at_write = pdp.decide(cockpit_subject(NOW + 6), "decision.record", res, {"now": NOW + 6})
print("09:00:06 re-check at write:", at_write["status"], at_write["reason"])
09:00:00 check: 200 assigned underwriter decides through the cockpit after step-up
09:00:06 check-then-act writes: yes (stale answer)
09:00:06 re-check at write: 403 underwriter is not assigned to SUB-1004

Diagram: Check then act · a reassignment between the check and the write (TOCTOU)

Time runs down the page. The cockpit backend asks the policy decision point at 09:00:00 and gets allow. Five seconds later a supervisor reassigns SUB-1004 to the peer. If the backend writes on the earlier answer, a decision is recorded by someone no longer assigned. The fix is to decide at the moment of the write: re-check, or make the write itself conditional on the assignment in one transaction.

  1. Cockpit backend (PEP) to Policy decision point: 09:00:00 decide. u-uw-w1, decision.record, SUB-1004.
  2. Policy decision point to Cockpit backend (PEP): allow 200. True at 09:00:00.
  3. Supervisor reassigns to Submission store: 09:00:05 reassign. The fact behind the allow changes.
  4. Cockpit backend (PEP) to Submission store: write on the old allow. The race: the write trusts an answer about a world that no longer exists.
  5. Cockpit backend (PEP) to Policy decision point: re-check at write. The safe shape: decide again inside the write path, or make the write conditional on the assignment.

Re-checking at write time narrows the gap; closing it means deciding and writing in one step. In a database that is a conditional write: the update names the fact the decision depended on, and zero rows updated means "refuse".

-- shape only, not run here: the write itself carries the assignment condition
UPDATE submissions
   SET decision = :decision, decided_by = :caller, authz_policy_version = :version
 WHERE submission_id = :submission_id
   AND assigned_underwriter = :caller
   AND decision IS NULL;
-- 0 rows updated: the assignment changed or someone already decided; answer 409 or 403, write nothing

A SELECT ... FOR UPDATE on the row, in the same transaction as the PDP call, is the other common shape. And never cache a decision for a protected write (Enforcement).

Access after revocation and failing open#

Access after revocation: a leaver's session and directory access must go at once, while a self-contained token stays valid until exp unless the API checks a deny list or introspects. The lifecycle tests prove both halves, and UW-A16 shows the window with and without a deny list. How fast access must go is on Operations.

Failing open: a PEP that allows when the policy decision point is down has turned an outage into a breach. Fail open is usually an accident of code shape (a try/except that logs and continues), which is why it needs a test. The example's PEP answers 503 and writes nothing (UW-A23):

UW-A23  PASS   IdP or PDP outage: writes fail closed                pdp_down_write=503
    ok  pdp_down_write: expected 503, actual 503
    ok  pdp_down_read: expected 503, actual 503
    ok  idp_down_write: expected 401, actual 401
    ok  idp_down_read_with_valid_token: expected 200, actual 200
    ok  decisions_written: expected 0, actual 0

The full design, including why 503 and not 403, is on Enforcement. The testing point is narrower: for every dependency a decision needs (the PDP, the key set, the directory, the audit log), write one test that makes it fail and asserts the refusal.

The definition-of-done evidence#

The site's specification (section 10) asks for three demonstrations: item 4, a resource-specific decision at every sensitive hop, not a valid token or an x-app-id header; item 5, negative tests proving that unauthorized users and agents cannot read another tenant's quotes, execute restricted tools or perform protected underwriting writes; item 6, accurate lifecycle consequences. Each maps to a test class:

Definition-of-done itemTest class in test_negative.pyTests
4: a resource-specific decision at every sensitive hopDod4ResourceDecisionAtEverySensitiveHop4
5: no cross-tenant read (broker, underwriter, agent, retrieval, precedent search, MCP)Dod5NoCrossTenantRead7
5: no restricted tool executionDod5NoRestrictedToolExecution5
5: no protected underwriting write by an agentDod5NoProtectedWriteByAgent5
6: provisioning, review, JIT elevation, revocation, auditDod6LifecycleConsequences7

Run in a scratch copy of examples/uw-auth on 2026-11-18 with /opt/homebrew/bin/python3 -m unittest test_negative -v, output as printed:

test_every_audit_record_of_a_decision_names_rule_resource_and_policy_version (test_negative.Dod4ResourceDecisionAtEverySensitiveHop.test_every_audit_record_of_a_decision_names_rule_resource_and_policy_version) ... ok
test_repository_hop_applies_the_tenant_predicate_even_without_a_pep_check (test_negative.Dod4ResourceDecisionAtEverySensitiveHop.test_repository_hop_applies_the_tenant_predicate_even_without_a_pep_check) ... ok
test_service_hop_decides_on_the_record_not_the_token (test_negative.Dod4ResourceDecisionAtEverySensitiveHop.test_service_hop_decides_on_the_record_not_the_token) ... ok
test_x_app_id_header_is_not_identity (test_negative.Dod4ResourceDecisionAtEverySensitiveHop.test_x_app_id_header_is_not_identity) ... ok
test_agent_cannot_read_another_units_submission (test_negative.Dod5NoCrossTenantRead.test_agent_cannot_read_another_units_submission) ... ok
test_broker_can_never_search_precedents (test_negative.Dod5NoCrossTenantRead.test_broker_can_never_search_precedents) ... ok
test_broker_cannot_read_another_tenants_quote_by_quote_id_or_submission_id (test_negative.Dod5NoCrossTenantRead.test_broker_cannot_read_another_tenants_quote_by_quote_id_or_submission_id) ... ok
test_mcp_tool_cannot_reach_another_units_submission (test_negative.Dod5NoCrossTenantRead.test_mcp_tool_cannot_reach_another_units_submission) ... ok
test_precedent_search_never_returns_another_units_precedent (test_negative.Dod5NoCrossTenantRead.test_precedent_search_never_returns_another_units_precedent) ... ok
test_retrieval_never_returns_another_tenants_chunk (test_negative.Dod5NoCrossTenantRead.test_retrieval_never_returns_another_tenants_chunk) ... ok
test_underwriter_cannot_read_another_units_submission (test_negative.Dod5NoCrossTenantRead.test_underwriter_cannot_read_another_units_submission) ... ok
test_agent_cannot_obtain_uw_decide_by_token_exchange (test_negative.Dod5NoProtectedWriteByAgent.test_agent_cannot_obtain_uw_decide_by_token_exchange) ... ok
test_agent_decision_refused_even_with_misgranted_uw_decide (test_negative.Dod5NoProtectedWriteByAgent.test_agent_decision_refused_even_with_misgranted_uw_decide) ... ok
test_agent_recommendation_outside_task_or_after_expiry_refused (test_negative.Dod5NoProtectedWriteByAgent.test_agent_recommendation_outside_task_or_after_expiry_refused) ... ok
test_injected_instruction_does_not_produce_a_write (test_negative.Dod5NoProtectedWriteByAgent.test_injected_instruction_does_not_produce_a_write) ... ok
test_pdp_outage_fails_closed_for_writes (test_negative.Dod5NoProtectedWriteByAgent.test_pdp_outage_fails_closed_for_writes) ... ok
test_agent_cannot_execute_the_decision_tool (test_negative.Dod5NoRestrictedToolExecution.test_agent_cannot_execute_the_decision_tool) ... ok
test_auditor_cannot_write (test_negative.Dod5NoRestrictedToolExecution.test_auditor_cannot_write) ... ok
test_discovery_hidden_tool_called_by_name_is_refused (test_negative.Dod5NoRestrictedToolExecution.test_discovery_hidden_tool_called_by_name_is_refused) ... ok
test_listed_tool_outside_delegation_is_refused (test_negative.Dod5NoRestrictedToolExecution.test_listed_tool_outside_delegation_is_refused) ... ok
test_revoked_grant_is_refused_although_still_listed (test_negative.Dod5NoRestrictedToolExecution.test_revoked_grant_is_refused_although_still_listed) ... ok
test_access_review_revoke_removes_the_group (test_negative.Dod6LifecycleConsequences.test_access_review_revoke_removes_the_group) ... ok
test_audit_log_never_contains_a_token_or_key (test_negative.Dod6LifecycleConsequences.test_audit_log_never_contains_a_token_or_key) ... ok
test_break_glass_leaves_alert_review_and_audit_trail (test_negative.Dod6LifecycleConsequences.test_break_glass_leaves_alert_review_and_audit_trail) ... ok
test_jit_elevation_expires (test_negative.Dod6LifecycleConsequences.test_jit_elevation_expires) ... ok
test_leaver_loses_access_and_session_at_once (test_negative.Dod6LifecycleConsequences.test_leaver_loses_access_and_session_at_once) ... ok
test_provisioning_needs_an_approver_other_than_the_requester (test_negative.Dod6LifecycleConsequences.test_provisioning_needs_an_approver_other_than_the_requester) ... ok
test_self_contained_token_outlives_deactivation_until_exp_only (test_negative.Dod6LifecycleConsequences.test_self_contained_token_outlives_deactivation_until_exp_only) ... ok

----------------------------------------------------------------------
Ran 28 tests in 0.008s

OK

Each test name is a claim that would be false without its control. The timing is what unittest printed, not a benchmark.

This site's own cases UW-A01 to UW-A24 run the same controls as scenarios, including holes on purpose. Run with /opt/homebrew/bin/python3 run_cases.py in the same scratch copy:

case    result title                                                first check
UW-A01  PASS   Own-tenant quote read                                status=200
UW-A02  PASS   Cross-tenant quote read is 404                       token_valid=true
UW-A03  PASS   Gateway-only check lets BOLA through                 gateway_only={"status": 200, "data_returned": true...
UW-A04  PASS   Forged x-app-id header                               trusting_service={"status": 200, "data_returned": true...
UW-A05  PASS   Token for another audience                           valid=false
UW-A06  PASS   Expired token                                        valid=false
UW-A07  PASS   Tampered payload                                     valid=false
UW-A08  PASS   alg none                                             valid=false
UW-A09  PASS   Peer underwriter decides (policy drift)              current={"allow": false, "status": 403, "reas...
UW-A10  PASS   Agent calls record_underwriter_decision              as_delegated={"allowed": false, "check": "scope"}
UW-A11  PASS   Agent recommendation outside its task                allowed=false
UW-A12  PASS   Decision without step-up                             status=401
UW-A13  PASS   RAG never returns another tenant's chunk             filter_first={"leaked": [], "A3_returned": false, ...
UW-A14  PASS   Precedent from another unit                          P2_returned=false
UW-A15  PASS   Kafka: consumer write and DLQ replay refused         consumer_write=false
UW-A16  PASS   Revocation window                                    self_contained_without_deny_list=true
UW-A17  PASS   Leaver                                               directory_allows=false
UW-A18  PASS   JIT elevation expires                                before=false
UW-A19  PASS   Break-glass is audited                               alerts=1
UW-A20  PASS   mTLS: expired, not allowlisted, rotated              expired={"handshake_ok": false, "failed": ["v...
UW-A21  PASS   MCP: discovery is not authorization                  hidden_tool={"listed": false, "allowed": false, "...
UW-A22  PASS   Token exchange narrows                               issued_scope=uw.read uw.recommend
UW-A23  PASS   IdP or PDP outage: writes fail closed                pdp_down_write=503
UW-A24  PASS   The audit log contains no token or key               leaks=[]

24/24 cases match their expected block. (Scripted stand-in agent: this shows the access controls, not model quality.)

"PASS" means the outcome matched the expected block, which for UW-A03, UW-A04 and the previous-policy line of UW-A09 is a recorded leak. Recording the hole next to the fix keeps the reason for each control visible.

The refusals, as an investigator sees them#

In production the same refusals show up as security events. The incident lab opens on the negative preset: only the authn.failure and authz.deny records, 13 of the 27 the example's audit log wrote.

Time (UTC)KindPrincipalRequestReason
09:00:00 to 09:00:10authn.failure (6)unknownreq-0001 to req-0006gateway: signature
09:01:32authz.denyu-broker-s1req-0008record belongs to another tenant: not found (existence not revealed)
09:01:37authz.denyu-broker-s1req-0009record belongs to another tenant: not found (existence not revealed)
09:03:47authn.failureu-broker-s1req-0011gateway: revocation
09:05:32authz.denywl-underwriting-agentreq-0015insufficient_scope: uw.decide required
09:05:47authz.denywl-underwriting-agentreq-0016record belongs to another business unit: not found (existence not revealed)
09:11:17authz.denyu-uw-w1req-0018step-up required: strong authentication within 300 s
09:22:22authn.failureunknownreq-0020gateway: kid

The two agent rows are the injected passage's two requests. The records themselves are explained on Audit.

In real frameworks (these ran)#

The same negative tests ran in maintained libraries (examples/frameworks/). Product (FastAPI 0.165.0 with PyJWT 2.15.1, run on 2026-11-18): ./run.sh <scratch-dir>, 19 passed. The negative lines:

test_app.py::test_no_token_401 PASSED                                    [  5%]
test_app.py::test_expired_401 PASSED                                     [ 10%]
test_app.py::test_wrong_audience_401 PASSED                              [ 15%]
test_app.py::test_wrong_issuer_401 PASSED                                [ 21%]
test_app.py::test_alg_mismatch_rs256_401 PASSED                          [ 26%]
test_app.py::test_alg_none_401 PASSED                                    [ 31%]
test_app.py::test_signed_by_other_key_401 PASSED                         [ 36%]
test_app.py::test_broker_reads_other_tenant_quote_404 PASSED             [ 47%]
test_app.py::test_other_tenant_and_missing_look_the_same PASSED          [ 52%]
test_app.py::test_tenant_claim_in_query_is_ignored PASSED                [ 63%]
test_app.py::test_missing_scope_403 PASSED                               [ 68%]
test_app.py::test_peer_underwriter_403 PASSED                            [ 84%]
test_app.py::test_other_unit_underwriter_404 PASSED                      [ 89%]
test_app.py::test_agent_with_uw_decide_scope_403 PASSED                  [ 94%]
test_app.py::test_broker_cannot_decide_403 PASSED                        [100%]

PyJWT proves the token is genuine and meant for this API; the tenant, assignment and agent checks are application code, and the tests prove that code.

Product (Spring Boot 4.1.2, Spring Security 7.1.2, run on 2026-11-18): mvn -q test, 17 tests, 0 failures. Among them:

"wrongAudience_401"
"algMismatchHs256_401"
"wrongIssuer_401"
"expired_401"
"signedByOtherKey_401"
"brokerReadsOtherTenantQuote_404"
"peerUnderwriter_403"
"agentWithUwDecideScope_403"
"jwtPostProcessor_skipsTheDecoder_soWrongAudienceStillPasses"

The last line is a warning written as a test. Product (Spring Security, as documented on 2026-11-18): the resource server does not validate aud by default, so the demo adds a validator; and the jwt() MockMvc helper builds a mock token "through any authentication APIs" without, as the run shows, the JwtDecoder: a jwt() request with the wrong audience got 200. Use jwt() for authorization tests; audience tests need real signed tokens, as the first five lines do.

What goes wrong without negative tests#

The opening is the common failure: positive tests only. Badly built negative suites fail more quietly.

FailureWhat it looks likeThe fix
Negative rows onlya policy that denies everything passes the whole suitepositive control rows next to every negative matrix
Broken credentials in authorization tests"cross-tenant" test sends an expired token; it proves authenticationgenuine credentials, one wrong fact
Status not asserteda cross-tenant read returning 403 passes, and confirms the record existsassert the exact status and the absent body
One policy version testedrollback reintroduces the peer-underwriter holerun negative rows against every deployable version

Choosing how much testing machinery#

SituationSimplest credible approachGrow whenAlternatives
One service, a handful of rolesnegative unit tests against the decision function, plus HTTP tests for 401/403/404more than one team writes policypolicy-engine tests (opa test ran in examples/frameworks)
Multi-tenant SaaSgenerated cross-tenant matrix, both identifiers, positive controlstenants with custom rolesmatrix in a reviewed file
Agents with toolsnegative tests per layer: token exchange, tool server, PDPmany tools and delegationsper-tool argument tests
High-impact writesthe above plus race and outage tests, and a known-deny probe in productionalwaysscheduled penetration tests

This site's recommendation: start with the threat model table and give each row one negative test; generate matrices once there are more than a few principals; keep the cases that demonstrate holes (like UW-A03).

In a coding agent. A coding agent editing authorization code works through a repo-scoped GitHub App and proposes a pull request. The negative suite runs in CI on it; if the change lets u-uw-w2 decide SUB-1004, a test fails and the merge is blocked. A person with merge rights reviews the policy diff, and the agent's sandbox holds no production credential. See Coding agents.
What this is not. Measuring how often the agent proposes an out-of-scope action is evaluation, on Eval Engineering. This page proves that whatever it proposes, the controls refuse what is not allowed.

Security notes#

Monday checklist#

Exercises#

Exercise 1. A teammate's cross-tenant test uses a token whose exp is in the past, "to be safe". It passes. What has it proved?

AnswerOnly that expired tokens are refused

The request was refused at the expiry check, before any authorization ran. It proves nothing about tenant isolation. Use a valid token of the other tenant, so the only wrong fact is the record.

Exercise 2. Your matrix test has 40 deny rows and no allow rows. A refactor makes pdp.decide return deny for every request. What does the suite report?

AnswerAll green

Every deny row still passes. Positive control rows (u-broker-n1 reads Q-5531) are what fail, and asserting the expected rule or status on each deny row catches refusals for the wrong reason.

Exercise 3. Which STRIDE letter and which control fit "the cockpit records a decision on an allow it got two minutes earlier, after the submission was reassigned"?

AnswerE (or T), closed by deciding at write time

It is elevation of privilege through a race: a person acts with a power they no longer hold (some teams file it under tampering, an unauthorized change of data). The control is a conditional write on the assignment in one transaction, or a re-check immediately before the write, and no decision cache for protected writes.

Advanced perspective#

Generate the matrix from the directory and the policyHand-written rows miss the combinations nobody thought of

A hand-written matrix encodes the author's mental model, which is where the gaps come from. The stronger form generates rows: every principal in the directory × every action × every record, with the expected outcome computed from a small, separately written statement of intent (for example "brokers only read and submit in their own tenant; only an assigned person who is an underwriter decides"). The test then compares the PDP's answer with the intent for every cell, which for this example is 9 principals × 10 actions × 3 records, a few hundred cells. Two cautions. The intent must not be a copy of the policy code, or the test agrees with every bug. And generated suites need a readable failure report: print the cell (principal, action, record, expected, actual, rule) so a reviewer can tell a policy bug from an intent bug. The OWASP Authorization Testing Automation sheet's "one test per point of view" grouping is a good report shape.

Does the suite notice when a check disappears?Mutation testing for policy

A negative suite is only as good as the controls it would notice losing. The direct test: remove one condition from the policy (the assignment check, the tenant comparison, the client check for svc-cockpit-bff, the step-up), run the suite, and confirm at least one test fails for each removal. The example has one such mutation built in, the previous policy authz-2026.10.3, and UW-A09 shows it is detected. Doing this for every condition, by hand or with a tool, turns "we have negative tests" into "every condition in R-AZ-09 has a test that fails without it". Conditions with no failing test are either untested or unnecessary; both are worth knowing. This is the same idea measurement sites apply to evaluation suites, applied to access control.

Negative tests in productionA known-deny probe, and why it needs its own principal

CI proves the code; it does not prove the deployed configuration: a gateway route added by hand, a policy bundle that failed to load, a PDP that answers but with a stale version. A synthetic probe in production sends a few known-deny requests on a schedule (the peer underwriter on SUB-1004, a cross-tenant read) and alerts if any is ever allowed, alongside a known-allow to detect deny-everything. The probe needs its own principals, clearly named as synthetic, with no access to real tenant data even when the control fails: its cross-tenant target should be a synthetic record in a real tenant's boundary, created for the purpose. Its refusals are security events and should be tagged so they do not drown real alerts, and its credentials are as sensitive as any service account's.

Where STRIDE is weak for agent systemsIt finds threats per boundary; delegation crosses several

STRIDE asks its six questions at one boundary at a time. The agent threats that matter most cross several: a passage injected through retrieval (one boundary) makes the model propose a write (no boundary at all, it is text), which the agent application sends with a delegated token (a second boundary) to a tool server that calls a domain service (a third). Per boundary each step looks fine. Two habits help. Draw the delegation chain as its own path and ask, at the end, "whose authority is this write using, and did that person grant this action on this record?" And treat the model's output as an untrusted input crossing a boundary into the agent application, so every tool call it suggests meets a deterministic check. The model suggests an action; deterministic code decides whether it may run.

In one breath#

A threat model lists the trust boundaries, asks STRIDE's six questions at each, and gives every threat a control and a test that would fail without it. Negative tests keep every input genuine except one (the tenant, the function, the audience, the moment), assert the exact refusal, and sit next to positive control rows. In the example, 28 negative tests and 24 cases are the definition-of-done evidence. Races and outages need their own tests, because check-then-act and fail-open are bugs of timing and code shape, not of policy.

The tests prove the controls hold today. Keys still rotate, caches still need purging, the identity provider will go down one morning, and a leaver's access must vanish in a stated time. Operations runs those lifecycles without opening the holes this page tests for.