The legal team is shipping software now. Who reviews its permissions?

Ngandu Kasuku is a Principal Product Counsel at GitHub. Buried in partnership deals last spring, he used Copilot CLI to build himself a contract drafting tool and named it terms-ai. The tool runs against a library of agreements he had already negotiated, and GitHub's post is careful about where that library lives: "These agreements remain in an approved, access-controlled internal environment."

That one sentence is the entire security model of a piece of software now running at GitHub. A lawyer built a tool whose primary asset is a corpus of executed contracts, and the guarantee protecting it is a clause in a blog post.

The control point for tools like this is credential issuance, not code review. Kasuku wrote terms-ai for himself, in plain language, and it worked: he reports cutting his review and drafting time roughly in half. Nobody was ever going to review it.

If your only lever is inspecting what a builder wrote, you have no lever at all, because the builder never opened a pull request you were watching.

Plain language is the new source code

The GitHub post opens by telling readers they already have the skills: "Whether you are just starting out or are not in a technical role at all, you likely already have the skills you need to build your own tools." The team writing it describes itself as "lawyers, program managers, and business professionals—not engineers". They asked for what they wanted in plain language, plugged into their repos, and the habit spread until every one of them was building something.

The second account in that post is the one security teams should read twice. Jesse Geraci, an Online Safety Counsel, started with a set of Copilot instructions for DMCA triage, code comparison, license checks, and circumvention review. His description of the build: "Instead of writing source code, I was able to use my language crafting skills as a lawyer to build structured legal judgment into the workflow itself."

Then it grew. He handed the workflow to the team, who started using it immediately and asked for more. It picked up separate analysis modes for clients and lawyers, integrated external data sources, and became a full desktop app covering contract review, NDA triage, risk assessment, compliance checks, and response drafting, routing work through reusable skills and agents for intake, playbook alignment, risk scoring, evidence verification, escalation routing, and report assembly. Geraci is clear that it keeps human review central and does not replace legal judgment. The post ends by telling every reader to go do the same: pick one thing that slows you down, open the CLI, ask it to build a fix.

That trajectory (personal prompt file, then team workflow, then app with data integrations) breaks the usual assumptions about how software arrives. A tool that reads production data and calls internal APIs used to come with a repo, a reviewer, a deploy pipeline, and an on-call owner. Now it comes out of someone's afternoon and outlives the quarter that produced it.

Code review was never going to catch this

Niklas Gruhn's "Don't be a meat proxy" landed the day before GitHub's post and hit 1,777 points and 713 comments on Hacker News. His argument is about relayed output:

By all means, prompt AI.

But don't just relay the output.

Read it, understand it, validate it, and then write a response in your own words

(a decent certificate that you've done the prior steps).

Making that effort is value you can add.

The certificate framing is the useful part. Understanding is invisible, so we accept proxies for it: an explanation in someone's own words, a name on an approving review. Every accountability system we have for software runs on proxies like these.

Agent-built departmental tooling removes them all at once. There is no reviewer to sign off, no shared repo to gate, and often no second person who has read the workflow instructions at all. The person who understands the tool best is the person who prompted it into being. That person is a lawyer with a contract deadline, and they have probably never met whoever owns your data classification policy.

You can respond by demanding review anyway. It will fail for the reason review always fails at volume: the reviewer is scarce and the builders are now every department. The alternative is to move the control from the artifact to the credential, from "what does this tool do" to "what can this tool reach, on whose authority, and for how long."

Give each tool its own credential

Three questions decide the blast radius of a departmental tool, and all three are answerable before a line of it exists.

Who does the tool act for? A tool one person built for their own work should carry that person's identity. User-scoped API keys do this structurally: the key belongs to an individual rather than the organization, inherits the permissions that person already has, appears under their profile, and is revoked when they are offboarded. Kasuku's contract tool is exactly this shape. It should be able to read what he can read and stop there. The tradeoff is worth saying out loud: a user-scoped key cannot be restricted below the user's own permissions, so if the person can issue refunds, so can their tool.

Does anyone still own it? Geraci's workflow stopped being personal the moment he handed it to the team. A tool serving an org-wide workflow needs a credential that nobody's departure breaks, which means an org-scoped key with a narrowed permission set: you designate which permissions are even available to keys, then issue one carrying only what that tool needs. Admins holding widgets:user-api-keys:manage-all get visibility across every key in the org, which is the closest thing you will get to an inventory of what your non-engineers have built. Let builders mint their own with widgets:user-api-keys:manage-self, and remember that keys are shown in full only at creation and masked afterward.

Does it run unattended? A nightly triage job with no human watching should not hold a long-lived secret. M2M applications issue short-lived JWTs through the OAuth client credentials flow: the tokens expire on their own without anyone remembering to revoke them, and they validate locally against a cached JWKS.

Get the second question wrong and the failure is quiet. A team tool running on one person's personal key keeps working after that person changes roles, right up until their permissions get trimmed and a workflow nobody remembers owning starts failing in a way nobody can trace. The other direction is worse: the credential outlives the person, and an orphaned tool keeps reading contracts on the authority of an account that should have been closed.

None of this requires the builder to understand any of it. The key widget, the permission list, and the expiry are platform decisions made once. What the builder experiences is a paved path: ask for access, get a credential that already fits.

Storage discipline is the part you cannot hand to a non-engineer. WorkOS keys are hashed at rest with no reversible path back to the original value. That protects the issuer, not the builder who pasted a key into a config file that syncs to a personal cloud drive.

Scope the data before the tool exists

Credentials bound what a tool may call. They do not bound what it may see, and for departmental tools the data is the whole point: the contracts, the customer records.

A Launch HN post on July 30 introduced Prized, a YC company whose pitch is letting non-engineer staff build secure internal tools. Its architecture is a useful reference for the data layer regardless of what you make of the product. The build sandbox holds no keys or connector secrets at all, only opaque placeholders for scoped session tokens, with the real values swapped into request headers at an egress proxy and the sandbox network set to deny by default once production data is connected. Each tool gets its own Postgres schema and role, queries run through an authenticated SQL gateway as that role, admins scope data to specific users or teams, and access is recorded in an audit log.

Per-tool database roles and a default-deny egress path are ordinary platform engineering. They are being sold here as the prerequisite for letting non-engineers build, rather than as hardening applied afterward. One commenter on that thread named the remaining gap precisely: no good governance model exists for LLM integrations, and what they wanted was a rule that an agent with access to PII cannot also hold Slack and internet access. That is a constraint on composition rather than a decision about any single call. You can only enforce it at issuance, before the tool gets wired together.

The audit log has to name a person

When a departmental tool does something regrettable, emails the wrong counterparty or exports a contract set to the wrong place, the question is who is accountable. A shared service account answers "the automation did it," which is not an answer.

An agent audit record needs to carry the human, the agent's own identity, what that agent was authorized to do, what it actually did with which arguments, on whose behalf, and whether a human approved it. RFC 8693 defines the on-behalf-of token exchange that makes the delegation chain real at the token level, and the audit record's job is to reflect it at the record level. For a legal team's DMCA tool, a record carrying those fields looks roughly like this:

  
    {
  "user": {
    "id": "user_01HXKP2M",
    "email": "counsel@example.com"
  },
  "agent": {
    "id": "agent_dmca_triage",
    "version": "3.1.0"
  },
  "authorization": {
    "scopes": [
      "reports:read",
      "notices:write"
    ],
    "session_type": "human_approved",
    "expires_at": "2026-08-04T15:21:03Z"
  },
  "action": {
    "tool": "export_report",
    "arguments": {
      "notice_id": "dmca_88pk3r"
    }
  },
  "delegation_chain": [
    {
      "actor": "user_01HXKP2M",
      "role": "initiator"
    },
    {
      "actor": "agent_dmca_triage",
      "role": "executor"
    }
  ]
}
  

‍


Your existing logging stack will not produce that, and adding a few fields to it will not fix the gap. Application logging is built for high-volume, low-context events: it aggregates, samples, and discards. Audit needs completeness over sampling, immutability, retention measured in years, and indexes that let you query by identity. You cannot sample 1% of what a legal team's DMCA tool did to a customer's account and call it a record.

The strongest argument against all of this

The best objection is that governance kills the thing that made this valuable in the first place. Kasuku halved his drafting time because there was nothing standing between his idea and a working tool. Put a credential request, a data scoping review, and an audit integration in front of that, and he drafts contracts by hand for another quarter while he waits on the platform team.

A commenter on that Launch HN thread made the pragmatic version of the counter:

Maybe that's a hot take, but it's probably still safer than how things are often done in reality when employees want to "get something done" and sign up for random services, download their own software or other variants of shadow IT.

He is right, and it matters what you compare against. The comparison class is not a well-governed engineering process. It is a marketing lead with a personal API token and a spreadsheet. A paved path with scoped credentials beats that even when the path is imperfect.

The failure mode to avoid is the one Prized's own founder was direct about when a commenter pushed on whether an LLM reviewer could enforce safety:

You're right, and we don't. The judge is best-effort screening not enforcement. Enforcement is deterministic with per-tool Postgres roles, proxy-injected creds, host allow-list, human approval on destructive writes. Those hold whether the judge is right or wrong.

Substituting a model's judgment for a permission boundary is how you end up with governance theater: a review step that produces confidence without producing constraint.

What to go look at

Find your company's terms-ai. Not the sanctioned tools. The prompt files in someone's personal repo, the desktop app the ops team passes around, the script that reads from the production replica every Monday. For each one, answer three questions: whose credential does it hold, what can that credential reach, and whose name appears in the log when it acts.

If any answer is a shared service account, you do not have an AI governance problem to solve next quarter. You have an unowned production integration, built by someone who was never told that is what they were building, and the fix is to issue them something better than what they took.