How to check an AI-built app for leaked API keys

A coding agent puts a key where it makes the code run. Often that is the file that uses it, and sometimes that file is sent to every visitor's browser. Nothing on the screen looks different. This page shows the three places a key leaks from, two ways to look for one, and the order to do things in when you find one.

The three places a key leaks from

The code you push. A key written into a source file, or into an environment file that git is tracking, is readable by anyone who can see the repository. For a public repository that is everyone.

The built site. Whatever the screen-side code can read is sent to the browser, so a visitor can read it too. With Vite, which many agent-built apps use, any variable whose name starts with VITE_ is compiled into the public files. Vite's own documentation says these variables should not hold sensitive information such as API keys.

The history. Deleting a key from a file does not remove it from earlier versions of that file. GitHub's guidance is that the first step is to revoke or rotate the secret, because copies of the old commits may still be reachable elsewhere.

Check the code

For a public GitHub repository, the quickest look is the nerdsystem scanner. Paste the repository's address and it reports, in a few seconds for most apps:

• keys in the formats used by AWS, GitHub, Stripe, Slack, Google, OpenAI, Anthropic, and SendGrid, plus private keys and database addresses with a password in them
• long values assigned to names such as api_key, secret, or token
• environment files that were committed
• packages with known vulnerabilities

It needs no account. Values are shown masked, the code is not kept, and the report gets its own address that you can share. It stops after 3,000 files or 40 MB of source and refuses repositories over 400 MB.

Know what it is not. It matches patterns in the files as they are now, so a key in an unusual format can pass, and it does not scan git history. A clean report is not a security audit.

For a private repository, put this to your coding agent:

Search this project for API keys, tokens, passwords, and private keys in every file, including environment files that git is tracking. For each one, give me the file and line and show only the first four and last four characters. Do not print a full value.

Check the built site

The code can be clean while the built site is not, because the build copies values in. Ask your agent:

Build the app, then search the built output folder for the same kinds of keys. Also list every environment variable the screen-side code reads, and tell me which of them are secrets.

A good rule to hold your agent to: a secret is used by the server and nowhere else. The screen asks your server, and your server asks the outside service.

If you find one

Do these in order.

1. Replace the key at the provider. Make a new one and delete the old one. Do this before touching the code. The old key is already out, and cleaning the file does not bring it back.
2. Look at the provider's usage page for activity that was not yours.
3. Move the new key to the server side, into the place your host keeps secrets, and have the agent change the code to read it from there.
4. Then remove the old key from the code. By now it opens nothing.

How nerdsystem handles it

For an app on nerdsystem, the same check runs on every deploy, on the source and on the built files. A key in either one stops the deploy, and the agent is told which file and line and what to change. A key found in a file that git tracks is handed to you to replace at the provider, and the agent is told to stop until you have. Secrets are set with one command and are not shown again afterwards.

That covers the patterns above. It does not make the app's own rules right, and it is one check among several, not a review of everything.

To start an app with those checks in place, paste this into your coding agent:

nerdsystem is free to start, no card. The sign-in email the agent sends creates your account; there is no password and no sign-up form.