What GitAlert checks


GitAlert reads every diff the same way, every time, and flags the mistakes AI-written (and rushed human) code makes most. Every flag points to the exact file:line.

Gamed tests

The most common way an AI agent makes a suite "pass" is to weaken the tests instead of fixing the code. GitAlert looks for these patterns in the diff:

  • Weakened assertions — an assertion that was loosened so it can no longer fail.
  • Skipped or disabled tests — a test marked skip / xfail / ignored.
  • Tautological assertions — checks that are always true (for example assert True).
  • No-op “fixes” — a change that claims to fix something but changes no real behaviour.
  • Test files edited alongside source — a source change that also quietly edits its own tests.
  • New mocks that replace the thing under test.

A hard rule firing here is what moves a PR to Likely needs attention.

Made-up (hallucinated) dependencies

When a pull request adds a new package to a manifest — requirements.txt, package.json, and similar — GitAlert checks that the package actually exists on the real registry:

  • Python packages are checked against PyPI.
  • JavaScript packages are checked against npm.

If the package can't be found, GitAlert flags it. This matters because AI agents confidently import packages that don't exist, and attackers now register those fake names on purpose ("slopsquatting") to slip malware into your build. Only the package name is sent to the registry — never your code.

{tip} This is how GitAlert answers the question "is this dependency change risky?" — it catches a brand-new dependency that resolves to nothing (or to a name nobody has published yet) before it reaches your lockfile.

Sensitive-file changes

GitAlert surfaces changes to a fixed set of high-risk files, so a quiet edit buried in an otherwise boring diff never slips through unreviewed. The set is:

Category Files matched
Secrets & credentials .env, .secrets, id_rsa, *.pem, credentials, secrets.yml / secrets.yaml
CI & pipeline config .github/workflows/*.yml / *.yaml, .gitlab-ci.yml, Jenkinsfile
Container config Dockerfile, docker-compose.yml / docker-compose.yaml
Infrastructure variables terraform/*.tfvars

This is informational only — GitAlert records the touched path so a human can double-check it. It never blocks the merge on a sensitive-file change alone.

Build & test verification (paid)

On paid plans, GitAlert can go beyond reading the diff: it builds your project and runs its existing tests in an isolated sandbox and reports whether the change genuinely holds up — real evidence, not just a green checkmark. See Sandbox runs for exactly how that works and how your code is protected.