Save Your Tokens for Judgment Calls

I once watched an LLM burn through a hundred thousand tokens reviewing code I’d already written static analysis rules for. The model was doing work that a script could do better, faster, and without hallucinating function names. I was paying for something I could have gotten for free.

Most people use LLMs as a runtime. Prompt for everything, every time. Dan Miessler has a better ratio: 80% code, 20% LLMs. Code handles the exhaustive, repeatable work; LLMs handle judgment calls over novel inputs. That split eliminates what models do badly and preserves the one thing that actually requires inference.

Self-linters

A self-linter is a project-specific check that no generic tool covers. matklad writes about this as one of the basic things every serious project needs. TigerBeetle has tidy.zig: no large blobs in git, no merge commits on feature branches, markdown wrapped at 80 columns.

This is where LLMs fit perfectly. You ask once: “What project-specific lint checks should this codebase have?” The model writes the checks, they run in CI forever, and its job is done.

They’re especially good at catching vibe-coding habits. LLMs love generating documentation: README files, CONTRIBUTING guides, markdown scattered across every directory. The drift accumulates fast. They mix casing conventions in filenames—README.md in one directory, readme.markdown in another—and reach for patterns that work in general but go against your project’s style. When you notice yourself correcting the same thing over and over, that’s a lint rule waiting to be written. Catch it at commit time instead of in review.

Project-specific linters encode your project’s actual standards. They’re auditable, version-controlled, reviewable. A prompt is none of those things.

Git hooks and small scripts

“What are useful git hooks for this project?” is a question that produces a set of scripts. The LLM answers once, the hooks fire on every commit, pre-push, post-merge, and after that you never burn another token on it.

Same idea for monitoring. You could ask an LLM “check if any interesting PRs landed on this repo today,” or you could have it write a script that runs gh pr list --json, filters by label and file path, and posts a notification. Put that on a cron job and it never hallucinates a PR number, never costs anything to run.

I use this pattern for tracking changes across bug bounty repositories. A small script calls gh and git commands, scores PRs by security relevance, and surfaces the ones worth reading. The LLM wrote the scoring logic once; the script has executed it thousands of times since without ever touching the model again.

Narrowing the search space

The examples above are all cases where code fully replaces the LLM. The pattern gets more interesting when they work together.

Consider security review. An LLM doing freeform code review makes silent judgment calls about what to focus on—it might decide a component is low-risk because it “just” handles RPC serialization, or skip a function because the surrounding code looks well-structured. These shortcuts are invisible, and they won’t show up in the output or in chain-of-thought reasoning. You can’t audit what the model chose not to look at.

Static analysis inverts this. You write rules that surface every instance of a pattern you care about—unchecked error returns, unsafe type casts, functions that touch user input without validation—and the tool is exhaustive. Every match, every time, regardless of how boring the surrounding code looks.

Hand that list to the LLM and now it’s evaluating a complete, deterministic set of findings instead of deciding what to look at in the first place. The code eliminated the coverage gaps; the LLM provides the judgment static analysis can’t. Is this finding exploitable? Is the context safe? Does the surrounding logic make this a false positive?

Static analysis can’t assess exploitability. LLMs can’t guarantee they checked every file. Together, the coverage is complete.

There’s a trap here, though. Code that narrows the search space also constrains it. If your rules miss a class of vulnerability, the LLM will never see it. You can read the rules. You can’t read the model’s attention. The deterministic side gives you blind spots you can name. That’s better than blind spots you can’t.

Tools that write tools

My blog’s content management is a set of rake tasks the LLM wrote: create a draft with slugified filename and frontmatter template, validate structure across all posts, spellcheck against a custom dictionary, import from external sources, publish when ready. Every step has a correct answer and a binary pass/fail, so the LLM authored the pipeline in a few sessions and now it runs on every post without its involvement. (This post went through it.)

These tools won’t last forever. Standards evolve, scripts need updating. That’s a conversation with the LLM every few months, not a prompt on every commit.

The principle underneath

Most of what people prompt for has a correct answer and a deterministic path to it. Code is cheap, auditable, and never hallucinates. Move that work off the model and you sharpen what’s left: the LLM stops doing grunt work and starts doing the thing it’s actually good at, which is judgment over situations no one wrote a rule for.

The ratio isn’t fixed. Keep moving specifiable work to the code side as you learn what qualifies.