Generative Adversarial PR Reviews
That's it. That's the whole post.
Well actually...
A practice my coworkers and I have been playing with:
- Generate spec/plan/tasks/code -> Create new PR
- Run preferred PR Reviewer Agent against PR
- Run preferred Agent against Generated PR Review
- Run preferred PR Reviewer Agent against Revised PR
- Run preferred Agent against Generated second pass PR Review
- ... this continues until the work week is exhausted.
Except, how much of the generated Review items become actual code changes?
I've noticed that I get a better sense of my own (generated) code, by having others point their Review-of-Shame Ralph Agents at it. By defending my code, improving what I think is necessary, saying 'wontfix' with increasing prejudice, I create a surface area of the PR that I stand by.
This seems to be a repeatable pattern that is resilient over time. Your preferred Agent/Reviewer agent may vary. Even running various ones against a branch before submitting a PR is useful, but after a point wearying.
My biggest complaint is that the careful consideration of code that was created is coming from the top down now, iteratively deepening, rather than the sideways and bottom-up that I usually take pre-LLM era.
Its exhausting because I have to read, think, and examine, retroactively. When normally I would read, think, design, build... once, then stand by my work and defend it with a reasonably open mind.
That last bit is largely lost here with LLMs. My work is not really my own until I get my hands into it, and thats now the last mile... not the first mile.