Green tests measure the wrong thing

A system I built produced junk content, and it passed 121 automated tests on the night it shipped.

I want to be precise about what those 121 tests actually checked, because the gap between 121 green checks and "this thing works" is the whole lesson. Every single test asked a machinery question: did the scheduled job fire, did the API call return a success code, did the row land in the right table with the right columns filled in. Those are real questions and worth testing. I do not regret writing any of them. But not one of the 121 asked the only question that actually mattered: was the content any good.

The feature shipped green. Dashboards lit up, the schedule ran on time, rows appeared where they were supposed to appear. By every metric the test suite tracked, the system was working exactly as designed. It was also producing content nobody would want to read, and nothing in the pipeline said so, because nothing in the pipeline was built to say so.

What actually caught it was not automation. It was me, sitting down and reading three pieces of the system's output by hand, the old fashioned way, with my own eyes. That reading did more to protect the business than the entire test suite, because a human reading the actual output is the only check that measures the thing you were trying to produce in the first place, rather than the plumbing that carries it.

This is an easy trap to fall into specifically because it looks like rigor. A build with 121 passing tests feels safer than a build with three. Green checkmarks are satisfying, legible, easy to report up. Quality, "is it any good," is none of those things. It is subjective and slow and resistant to automation, so it is the thing that gets skipped when a team is racing to ship. I skipped it. I am the whole team here in the sense that matters, and I still skipped it, because the machinery tests were sitting right there, easy to write and easy to trust.

The fix was not more tests of the same kind. It was adding one different kind of check: nothing reaches my approval queue anymore without a cold quality score attached first, a number that has nothing to do with whether the pipeline ran and everything to do with whether the output is worth a reader's time. That score is not perfect. It is a proxy, same as the 121 tests were proxies. But it is a proxy for the right thing.

The standing measure I use now for any content source, whether it is a person on my team or one of the AI agents running the pipeline, is not whether its tests pass. It is how often I skip what it sends me. A source I trust, I approve fast and rarely reject. A source I do not trust, I find myself opening the raw output and reading it line by line, the same kind of reading that caught the junk content in the first place. That instinct, how often you feel the need to double check something yourself, is a better quality signal than any test suite will ever give you. If you are running any kind of automated pipeline, human or AI, ask yourself right now: does anything in it actually check whether the output is good, or does it only check whether the machinery ran.