The Rubber Stamp Now Moves at Machine Speed

In November 2021, Zillow shut down its home-buying business: a $304 million write-down, a quarter of the workforce gone, roughly 7,000 houses to unwind, more than half a billion dollars in combined losses. The CEO's words: "the unpredictability in forecasting home prices far exceeds what we anticipated."

Here's the part the headlines missed. Zillow's algorithm wasn't uniquely bad. The failure was organizational: as the post-pandemic market turned, the humans stopped overriding it. Offer by offer, the company's judgment had been delegated to a model that would defend a wrong valuation exactly as confidently as a right one.

That's not a Zillow quirk. It's a documented human pattern with a name — automation bias — and it predates the AI boom by decades. Skitka and Mosier showed in 1999 that people with automated aids miss problems the aid doesn't flag and follow its recommendations even against contradictory evidence they were trained to recognize. The machine proposes; the human approves; the approving becomes reflex.

Speed is not wisdom. A confident answer arriving faster only means the rubber stamp now moves at machine speed.

The AI era didn't create that reflex. It supercharged it. KPMG's 47-country study found 66 percent of AI users rely on output without evaluating its accuracy — and 56 percent have already made AI-caused mistakes at work. Microsoft and Carnegie Mellon researchers found the cruelest twist: the more confident workers are in the AI, the less critical thinking they do. Fluency disarms scrutiny.

The distinction that protects you: AI supplies analysis — options generated, ranked, argued with perfect fluency from any side. It cannot supply discernment — separating true from plausible, signal from noise, wise from merely fast. Discernment requires a stake in the outcome, and a machine has none.

So design for the catch, not the click. Two tests for any system that touches consequential decisions:

• The attention test. Does it reduce what a person must look at, or multiply it? A system that floods attention is spending judgment, not building it.
• The doubt test. Does it make questioning its output easier, or harder? Show the evidence for, the confidence level, and what cuts against — or expect bulk approval.