Context
Zillow is known for its home valuation service in the United States. In 2018 the company launched Zillow Offers: it bought homes itself at prices calculated by an algorithm, renovated them quickly and resold them. The model made money on the accuracy of the price forecast and the speed of turnover.
Symptom
In 2021 the housing market was changing fast, while the algorithm kept offering high prices. In the third quarter of 2021 the company bought 9,680 homes and sold 3,032. Some homes had to be listed below their purchase price.
Cause
A forecast error of a few percent, acceptable for a reference estimate, turns into a direct loss when you buy thousands of homes. The model did not have enough limits: it kept buying when its forecasts had already diverged from the market.
The fork
Keep scaling purchases in the hope that the model would learn, or admit that the risk in forecasting prices was greater than acceptable.
Decision
On November 2, 2021 the board decided to shut down Zillow Offers. Management explained that the unpredictability in forecasting home prices was far greater than the company had anticipated.
Execution
The company began selling homes from its inventory, including through deals with institutional investors, and announced that it would cut about 25 percent of its workforce, some 2,000 people.
Outcome
For the third quarter of 2021 Zillow wrote down $304.4 million of home inventory and expected another $240–265 million in costs in the fourth quarter. The day after the announcement the company's shares fell by about a quarter.
What it means for you
An AI model is always wrong sometimes; the question is what an error costs. If a model makes decisions with money, it needs limits and a person who can stop it.
- Which decisions in your company are already made by an algorithm, and who checks its errors?
- What does one error of the model cost, and how many such errors in a row can you absorb?
- By what signs will you stop an automated decision?
We would start by calculating the cost of an error for every decision you want to hand to AI, and determining where a person is needed to check it.
Key dates
| Year | Event |
|---|---|
| 2018 | Zillow Offers launches |
| 2021, Q3 | 9,680 homes bought, 3,032 sold; $304.4 million write-down |
| 2021, November 2 | Decision to shut down Zillow Offers and cut about 25 percent of staff |
Sources
- Zillow Group. Form 8-K, November 2, 2021 (third-quarter 2021 shareholder letter): Source
- Zillow Group. Form 10-Q for the third quarter of 2021: Source
- CNBC. *Zillow says it's closing home-buying business, reports Q3 results*, November 2, 2021: Source
- Stanford Graduate School of Business. *Flip Flop: Why Zillow's Algorithmic Home Buying Venture Imploded*: Source
Next case analysis: Netflix. Protect the profitable model or invest in the next one →
