Case Study 3.1 — BERT (2019): When Google Learned to Read the Intent, Not the Words

A real, public case. Facts below are drawn from Google's own announcement; figures attributed to Google are Google's stated figures, not independent measurements. See further-reading.md for sources.

Background

On October 25, 2019, Google announced that it had begun using a language-understanding system called BERT (Bidirectional Encoder Representations from Transformers) in Search. Google's public framing was unusually strong: it called the change one of the biggest leaps forward in the history of Search, and said BERT would affect roughly 1 in 10 of all English-language queries in the United States at launch (Google's stated figure). BERT is introduced as a ranking system in Chapter 2; here we look at it through the lens of intent, because that is what it was built to read.

The problem BERT was designed to solve is exactly the problem this chapter is about. For most of search history, matching a query to a page leaned heavily on the words — which pages contained the searched terms. But human language does not work by keywords. Small words — "to," "for," "no," "stand" — change the entire meaning of a sentence, and therefore the intent behind a search. A system that treats a query as a bag of keywords will miss the meaning; a system that reads the query as language can grasp the intent.

The search / SEO issue

Google published concrete before-and-after examples. Consider the query:

"2019 brazil traveler to usa need a visa"

Before BERT, Google's systems could under-weight the word "to," and might return results about a U.S. citizen traveling to Brazil — the reverse of what the searcher meant. The searcher's intent is about a Brazilian traveling to the USA. BERT, reading the sentence as a whole, understood that "to" is load-bearing: the direction of travel is the entire point. Google's example results improved accordingly.

Google gave others in the same announcement, including:

  • "do estheticians stand a lot at work" — understanding "stand" as being on one's feet, not as a "stand-alone" role.
  • "can you get medicine for someone pharmacy" — grasping that the searcher wants to know about picking up a prescription for another person.

Each example is the same lesson wearing different clothes: the keywords were present in the old results too; what was missing was an understanding of the intent the words encoded. BERT closed part of that gap.

What it shows

Three things, all central to this chapter.

  1. Google is investing enormous resources in understanding intent, not matching strings. BERT is a statement of direction: the future of Search is language and meaning. A strategy built on satisfying intent is aligned with where Google is going; a strategy built on keyword repetition is aligned with the past Google is deliberately leaving behind.
  2. You cannot "optimize for BERT." Google said as much directly — there is nothing to tune, no setting to flip. The only response that makes sense is to write naturally, for humans, about what people actually want to know. That is not a dodge; it is the whole point. When the machine reads meaning, the way to win is to mean well and clearly.
  3. Better intent-reading helps good, plainly-written pages and hurts keyword-gamed ones. A page that answered the real question in natural language stood to gain from a system that finally understood the question. A page engineered around exact-match phrases had less to offer a system that no longer needed the exact phrase.

Outcome

BERT rolled out in English and then expanded to many more languages over the following months, and Google folded ever-more-capable language models (it later described using its MUM system, and language understanding is now woven throughout ranking). There was no penalty, no manual action, no "BERT recovery" process — because BERT is not a spam filter. It is a comprehension upgrade. Sites did not need to do anything; the SERPs simply got a little better at matching pages to what searchers meant. That is precisely why there was no "BERT optimization industry" worth the name: there was nothing to sell.

The lesson

BERT is the clearest possible confirmation of this chapter's thesis from Google's own engineering priorities: Search is a machine for understanding intent, and it is getting better at it every year. The durable response is not to chase the algorithm but to do the thing the algorithm is trying to reward — understand what your searcher wants and answer it clearly, in real language, better than anyone else. Keyword tricks lose value every time Google's comprehension improves. Genuine intent-matching gains value.

Discussion questions

  1. Google said you "can't optimize for BERT." In your own words, what should a publisher do in response to a comprehension upgrade like this — and why is "nothing special, just write clearly for the reader" a serious answer rather than a cop-out?
  2. Pick one of Google's example queries ("2019 brazil traveler to usa need a visa," "do estheticians stand a lot at work," "can you get medicine for someone pharmacy"). Rewrite it as a keyword-first SEO of 2012 might have interpreted it, then as BERT interprets it. What intent was being missed?
  3. If Google keeps getting better at reading intent, what happens over time to pages that rank mainly because they repeat exact-match keywords? What happens to pages that genuinely answer the question?
  4. How does BERT connect to the "apple problem" from §3.2 and to the claim in §3.3 that "the SERP is the answer key"? (Hint: better comprehension makes the SERP a more trustworthy read of intent, not less.)