Case Study 1.2 — The Website That Vanished: A Single Line That Turned Off a Whole Site

A complementary angle. Case Study 1.1 looked at the pipeline working as designed, described in Google's own words. This one looks at the pipeline breaking — at a single point, silently, and catastrophically — to make the chapter's central claim concrete: a page (or a whole site) can be excellent and still be invisible, because a failure at one early stage stops everything downstream.

This is a labeled composite. It is not one named company; it is a pattern that has played out, publicly and privately, countless times, and that Google's own Search Relations team has warned about repeatedly. Every specific number below is illustrative. What is entirely real is the mechanism — and how ordinary the mistake is.

Background: the relaunch everyone was proud of

A mid-size company — call it a regional retailer with a few hundred pages and a healthy stream of organic traffic — decides its website looks dated. It commissions a beautiful redesign. For months, the new site is built on a staging server, a private copy where the team can work without the public seeing half-finished pages. To keep that private staging site out of Google, the developers do exactly what best practice tells them to do: they add a site-wide instruction telling search engines not to index it. In practice that is usually one of two things — a <meta name="robots" content="noindex"> tag applied to every page, or a Disallow: / line in the staging site's robots.txt. Sensible. Correct. Responsible.

Launch day arrives. The team copies the new site from staging to the live server, checks that the pages load and look right, celebrates, and moves on.

They copied the noindex with it.

The issue: four green lights and one red

Walk the new live site through the pipeline from Chapter 1 and you would find almost everything healthy.

  • Discovery: fine. The pages are linked in the navigation; the sitemap lists them.
  • Crawling: fine. Googlebot fetches them; they return 200 OK.
  • Rendering: fine. The content is right there in the HTML.
  • Indexing: broken. Every page carries noindex. Googlebot crawls each one, reads the instruction that says do not keep this page, and obediently drops it from the index — including pages that had ranked for years.
  • Ranking: moot. A page that isn't in the index cannot rank for anything.

Here is the cruel part, the part that makes this failure mode so dangerous: from the outside, nothing looks wrong. The site is up. It's fast. It's gorgeous. Visitors who arrive from a bookmark or a direct link see a perfect experience. The only symptom is that organic traffic — the visitors who used to arrive from Google — begins to evaporate, page by page, as Google re-crawls the site over the following days and honors the instruction to forget each one.

text FIGURE 1.4 — "The vanishing, week by week" [constructed teaching example] WEEK 0 (launch) Organic traffic normal. All key pages indexed. Everyone happy. WEEK 1 Traffic down ~20%. A few high-crawl-rate pages already dropped. "Probably noise." WEEK 2 Traffic down ~55%. Panic begins. The redesign gets blamed ("the new design killed us"). WEEK 3 Traffic down ~80%. Someone finally checks a page's source and finds the noindex tag. (All figures illustrative — the SHAPE, a compounding decline as Google re-crawls, is the real lesson.)

Notice the misdiagnosis in week two. The natural human story is "we changed the design and traffic fell, so the design must be the problem." That story is wrong, and acting on it — reverting the design, rewriting content, second-guessing the whole project — would waste weeks and fix nothing, because the design was never the problem. One line of leftover code was.

What it shows

This case is the whole chapter in one disaster:

  1. The pipeline has stages, and an early break stops everything after it. No amount of ranking-stage work — content, links, keywords — can help a page that indexing has thrown out. You must fix the broken stage, not a later one.
  2. "Indexed" is not automatic. The site owner never consciously decided "let's not be in Google." The decision was made implicitly, by a machine reading an instruction. Indexing is a gate, and a gate can be accidentally locked.
  3. The symptom points away from the cause. Falling traffic after a redesign looks like a content or design problem. The pipeline model is what lets you resist the obvious-but-wrong story and ask, stage by stage, "where exactly is the break?" The answer — check whether the pages are even indexed — is a two-minute diagnosis that saves two months of thrashing.
  4. Diagnosis is cheap; the absence of diagnosis is expensive. The fix here takes minutes: remove the noindex, resubmit the pages, wait for Google to re-crawl. The damage came entirely from not looking at the right stage for three weeks.

Outcome

In the fortunate version of this story, someone runs a site: search, sees far too few pages, inspects a URL in Search Console, finds "Excluded by 'noindex' tag," and the cause is obvious. The tag is removed, the sitemap is resubmitted, and over the next couple of weeks Google re-crawls and re-indexes the pages. Rankings usually return to roughly where they were, because the pages themselves never lost their quality or their links — they were just temporarily evicted. The recovery is real but not instant: you are at the mercy of how quickly Google re-crawls, which is exactly the crawl-scheduling reality from §1.2.

In the unfortunate version, the mistake goes unnoticed for months, the business concludes the redesign "failed," and it makes a cascade of unnecessary changes chasing a phantom — sometimes never realizing the real cause at all.

The lesson

The transferable principle is the one this whole book is built on: before you treat a symptom, diagnose the stage. A launch or migration checklist should include, in bold, "remove staging noindex / robots.txt blocks; confirm key pages are indexed" — and we will build exactly that checklist in Chapter 21. But the deeper lesson is a habit of mind. When traffic moves, the professional does not reach first for the most visible or most recent change ("the design"). They walk the pipeline: discovered? crawled? rendered? indexed? ranked? The break is somewhere specific, and finding where is most of the work.

Connect it to Rivertown. Rivertown's site has never been deliberately blocked, but this is precisely the kind of failure we will check for in its Chapter 14 crawl-and-index audit — because the most dangerous problems in SEO are the invisible ones, the single lines that quietly turn a stage off.

Discussion questions

  1. Why does the traffic in Figure 1.4 decline gradually over three weeks rather than dropping to zero the instant the site launches? Tie your answer to how Google crawls (§1.2).
  2. In week two, the team blamed the redesign. Explain, using the pipeline, why that hypothesis is both understandable and wrong — and what single check would have falsified it in minutes.
  3. This case failed at the indexing stage. Construct a parallel story in which an equally invisible, equally cheap-to-fix mistake breaks the discovery stage instead.
  4. A stakeholder asks, "If this is so easy to check, why does it happen to so many companies, including sophisticated ones?" Give a genuine answer that goes beyond "they were careless."
  5. Is the disappearance in this case a penalty? Explain the difference between a page Google was told not to index and a page Google chose to demote — and why the distinction changes how you'd respond. (Chapter 6 and Chapter 26 develop this.)