Case Study 2: Facing the Camera — Surveillance, Identity, and Consent

Facial Recognition Deployment in Public Spaces

The Scene

Imagine walking through your city's downtown on a Saturday afternoon. You pass a department store, a transit station, a park, and a government building. You're not doing anything unusual. You're not thinking about surveillance.

But here is what may be happening, depending on where you live:

• The department store's security cameras are running facial recognition to match your face against a database of known shoplifters.
• The transit station is using facial recognition as part of a fare enforcement pilot program.
• The park has cameras connected to a law enforcement database, scanning for individuals with outstanding warrants.
• The government building is using biometric access control.

At no point were you asked for consent. At no point were you informed. And depending on the accuracy of these systems — and the demographics of your face — the consequences could range from nothing at all to being incorrectly flagged, detained, and questioned.

This is not science fiction. This is the current state of facial recognition technology (FRT) deployment in many cities around the world.

A Tale of Two Deployments

To understand the full spectrum of how facial recognition plays out, consider two real-world scenarios.

Deployment 1: Finding Missing Children

The National Center for Missing & Exploited Children (NCMEC) receives reports of missing children and works with law enforcement to locate them. In some cases, facial recognition technology has been a genuine lifesaver. The technology can scan images and video for matches against databases of missing children, including age-progressed photos of children who have been missing for years. India's government has reported that facial recognition systems helped trace over 3,000 missing children in just four days during a 2018 trial in New Delhi, though independent verification of this figure has been limited.

In these cases, the technology serves a purpose that is nearly universally supported: protecting children. The subjects being searched for cannot consent (they are children, and they are in danger), and the societal interest in finding them is overwhelming.

Deployment 2: Surveilling Protesters

During the 2020 protests following the killing of George Floyd, law enforcement agencies in several U.S. cities used facial recognition technology to identify protesters. The Drug Enforcement Administration was granted temporary authority to conduct "covert surveillance" of protesters. U.S. Customs and Border Protection deployed surveillance aircraft over cities. And local police departments used social media photos run through facial recognition databases to identify individuals who attended protests.

In these cases, the technology was being used against people exercising their constitutional right to assemble and protest. The subjects were overwhelmingly peaceful. The surveillance was conducted without warrants and without the knowledge of those being scanned. And the technology's documented racial accuracy disparities meant that the system was least reliable for the very communities most represented at these protests.

Same technology. Profoundly different implications.

The Accuracy Divide, Revisited

The chapter discussed the Gender Shades study's findings about accuracy disparities. Let's put those findings in a concrete context.

Robert Williams, a Black man living in a suburb of Detroit, was arrested in January 2020 on his front lawn, in front of his wife and two young daughters. The charge: stealing watches from a Shinola store. The evidence: a facial recognition system had matched a grainy surveillance image to Williams's driver's license photo.

The match was wrong. Williams did not commit the crime. He didn't resemble the actual suspect beyond the fact that both men were Black. After being held for 30 hours, he was released. The case was eventually dropped.

Williams was not alone. Nijeer Parks was wrongfully arrested in New Jersey in 2019 based on a facial recognition match. He spent 10 days in jail before the case was dismissed. Michael Oliver in Detroit was wrongfully accused of a felony based on a facial recognition match. In every documented case of wrongful arrest based on facial recognition in the United States as of 2024, the person wrongfully accused has been Black.

This pattern is not a coincidence. It is a direct consequence of the training data disparities documented in the Gender Shades study and subsequent research. Systems trained predominantly on lighter-skinned faces are less accurate on darker-skinned faces. When those systems are deployed in law enforcement — where a false positive means wrongful arrest — the burden falls disproportionately on communities of color.

The Consent Problem

Beyond accuracy, a fundamental question underlies every facial recognition deployment: consent.

Most facial recognition in public spaces operates without the knowledge or consent of the people being scanned. This distinguishes it from other forms of identification:

• Fingerprinting requires physical contact — you know you're being fingerprinted.
• ID checks are visible — someone asks for your license, you hand it over.
• DNA analysis requires a sample — you know it's been taken.

Facial recognition can happen at a distance, without any interaction, in real time. You cannot "refuse" to have your face scanned when cameras operate in public spaces. You cannot choose to leave your face at home the way you might leave your phone behind. Short of wearing a mask or distorting your appearance (and some jurisdictions have anti-mask laws), your biometric data is collected without any affirmative act on your part.

Some argue this is no different from being seen in public — that there is no reasonable expectation of privacy for your face in a public place. But there is a meaningful difference between being seen by passersby (who will forget you in moments) and being identified, cataloged, and stored in a database that can be searched indefinitely. The first is observation. The second is surveillance.

The Regulatory Patchwork

As of the mid-2020s, the regulation of facial recognition is a patchwork of inconsistent rules:

Bans and restrictions: - San Francisco, Boston, Minneapolis, New Orleans, Portland (Oregon), and several other U.S. cities have banned government use of facial recognition. - The European Union's AI Act classifies real-time biometric identification in public spaces as "high risk" and imposes significant restrictions, though with exceptions for law enforcement under certain conditions. - Illinois's Biometric Information Privacy Act (BIPA) requires informed consent before collecting biometric data, including face geometry. It remains one of the strongest biometric privacy laws in the world and has resulted in major settlements (Meta paid $650 million in 2021 for collecting facial recognition data from Illinois users without consent).

Expansions: - China has deployed extensive facial recognition infrastructure, including systems integrated with social credit programs and ethnic minority surveillance systems in Xinjiang. - India has rolled out one of the world's largest facial recognition systems for law enforcement and government services. - The United Kingdom uses live facial recognition at events and in public spaces, though the practice has faced legal challenges. - Clearview AI, a controversial company, scraped billions of photos from social media and the web to build a facial recognition database used by thousands of law enforcement agencies. It has faced lawsuits and regulatory action in multiple countries.

The gap: Many jurisdictions have no specific regulations governing facial recognition at all. In these places, deployment decisions are made by police chiefs, store managers, transit authorities, and school administrators — often without public debate, oversight, or impact assessment.

The Stakeholder Map

Understanding facial recognition requires understanding who has a stake in its deployment:

Notice that the benefits of facial recognition tend to flow to institutions (law enforcement, corporations, governments), while many of the risks are borne by individuals — particularly individuals from marginalized communities.

Framework for Evaluation

When evaluating any proposed facial recognition deployment, consider these questions:

1. Purpose: What specific problem is this deployment designed to solve? Is facial recognition the least invasive technology that could solve it?

2. Accuracy: Has the system been independently tested for accuracy across demographic groups? Are the error rates acceptable given the consequences of a false match?

3. Consent: Are the people being scanned aware? Did they agree? If consent isn't feasible (as in law enforcement), what oversight mechanisms ensure the technology isn't abused?

4. Data governance: Who stores the biometric data? How long is it kept? Who has access? What happens if it's breached?

5. Accountability: If the system makes an error — a wrongful arrest, a false accusation — who is responsible? What recourse does the affected person have?

6. Proportionality: Are the benefits of deployment proportional to the risks? Finding missing children may justify different measures than optimizing retail layouts.

7. Alternatives: Would a non-biometric approach achieve the same goal with fewer civil liberties concerns?

Where CityScope Predict Enters

Consider our anchor example CityScope Predict — the predictive policing system introduced in Chapter 1. Imagine the city council is considering adding facial recognition cameras in the same neighborhoods where CityScope Predict already directs police patrols. The predictive algorithm sends officers to certain neighborhoods more frequently. Now facial recognition would also be scanning faces in those neighborhoods more frequently.

The compounding effect is significant: residents of neighborhoods flagged as "high risk" by the predictive algorithm would experience more police presence and more biometric surveillance than residents of other neighborhoods. If those flagged neighborhoods are disproportionately communities of color — and the research literature on predictive policing consistently shows this pattern — then the combination of predictive policing and facial recognition creates a surveillance feedback loop that concentrates scrutiny on the people least likely to benefit from it.

This is not a hypothetical concern. It is the logical extension of two technologies, each with documented racial disparities, being deployed in the same context.

Discussion Questions

1. Is there a meaningful moral distinction between using facial recognition to find missing children and using it to identify peaceful protesters? If so, what principles would you use to draw the line?

2. Robert Williams was arrested on his front lawn in front of his children. What harm did the wrongful arrest cause beyond the legal inconvenience? Consider the psychological, social, and family dimensions.

3. Illinois's BIPA requires informed consent before collecting biometric data. Critics argue this makes beneficial uses of facial recognition too difficult. Defenders argue that consent is a fundamental right. Where do you stand, and why?

4. Some people argue that if facial recognition technology were equally accurate across all demographics, the civil liberties concerns would be largely resolved. Do you agree? What concerns would remain even with perfect accuracy?

5. Design a facial recognition policy for your campus. Under what circumstances, if any, would facial recognition be permitted? What safeguards would you require? Who would have oversight? Write your policy in no more than five bullet points and then stress-test it against the scenarios in this case study.

6. Return to the project checkpoint for this chapter. If your chosen AI system involves any form of biometric identification or visual surveillance, how do the consent and accuracy issues discussed here apply?