Appendix A: Research Methods in Surveillance Studies

How to Study Watching and Being Watched Empirically

Introduction: Why Method Matters in Surveillance Research

Surveillance studies is an inherently interdisciplinary field — part sociology, part political science, part legal studies, part computer science, part history, part philosophy. This interdisciplinarity is its strength: surveillance is a social phenomenon with technical dimensions and historical roots and legal expressions and philosophical implications, and no single discipline can capture all of it. But interdisciplinarity creates methodological challenges: what counts as good evidence in surveillance research? How do we know what we claim to know about who is watched, why, and with what effects?

This appendix introduces the methodological landscape of surveillance research. It is not a statistics textbook or a research design manual; it is a guide to understanding how researchers approach the study of surveillance and how you, as a student and eventually as a researcher, can evaluate and produce surveillance scholarship. The goal is methodological literacy: the ability to understand how knowledge is produced and to assess its quality, limitations, and assumptions.

One preliminary note: surveillance studies is politically engaged scholarship. Most surveillance researchers are not neutral observers — they bring values about privacy, power, civil liberties, and justice to their work. This does not make their work bad scholarship; it makes it motivated scholarship, and motivated scholarship is not inherently flawed. But it does mean that methodological transparency — being explicit about one's assumptions, methods, and the limits of one's evidence — is especially important. A researcher who cares about surveillance accountability has all the more reason to be rigorous about method, so that their conclusions cannot be dismissed as the artifacts of motivated reasoning.

Section 1: Epistemological Approaches

Epistemology is the study of how we know what we know — the philosophical foundation of research method. Three broad epistemological traditions shape how surveillance researchers approach their work.

Positivist Approaches

Positivist research assumes that the social world, like the natural world, follows discoverable regularities that can be measured and tested. Positivist surveillance research asks questions like: Does CCTV reduce crime? Do people who know they are tracked change their behavior? What percentage of websites use third-party tracking scripts? How much less likely are individuals to search for certain terms after surveillance disclosures?

Positivist approaches value: precise measurement, replicability, generalizability, and causal inference. They tend to produce quantitative data — numbers, statistics, rates — that can be compared, tested for significance, and used to make policy arguments with precision.

Limitations: Positivist approaches can struggle with questions that are inherently evaluative (is this surveillance legitimate?), with phenomena that resist quantification (the subjective experience of being watched), and with the power dynamics embedded in measurement itself (who decides what gets measured and how?). Surveillance capitalism is particularly resistant to positivist measurement because the most important data — what behavioral data is collected, how it is processed, how much it affects behavior — is proprietary and inaccessible to outside researchers.

Interpretivist Approaches

Interpretivist (or constructivist) research assumes that social meanings are not discovered but made — that people actively interpret their situations, and that the researcher's task is to understand those interpretations. Interpretivist surveillance research asks: How do people understand and make sense of being watched? How do different communities experience surveillance differently? How is surveillance justified and contested in policy discourse?

Interpretivist approaches value: depth over breadth, context over generalizability, and the perspectives of research subjects. They tend to produce qualitative data — interview transcripts, ethnographic field notes, document analyses — and rich, contextual descriptions of particular cases.

Limitations: Interpretivist research can be difficult to generalize; findings from one community or context may not transfer. Interpretivism also faces challenges of researcher influence — the researcher's presence and perspective inevitably shapes the data, which requires careful reflexivity.

Critical Approaches

Critical research combines empirical investigation with normative evaluation and a commitment to understanding power. Critical surveillance researchers ask: Who benefits from this surveillance arrangement? Who bears its costs? How does surveillance perpetuate or challenge racial, class, gender, and other inequalities? What would a just surveillance regime look like?

Critical approaches draw on traditions including Marxism, feminism, critical race theory, postcolonialism, and queer theory. They tend to combine empirical methods with explicit normative argument, and to measure research success not just by methodological rigor but by its contribution to social justice.

Limitations: Critical research can be accused of confirmation bias — finding what it was looking for. Critical researchers must be especially attentive to evidence that complicates or contradicts their normative commitments, and must distinguish their empirical claims (which are subject to evidence) from their normative claims (which are subject to argument).

In practice, surveillance research frequently combines elements of all three approaches. A study of facial recognition bias might use quantitative testing methods (positivist), interpret what the findings mean for affected communities (interpretivist), and argue for structural reforms based on a theory of racial justice (critical). Methodological pluralism is appropriate given the complexity of the subject.

Section 2: Quantitative Methods in Surveillance Research

Surveys

Surveys measure attitudes, beliefs, behaviors, and self-reported experiences across large populations. In surveillance research, surveys have been used to measure: public awareness of data collection practices; privacy attitudes and their relationship to demographic factors; behavioral changes following surveillance disclosures; trust in institutions and platforms; and self-reported responses to being watched.

Well-known examples include: The Pew Research Center's ongoing series of surveys on privacy and surveillance (documenting, among other findings, the sharp decline in public confidence in digital platforms following the Snowden revelations); the ongoing Annual Privacy Study conducted by various academic centers; and behavioral surveys examining the "privacy paradox" — the gap between expressed privacy concern and actual behavior.

Survey design challenges in surveillance research: - Social desirability bias: Respondents may underreport surveillance-related behaviors they find embarrassing (checking partners' phones) or overreport socially desirable behaviors (using privacy tools). - Knowledge gaps: Surveillance operates invisibly for most people; survey responses about data collection practices may reflect guesses rather than informed assessments. - Question framing effects: Small changes in how surveillance questions are worded produce large changes in responses. Questions that frame surveillance as "security" versus "privacy" produce dramatically different response patterns.

Content Analysis

Content analysis systematically examines texts — documents, websites, policy statements, news coverage, terms-of-service agreements — to identify patterns, categories, and themes. In surveillance research, content analysis has been used to: analyze the readability of privacy policies; map the surveillance capabilities described in corporate annual reports; trace how surveillance is covered in news media across political contexts; and systematically examine government surveillance authorization documents.

A content analysis can be quantitative (counting the frequency of particular terms or categories) or qualitative (analyzing the meaning and framing of content). Intercoder reliability — the degree to which different researchers applying the same coding scheme to the same texts reach the same conclusions — is the primary validity check for quantitative content analysis.

Network Analysis

Network analysis maps and measures relationships between entities — people, organizations, platforms, devices — as a network of nodes and edges. In surveillance research, network analysis has been applied to: mapping social graph data to understand what "metadata" reveals; analyzing the corporate networks of data broker relationships; tracing how surveillance technologies diffuse through law enforcement networks; and studying information flows in intelligence communities.

Key metrics in network analysis include: degree centrality (how many connections does a node have?), betweenness centrality (how often does a node appear on the shortest path between other nodes?), and community detection (what clusters or communities exist within the network?). These metrics have direct surveillance relevance: betweenness centrality identifies "connectors" whose communications are most informative about a network; community detection identifies clusters that intelligence analysts might designate as "affiliates."

Web Measurement Studies

Web measurement — automated testing of what data websites collect and how — has become one of the most important quantitative methods in surveillance research. The Princeton WebTAP study, the OpenWPM framework, and subsequent measurement research have documented the prevalence of trackers, the persistence of cookies, the use of browser fingerprinting, and the extent of real-time bidding data flows across the web.

Web measurement requires: web crawling infrastructure (tools that visit large numbers of websites programmatically); traffic analysis tools (tools that capture and analyze all network traffic generated by the crawl); and coding frameworks that identify known tracking technologies. The primary validity challenge is that web tracking varies by user profile, location, and time; measurements represent a sample of the web at a particular moment, not a fixed reality.

Section 3: Qualitative Methods

Interviews

Semi-structured interviews allow researchers to explore surveillance experiences in depth, following the respondent's own framing while covering key topics. Surveillance interview research has documented: how workers understand and respond to algorithmic management; how domestic violence survivors experience and escape stalkerware; how privacy advocates understand the limits of legal protection; how intelligence agency employees understand their mandate; and how communities targeted by predictive policing describe their experiences.

Interview design for surveillance research: - Sampling: Who you talk to shapes what you learn. Studies of corporate surveillance workers, surveillance system administrators, and activists produce different knowledge than studies of surveilled populations. Both are necessary. - Informed consent: Interview participants must understand what the research is for and how their data will be used. In sensitive surveillance contexts (domestic violence, undocumented status, political activism), consent must include genuine explanation of how researchers will protect confidentiality. - Reflexivity: Interviewers bring their own assumptions about surveillance; careful reflexivity — explicit attention to how the researcher's identity and perspective shapes data collection and interpretation — is essential.

Ethnography

Ethnographic research involves sustained immersion in a research setting — observation, participation, and engagement over time. In surveillance research, ethnography has been used to study: workplace monitoring in specific organizations; the everyday practices of privacy advocates and security researchers; community responses to police surveillance programs; and how surveillance technologies are built and deployed inside technology companies.

Ethnography produces rich, contextual knowledge of how surveillance operates in practice, including the gap between how surveillance is described in policy documents and how it actually works. An ethnographer studying an algorithmic management system would observe not just the system's outputs but how managers use and override it, how workers strategically respond to it, and how its meaning is contested and negotiated in daily life.

Document Analysis

Document analysis applies systematic interpretive methods to written sources — policy documents, internal communications, legal filings, technical specifications, regulatory decisions. In surveillance research, document analysis has been central to understanding programs that are deliberately obscured: FOIA documents from surveillance programs, leaked NSA technical specifications, Facebook's internal research documents disclosed in litigation, and Ring's police partnership agreements all became the basis for significant surveillance scholarship.

Document analysis requires attention to: provenance (where did this document come from and how does that affect its reliability?); intended audience (documents written for internal audiences say different things than documents written for public consumption); and context (what organizational pressures and historical moments produced this document?).

Section 4: Mixed Methods

Many of the most important surveillance studies combine quantitative and qualitative methods. This is appropriate given that surveillance is simultaneously a technical phenomenon (measurable through web crawling and network analysis) and a social phenomenon (experienced, interpreted, and contested by people).

A well-designed mixed-methods study might: use web measurement to document the prevalence of third-party trackers on health information websites (quantitative) and then use interviews with users to understand how people discover and respond to this tracking (qualitative). The quantitative component establishes the scope of the phenomenon; the qualitative component explains its social meaning and consequences.

The primary design challenge in mixed methods is integration: how do the quantitative and qualitative components speak to each other? A study that simply reports quantitative findings and qualitative findings separately has not truly integrated its methods. Integration requires designing each component in light of the other and synthesizing findings that address the same research question from different angles.

Section 5: Ethical Challenges Specific to Surveillance Research

Surveillance research raises distinctive ethical challenges that go beyond standard human subjects protocols.

IRB and Human Subjects Review

Institutional Review Boards (IRBs) at US universities review research involving human subjects for compliance with federal ethical standards. IRB review is required for surveillance research involving: interviews and surveys with human participants; ethnographic observation in organizational settings; and analysis of data about identifiable individuals.

IRBs may be less experienced with digital methods. Research that analyzes social media data, web tracking, or public records databases may technically fall outside standard IRB coverage while still raising significant ethical issues. Researchers should engage their IRBs proactively and not assume that "public" data is automatically exempt from ethical consideration.

Informed Consent in Surveillance Research

Obtaining genuinely informed consent is complicated in surveillance research. Participants may not understand surveillance concepts well enough to consent meaningfully. In sensitive contexts (domestic violence survivors, undocumented immigrants, political activists), the risks of research participation include potential surveillance by the very actors being studied. Researchers studying vulnerable populations have developed protocols including: using secure communication for participant recruitment and interviews; anonymizing participant data more aggressively than standard practice; and designing studies so that participation itself is not identifiable by adversaries.

Researcher Positionality

Surveillance researchers study powerful actors (governments, corporations) and vulnerable ones (surveilled communities, activists, workers). Who the researcher is — their institutional affiliation, demographic identity, prior publications, and professional relationships — affects both access (who will talk to them) and the knowledge produced. A researcher affiliated with a major technology company brings different access and different blind spots than an independent academic.

Positionality statement practice: Many surveillance scholars include explicit statements in their research about their own position — what they believe about surveillance, what institutional affiliations they carry, and how they expect their position affects their work. This is not a confession of bias but an act of transparency that allows readers to evaluate the research appropriately.

The Ethics of Studying Corporate Surveillance

Studying how surveillance capitalism operates often requires accessing data and systems without explicit cooperation from the entities being studied. Web measurement research — crawling websites to document tracker deployment — is conducted without the consent of the websites or the companies whose trackers are documented. This raises ethical questions: Is reverse engineering a tracking technology, to understand how it works, ethical without the company's consent? Is publishing findings that reveal proprietary tracking methods a form of disclosure that the researcher is entitled to make?

The surveillance studies community has generally answered: yes, public-facing surveillance technologies used on millions of people without their knowledge are subject to independent scrutiny, and documentation of such technologies serves the public interest. But this requires careful judgment about specific cases, and researchers should be familiar with the Computer Fraud and Abuse Act and similar laws that create legal risks for certain forms of system testing.

Section 6: Accessing Primary Source Documents

FOIA Requests

The Freedom of Information Act (FOIA) allows any person to request federal government records. FOIA has been central to surveillance research: COINTELPRO documents were obtained through FOIA; NSA program documents have been partially disclosed through FOIA; police department facial recognition contracts have been obtained through state-level public records requests.

Practical FOIA guidance: - Specificity: The more specific your request, the faster you will receive a response. Requests for "all documents about surveillance" will be denied or returned unmanageably large. Requests for "contracts between [specific police department] and [specific vendor] for facial recognition services, 2015-2023" are more tractable. - State equivalents: Most states have their own public records laws (often called "sunshine laws" or "open records laws") that apply to state and local agencies. For police surveillance research, state records requests are often more productive than federal FOIA. - FOIA litigation: Agencies frequently deny or delay requests. FOIA litigation — suing to compel disclosure — is an important tool. Organizations including the ACLU, EFF, and Reporters Committee for Freedom of the Press maintain resources for FOIA requesters. - Appealing denials: FOIA denials must cite a specific exemption; you can appeal denials within the agency and then to federal court.

Document Archives

Several organizations maintain searchable archives of surveillance-related documents: - The Internet Archive maintains FOIA document collections. - The National Security Archive at George Washington University maintains declassified government documents. - MuckRock is a collaborative platform for filing and sharing FOIA requests. - DocumentCloud is used by journalists to publish and annotate primary source documents. - The Snowden Archive, maintained at The Intercept and other publications, contains searchable NSA documents.

Section 7: Online Research Methods

OSINT (Open Source Intelligence)

Open-source intelligence research uses publicly available information — social media, property records, corporate filings, satellite imagery, court documents — to investigate surveillance deployments and actors. Surveillance researchers have used OSINT to: map the locations of CCTV infrastructure; identify vendors of surveillance technologies sold to authoritarian governments; document the corporate networks of data brokers; and trace financial relationships between surveillance companies and government contractors.

OSINT requires: systematic search methodology; documentation of sources (including timestamps and URLs); verification of information through multiple sources; and legal literacy about what searching and documenting publicly available information may or may not entail legally in different jurisdictions.

Social Media Analysis

Social media platforms generate enormous amounts of data relevant to surveillance research. Public posts, account networks, hashtag patterns, and temporal analysis of social media data can illuminate: how surveillance news travels; how communities organize responses to surveillance; how platforms enforce policies in discriminatory patterns; and how surveillance-facilitating narratives are constructed and spread.

Ethical and legal considerations: Platforms' terms of service often restrict automated data collection (scraping), and this creates tension with research interests. The academic exemption in GDPR allows some processing of personal data for legitimate research, but the scope of this exemption is debated. IRBs and researchers should assess whether public social media data collection requires consent, and whether the research can be conducted with aggregated rather than individual-level data.

Section 8: Evaluating Surveillance Studies Research

When reading surveillance studies scholarship — or producing your own — use the following evaluative framework.

Validity

Internal validity asks: does the study actually measure what it claims to measure? A study claiming to measure "the effect of surveillance on behavior" must ensure that observed behavioral changes are actually caused by the surveillance being studied, not by other factors. External validity asks: do the findings generalize beyond the specific study context? A finding about surveillance responses in a US university population may or may not apply to workers in gig economy platforms.

Reliability

Reliability asks: would the same methods applied by different researchers in the same context produce the same results? Qualitative research is inherently less reliable than quantitative research in this technical sense — different researchers interviewing the same people would produce different transcripts and interpretations — but can achieve inter-rater reliability on specific coding decisions.

Positionality and Reflexivity

For critical and interpretivist research: has the researcher been transparent about their position, their assumptions, and how these affect their work? Has the researcher engaged seriously with evidence that complicates their argument? Has the researcher given appropriate voice to the perspectives of research subjects, especially those who are surveilled?

Practical Significance

Statistical significance is not the same as practical significance. A statistically significant reduction in crime following CCTV installation may be so small in absolute terms as to have no practical policy relevance. Surveillance research should evaluate both statistical and practical significance.

Section 9: Sample Research Design

Studying the Racial Demographics of CCTV Deployment in a Mid-Size City

Research Question: Is CCTV camera deployment in [City] distributed proportional to crime rates, or does deployment correlate with the racial composition of neighborhoods, independent of crime rates?

Significance: If surveillance infrastructure is concentrated in neighborhoods with high proportions of residents of color, this may reflect discriminatory targeting rather than rational crime-control allocation. This question has both descriptive (what is the distribution?) and explanatory (why is the distribution as it is?) dimensions.

Data Collection:

Camera location data: Most cities maintain public records of CCTV installations in public spaces or use FOIA requests to obtain records of camera locations from police departments, transit authorities, and city administrative offices. Some cameras can be mapped through street-level observation. Google Street View has been used in published research to document CCTV presence.

Demographic data: US Census Bureau data (American Community Survey) provides neighborhood-level racial composition data at the census tract level.

Crime data: Most police departments publish annual crime statistics by precinct or neighborhood. FBI Uniform Crime Reports provide comparison benchmarks.

Analysis:

Quantitative analysis would use regression methods to model camera density (cameras per block or square mile) as a function of neighborhood characteristics including: racial composition (% residents of color); crime rate; median household income; population density; and proximity to commercial districts. The key question is whether racial composition predicts camera density after controlling for crime rate and other legitimate allocation factors.

Qualitative analysis would supplement the quantitative findings by examining: city council records and police department planning documents describing allocation criteria; interviews with city planners, police administrators, and community members; and analysis of contracts with surveillance vendors that might reveal deployment criteria.

Ethical Considerations:

IRB review required for interview components. FOIA requests are public activities but may create friction with city officials. Presenting findings to affected communities before publication is good practice for community-engaged research. Consider whether publication could have unintended effects (e.g., alerting actors to gaps in coverage).

Expected Challenges:

Camera location data is often incomplete, outdated, or withheld. Private CCTV (on businesses and residences) may be inaccessible even when police have access to the footage. Crime statistics reflect enforcement patterns, not underlying crime rates, which means crime statistics in heavily policed neighborhoods may reflect prior discriminatory policing rather than objective criminality.

What This Research Design Demonstrates:

This design integrates quantitative spatial analysis with qualitative institutional analysis and community-based research. It takes seriously both empirical rigor (controlling for legitimate allocation factors) and structural critique (asking whether ostensibly race-neutral factors mask racial targeting). It engages with surveillance as both a technical system (mappable cameras) and a social institution (shaped by political decisions, community relationships, and racial power structures).

For further reading on research methods in surveillance studies, see the methods-focused chapters in Lyon's Surveillance Studies: An Overview (2007) and the methodological appendices in Benjamin's Race After Technology (2019) and Eubanks's Automating Inequality (2018). For digital methods specifically, consult Salganik's Bit by Bit: Social Research in the Digital Age (2019), which is available free online.