Case Study 5.1: Method Shapes What You Find — Three Studies of the Kalosverse

Introduction: The Same Community, Different Discoveries

This case study demonstrates a principle that Chapter 5 argues but cannot fully show within the chapter's scope: the choice of research method shapes not just how you find things but what you find. Three hypothetical studies of the Kalosverse MCU fan community — one using network analysis, one using digital ethnography, and one using computational content analysis — would produce genuinely different accounts of the same community. Some of those differences are complementary; others are, at least superficially, contradictory. Understanding how method shapes finding is essential for reading fan studies scholarship critically and for designing your own research.

Study A: Network Analysis of the Kalosverse Reddit Community

Research Question: What is the structural organization of the Kalosverse subreddit community, and which accounts serve as most central to community communication?

Data and Method: The researcher uses the Reddit API (during a period of open API access) to collect all posts and comments from the Kalosverse subreddit over a twelve-month period: approximately 18,000 posts and 240,000 comments from 12,000 unique accounts. They construct a directed network in which nodes are accounts and edges represent comment responses (Account A responded to Account B's post/comment). They compute standard network metrics: degree centrality (how many connections does each node have?), betweenness centrality (how often does each node appear on the shortest path between other nodes?), and clustering coefficient (how interconnected are a node's neighbors?). They also use community detection algorithms to identify sub-clusters within the network.

Key Findings:

  1. The network has a strongly skewed degree distribution: a small number of accounts are extremely highly connected, while the majority of accounts have very few connections. The top 1% of accounts by degree centrality account for approximately 35% of all connections. This is a "power law" distribution, typical of online communities.

  2. KingdomKeeper_7 appears as the second-highest betweenness centrality node in the network — positioned between multiple sub-communities in a way that makes them a structural bridge. When the researcher removes KingdomKeeper_7 from the network and recalculates, the community separates into more distinct clusters with less cross-cluster communication.

  3. Community detection reveals five identifiable sub-clusters, loosely corresponding to: main MCU film discussion; Disney+ series discussion; fan art and fan fiction community; character-specific communities (Iron Man/Captain America focus, Wakanda/Black Panther focus, etc.); and a smaller meta-commentary cluster discussing fandom itself.

  4. IronHeartForever is highly central within the fan art sub-cluster but has fewer connections to the main film discussion cluster. Priya Anand's account shows high betweenness centrality across several sub-clusters.

What this study sees: Community structure, central actors, sub-community organization, the critical role of specific bridging accounts in maintaining community cohesion.

What this study cannot see: Why people participate, what the community means to them, the norms and culture that govern interaction, the emotional dimensions of community membership, the racial and gender dynamics that shape who participates and how. Network analysis treats all nodes as equivalent — it cannot see that IronHeartForever is a Black woman fan artist or that her relative isolation from the main discussion cluster might be connected to the racial dynamics of the community.

Study B: Digital Ethnography of the Kalosverse Discord

Research Question: How do Kalosverse community members negotiate the norms around "serious" versus "casual" MCU engagement, and how does this norm negotiation shape who feels like a full community member?

Data and Method: The researcher, with IRB approval and community notification, spends eight months in immersive observation of the Kalosverse Discord server — reading conversations in real time, participating minimally (asking occasional clarifying questions but primarily observing), maintaining detailed field notes, and conducting twelve semi-structured interviews with community members including KingdomKeeper_7, IronHeartForever, and three community members who describe themselves as "newer" fans.

Key Findings:

  1. The Kalosverse Discord has an explicit hierarchy of channels that reflects an implicit hierarchy of engagement: the #lore-deep-dives channel (discussed in detail, high status) versus #first-impressions (low status, newcomer-facing). Long-term members who have been in the community since before the Disney+ era refer to the newer members who came in through Disney+ series as "stream fans" — a term with mixed connotations, sometimes affectionate, sometimes dismissive.

  2. IronHeartForever describes, in interview, a specific experience of her fan art: "When I post Riri art, I get nice comments from maybe thirty people. When I post Tony Stark art, I get similar comments from maybe three hundred people. The community tells me my Riri art is good, but what it actually does with my art is different." She has continued to post primarily Riri art: "That's who I'm here for. I'm not changing what I make for the engagement counts."

  3. KingdomKeeper_7's moderation work involves not just enforcing formal rules but maintaining what they describe as "the vibe" — an informal sense of the community's identity and culture that long-term members carry and newer members slowly learn. Mod decisions are frequently about whether a post fits the vibe more than whether it violates a specific rule.

  4. Priya Anand is observed navigating a specific tension: she participates as a genuine community member, but her grad student identity (when it comes up) produces a subtle shift in how community members interact with her — simultaneously more careful and more philosophical, as if they are aware of being observed for academic purposes.

What this study sees: Community norms, the lived experience of membership, the emotional dimensions of belonging and not belonging, informal hierarchies that formal community documents do not describe, the specific texture of IronHeartForever's experience.

What this study cannot see: How widespread these patterns are — the ethnographer has been in one server, observing specific interactions; she cannot know whether what she has observed is representative of the Kalosverse community more broadly. She cannot systematically describe the racial patterns in fan art engagement that IronHeartForever describes — she has one data point, which is IronHeartForever's reported experience.

Study C: Computational Content Analysis of Kalosverse Fan Fiction

Research Question: How do Kalosverse fan fiction authors represent characters of color relative to white characters in terms of frequency, narrative centrality, and sentiment?

Data and Method: The researcher uses the AO3 API to collect all fan fiction tagged with relevant Kalosverse tags: 4,300 works, approximately 18 million words. They develop a named entity recognition (NER) model fine-tuned on Marvel character names to identify character mentions. They use a sentiment analysis model to score the emotional valence of sentences in which characters appear. They analyze: how often each character is mentioned; in how many works each character appears as a main character (operationalized as appearing in more than 15% of the work's sentences); and the average sentiment of sentences in which each character appears.

Key Findings:

  1. Iron Man/Tony Stark appears in 72% of all works; Riri Williams/Iron Heart appears in 11%. When controlling for canonical screen time, the disparity remains significant: characters with comparable canonical roles show substantially different fan fiction representation rates, with white characters consistently overrepresented.

  2. The sentiment scores for sentences featuring characters of color are not consistently negative — they are not portrayed more negatively than white characters. But they appear in a narrower range of narrative contexts: they are more likely to appear in sentences related to action or conflict, less likely to appear in sentences related to interiority, emotional complexity, or relationship development.

  3. Works by authors tagged as members of the Kalosverse community (identified through cross-referencing with subreddit activity) show the same patterns, suggesting this is not an artifact of studying a broader MCU archive.

What this study sees: Systematic patterns across thousands of texts that no human reader could observe; the scale makes invisible patterns visible; the findings can be stated with statistical precision.

What this study cannot see: Why these patterns exist — whether they reflect author preferences, community norms, the gravitational pull of canonical prominence, or racial dynamics in who writes fan fiction and for whom. It cannot see the meaning that IronHeartForever and other fans give to their deliberate focus on characters of color. It cannot see the community norms around representation debates, the emotional stakes of the patterns it identifies, or whether fans are aware of these patterns.

Synthesis: What Three Studies Together Tell You

The three studies find: - Study A: IronHeartForever occupies a more isolated structural position in the Kalosverse network than fan artists focused on white characters. - Study B: IronHeartForever is aware of and describes a specific engagement differential between her Riri art and art of white characters, and has made a deliberate choice to continue posting Riri art despite this differential. - Study C: Characters of color, including Riri Williams, are systematically underrepresented in Kalosverse fan fiction in terms of narrative centrality and emotional complexity.

Together, these findings constitute a coherent account: the Kalosverse community shows systematic patterns of racially differentiated engagement that manifest at the structural level (network), the interpersonal level (IronHeartForever's described experience), and the textual level (fan fiction representation patterns). No single study could have established this account. The convergence of findings across three different methods, using three different types of data, makes the account significantly more robust than any single study would have been.

At the same time, the studies are not perfectly complementary. Study B's finding that IronHeartForever has chosen to continue focusing on characters of color, despite lower engagement, introduces a dimension of agency and deliberate resistance that Studies A and C cannot capture. The network analysis sees her as occupying a less central position; the ethnography sees her as occupying a position she has deliberately chosen as an act of creative commitment. Both are true; they describe different dimensions of the same situation.

Discussion Questions

  1. If you could only conduct one of these three studies, which would you choose, and why? What question is most important for understanding the Kalosverse community, and which method best addresses it?

  2. The three studies produce findings that are complementary in some respects and potentially in tension in others. How should researchers handle convergent vs. divergent findings across methods? When do divergent findings indicate a flaw in one study, and when do they indicate that both findings are capturing something real?

  3. Study B raises a finding that Study C cannot address: IronHeartForever's deliberate choice to continue posting Riri art despite lower engagement. How does this finding change your interpretation of Study C's findings? Does it make the patterns Study C identified more or less significant?

  4. Each study required different ethical protocols. Which study raises the most significant ethical concerns, in your view, and why? What would you change about the ethical design of the most ethically challenging study?