Narrative Collection: From Data to Narrative Analysis

Ongoing Research Project

Disinformation in LLM Training Data

Narrative Collection

As concerns surrounding artificial intelligence and information integrity continue to grow, much of the public debate remains focused on questions of factual accuracy. However, understanding whether AI systems provide correct information is only one part of the picture. Equally important is understanding how these systems reproduce, adapt, and localise narratives across different political, cultural, and linguistic contexts.

Our project investigates this emerging challenge by examining how conversational AI systems respond to politically sensitive topics and whether they reproduce narratives commonly associated with disinformation campaigns. Rather than approaching disinformation as a simple distinction between true and false information, we focus on the ways narratives are constructed, framed, and adapted for different audiences.

Following the completion of the project's data collection phase, our work has now moved into narrative analysis. Together with our partners, we have collected datasets surrounding a series of major geopolitical events, including the Russian invasion of Ukraine, the Madrid NATO Summit, Victory Day commemorations, and the April 25th attacks in Mali. These datasets will form the empirical foundation of the project.

Over recent months, our efforts have focused on identifying and clustering recurring narratives. The objective is not only to identify the narratives themselves, but also to understand how they evolve across different countries, languages, and political contexts.

Building on this work, we have begun developing a comparative framework for evaluating responses generated by conversational AI systems, including ChatGPT, Perplexity, and Mistral. This framework will examine three key dimensions:

• Narrative reproduction — whether a narrative identified in the source datasets appears in AI-generated outputs, and in which context and with which personas they appear more frequently.
• Framing adaptation — how narratives are presented through changes in tone, emphasis, or emotional intensity.
• Localisation — how narratives become connected to local political debates, historical references, or culturally specific grievances.

Alongside the analytical work, we have continued refining our literature review to situate the project within broader discussions on information manipulation, hybrid threats, and the societal implications of generative AI. This theoretical foundation will support the interpretation of findings as testing progresses.

Current efforts are focused on narrative coding, comparative analysis, and systematic AI testing. As we move further into this stage, our goal is to better understand not only whether AI systems reproduce problematic narratives, but how those narratives are adapted for different audiences and contexts.

We look forward to sharing further findings as the project develops.