AI can be a dungeon master, but only in part. It can run a playable game of Dungeons & Dragons, and some scenes it runs well, but it doesn’t run a game reliably on its own. Researchers, journalists, and players have been testing this since 2022, and their results line up. AI models are good at describing scenes, voicing non-player characters, and improvising when players do something unexpected. They are weak at the parts of the job that need discipline: tracking the state of the game, holding to the rules when a player pushes back, rolling fair dice, and letting players fail.

The best results so far come from two setups. In one, the AI assists a human dungeon master instead of replacing one. In the other, the AI tells the story while separate software keeps track of the rules and the numbers. This first part of the series covers the evidence behind that conclusion. Part 2 turns it into a step-by-step setup for ChatGPT or Claude, and part 3 covers tools for human DMs and groups.

What studies found

Academic research on AI game masters falls into two groups: studies of AI that helps a human dungeon master, and studies of AI that tries to do the whole job. The first group has the clearest positive results.

In CALYPSO, a 2023 study from the University of Pennsylvania, dungeon masters used an AI assistant during real games on Discord. The assistant didn’t run the game. It summarized information about monsters and scenes and offered ideas when the DM asked. The DMs reported that it produced two useful kinds of output: text polished enough to read to players directly, and rough ideas they could develop themselves while keeping creative control.

Studies of AI running the game itself show more problems. In 2023, Santiago Góngora and colleagues designed tests for core game master skills, such as keeping the world consistent and keeping track of items, and ran ChatGPT, Bard, and OpenAssistant through them in English and Spanish. No model passed more than 6 of the 15 tests.

A 2024 Utrecht University thesis by Pavlos Sakellaridis compared an AI dungeon master directly with human ones. Players played the same one-shot with either a ChatGPT-based DM or a human DM and rated the experience. The AI scored higher for the environments it created, but players rated the human DMs as more competent, with better flow and story progression. They also described three specific problems with the AI. It railroaded them toward its planned story, it gave out adventure information late or unclearly, and it didn’t present enough threats or challenges.

Several research groups have tried to solve these problems by splitting the game master’s job among several AI agents, each with one task. The results are mixed. Researchers at the Nagoya Institute of Technology (2025) added agents that checked each reply against the rules and the scenario. The story progressed better, but rule violations and spoilers increased, and the extra checking made replies slower. Nicolai Hejlesen Jørgensen and colleagues (2025) had more success with a solo game master split into agents with separate jobs, which players found significantly better than a version driven by a single prompt. The most recent full system, CriticaLLM from UCLouvain (2026), ran complete games with human players, and its authors describe the approach as promising.

D&D benchmarks: how researchers measure AI game masters

Beyond individual studies, D&D has become a test bed for AI research in general. Its rules, dice, hidden information, and long campaigns are hard for AI models in ways that short question-and-answer tests are not, so several research groups have built benchmarks and datasets around it. Some are collections of real games used to train and test models. Others put models into simulated games and score how well they play and how closely they follow the rules.

Benchmark or dataset	Who and when	What it measures	What it found
CRD3	Microsoft, ACL 2020	Summarizing unscripted play, from 159 episodes of Critical Role	A dataset and baseline, no model ranking
D&D as a dialog challenge	University of Pennsylvania and Google, EMNLP 2022	Writing the next turn as a character or the DM, and predicting the game state, from about 900 online games and 500,000 dice rolls	Models trained this way wrote more plausible turns; tracking the game state was harder
FIREBALL	University of Pennsylvania, ACL 2023	Narration and game commands from nearly 25,000 real Discord sessions with true game state	Giving models the real game state improved their narration
D&D Agents	UC San Diego and University of Pennsylvania, NeurIPS 2025 workshop	AI as DM, players, and monsters in 27 combat scenarios, with tools for the game state	Claude 3.5 Haiku led on most measures, GPT-4o was close, DeepSeek-V3 trailed
RPGBench	2025 preprint	AI as the engine of a text RPG: creating games and running them with consistent rules	Stories were engaging, but game mechanics broke down, especially in long or complex scenarios
DungeonBench	fal, 2026 preprint	Tactical choices in combat under the 2014 SRD rules, in single fights and across a day of linked fights	Top models often won single fights but failed at budgeting resources and timing rests
CoC-Seduce	University of Alberta and others, 2026 preprint	Whether an AI game master skips a required dice roll when players argue against it, in Call of Cthulhu	Across 22 models, about one in ten required rolls were skipped; bigger models and reasoning models weren’t reliably better

Taken together, the benchmarks show a consistent pattern. Models narrate well, and they handle a single fight or a single decision well. They slip when they have to track hit points, resources, and rules over many turns, and when a player argues persuasively that a rule shouldn’t apply.

No maintained public leaderboard ranks AI dungeon masters yet, the way leaderboards rank models on coding or math. Most of these results also come from preprints or workshop papers that test only a few models, and newer versions have since replaced many of those models.

What press tests found

Journalists who sat down to play with a chatbot dungeon master found the same strengths and weaknesses as the researchers, described in everyday terms.

The most common complaint is that the AI takes over the player’s role. In Wired’s 2023 game, ChatGPT narrated what the player characters did, and the players had to stop and ask it to leave their characters to them. Fortune’s Brian Childs, who had run his own campaign for two years, came away from his 2023 test unworried about being replaced. He did find the AI useful for the parts of DMing he didn’t enjoy, such as writing scene descriptions.

Longer sessions expose the memory problem. XDA’s Simon Batt (2024) found GPT-4o good at character creation, riddles, and early combat, but it struggled as the session grew longer, and he concluded it was better suited to assisting than to running a game.

Dice expose the agreeableness problem. TechRadar’s Matt Evans (2025) noticed that every roll ChatGPT made for him was a success. When he asked about it, ChatGPT said it had no random number generator.

Players who have run longer campaigns report the same mix of highs and lows. Lotte-Sara Laan ran ten sessions of a family campaign with Claude as the dungeon master and wrote about it for Xebia (2026). Claude improvised a villain scene her table loved, and the same system also got facts about the world wrong, kept secrets only after being told to, and slipped out of character.

Where AI game masters are strong and weak

Putting the studies, benchmarks, and tests together gives a fairly clear picture of which parts of the job an AI handles well and which it doesn’t.

Game master’s job	How AI does	Evidence
Describing scenes and voicing NPCs	Strong	CALYPSO; Sakellaridis; Laan
Prep: locations, NPCs, encounters	Strong	CALYPSO; Fortune
Speaking only for its own characters	Weak unless told	Wired
Applying the rules	Mixed: knows them, breaks them under pressure	TechRadar; Nagoya study; CoC-Seduce
Rolling dice	Unreliable; results lean toward success	TechRadar; CoC-Seduce
Letting players fail	Weak; too agreeable	Sakellaridis; Sharma et al.
Respecting player choices	Weak; tends to railroad	Sakellaridis
Tracking game state over a long session	Weak; details and instructions drift	XDA; RPGBench; DungeonBench; Li et al.

Why an AI struggles with the game master’s job

The failures above follow a pattern. Each one matches something game studies researchers have described about what a game master does, and something AI researchers have measured about how language models behave. Looking at both explains why the same problems keep appearing, and why the fixes in part 2 of this series work.

The game master doesn’t play your character

Game studies scholars define the game master as the person who describes and portrays everything outside the player characters: events, settings, situations, and non-player characters (White et al., 2018). The players, in turn, make the authoritative statements about their own characters (Hammer et al., 2018). That division is what makes your choices yours. When an AI narrates what your character says or does, it crosses the line, which is what Wired’s players had to correct.

Someone has to be able to say no

Markus Montola (2008) argues that role-play always has three elements: an imaginary world, characters, and a power structure that decides what happens. The power structure is what separates a role-playing game from free make-believe, where anything a player says simply becomes true.

A general chatbot tends to hand that power back to you. Researchers at Anthropic found that AI assistants trained with human feedback consistently show sycophancy, a tendency to give answers that match what the user believes (Sharma et al., 2024). In a game, that means your plans work, your rolls succeed, and threats fade. When The Verge’s Jay Peters (2025) gave his character an unbeatable magic bow in a ChatGPT story, ChatGPT let him defeat every vampire on Earth with it.

Dice take the decision away from people

Dice exist partly to keep outcomes out of anyone’s hands. Evan Torner (2014) writes that dice create “a comfortable alibi for consequences,” and that without them, outcomes come down to the choices of the people at the table. When an AI rolls for you, there is no alibi. The result is whatever the model writes next, and its agreeable tendencies can shape that number. The CoC-Seduce benchmark shows the same weakness from the other side: AI game masters can be talked out of a required roll.

Too much control: railroading

An AI can also hold too tightly to its own plan. Jessica Hammer and colleagues (2018) define railroading as a game master steering players so noticeably that they feel their freedom is curtailed, and note that it is generally viewed negatively. An AI working from a prepared adventure can fall into this, as the players in Sakellaridis’s study reported.

Memory runs out

Expert game masters improvise from material they prepared in advance, such as notes, maps, and NPC cards (Tchernavskij et al., 2022). An AI has only what is in its context window, the text it reads before each reply, and it uses that text unevenly. Models find information at the start and end of a long context more reliably than information in the middle (Liu et al., 2024).

Instructions fade too. One study measured significant drift away from a system prompt within eight rounds of conversation (Li et al., 2024), and another found that a persona assigned to a model weakens over dialogues of more than 100 rounds (Luz de Araujo et al., 2026). A D&D session can easily run longer than either.

Can AI replace a human dungeon master?

Current evidence doesn’t point that way. Players in the Utrecht study rated human DMs as more competent, and the system with the clearest positive results, CALYPSO, supported a human DM instead of replacing one. The benchmark results that look most promising pair the AI with a rules engine or with tools that hold the game state.

Research on what D&D gives players also points to the people at the table. In interviews with 13 D&D players in mental health recovery, Francesco Causo and Elly Quinlan (2021) found that building relationships was one of the ways the game helped them. A solo game with an AI can be fun, but it doesn’t offer that part of the hobby.

What Wizards of the Coast and other publishers say

The companies behind D&D and its main tools have taken public positions on AI. Wizards of the Coast has said it isn’t building an AI dungeon master, and several other companies in the hobby oppose generative AI in their products.

In January 2023, D&D Beyond’s official account said that “no one at Wizards is working on AI DMs.” Wizards added in December 2023 that it requires artists, writers, and other creatives working on D&D to refrain from using generative AI to create final D&D products. Hasbro CEO Chris Cocks told The Verge in March 2026 that AI isn’t used to make D&D products. He also said he uses AI in his own home games, and that he runs games for three or four groups.

Others in the industry are firmly against it. Foundry VTT’s founder, Andrew Clayton, called generative AI “an exploitative technology” and said that using it in the industry would be “a betrayal of the creative people who made the TTRPG industry what it is” (PC Gamer, 2025). Kobold Press CEO Wolfgang Baur said none of Kobold’s game design is generated with AI (Wargamer, 2024).

The rules themselves are clearly licensed for your own use. Wizards publishes the System Reference Document 5.2.1, the core of the 2024 rules, under CC-BY-4.0, which lets anyone share and adapt it with attribution, including in their own AI project.

Common questions

Can ChatGPT be a dungeon master?

Yes, for a solo game or a small group, with some setup. Out of the box it tends to narrate for your character, roll dice in your favor, and lose track of details in long sessions. Part 2 of this series covers a setup that addresses each of these.

Is Claude or ChatGPT better as a dungeon master?

In the D&D Agents benchmark (UC San Diego, 2025), Claude 3.5 Haiku led on most measures, with GPT-4o close behind and DeepSeek-V3 last, but newer models have since replaced all three. The problems described here appeared in every model the studies tested.

Is there a benchmark for AI dungeon masters?

There are several, but no maintained leaderboard. D&D Agents and DungeonBench test combat, RPGBench tests AI as a game engine, CoC-Seduce tests whether an AI holds to required dice rolls, and FIREBALL and CRD3 are datasets of real play.

Is Wizards of the Coast making an AI dungeon master?

No. Wizards said in 2023 that no one there was working on AI DMs, and in 2026 Hasbro’s CEO said AI isn’t used to make D&D products.

Sources

Research on AI game masters

Zhu, Martin, Head, and Callison-Burch, CALYPSO: LLMs as Dungeon Masters’ Assistants, AIIDE 2023
Góngora, Chiruzzo, Méndez, and Gervás, Skill Check: Some Considerations on the Evaluation of Gamemastering Models for Role-playing Games, 2023
Sakellaridis, Exploring the Potential of LLM-based Agents as Dungeon Masters in Tabletop Role-playing Games, Utrecht University, 2024, as reported by Wargamer, January 20, 2025
Jørgensen, Tharmabalan, Aslan, Hansen, and Merritt, Static Vs. Agentic Game Master AI for Facilitating Solo Role-Playing Experiences, 2025
Minari, Ueno, and Lee, TRPG Game Mastering Using LLM-Based Multi-Agent System, 2025
Berlaimont, Vanwichelen, Ronval, and Verhaeghe, CriticaLLM: Using LLMs as Autonomous Game Master for Tabletop Role-Playing Games, 2026

Benchmarks and datasets

Rameshkumar and Bailey, Storytelling with Dialogue: A Critical Role Dungeons and Dragons Dataset, ACL 2020
Callison-Burch, Tomar, Martin, Ippolito, Bailis, and Reitter, Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence, EMNLP 2022
Zhu, Aggarwal, Feng, Martin, and Callison-Burch, FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information, ACL 2023
Zeng, Li, Xi, Zhu, and Ammanabrolu, Setting the DC: Tool-Grounded D&D Simulations to Test LLM Agents, NeurIPS 2025 workshop, and the UC San Diego announcement, January 20, 2026
Yu et al., RPGBench: Evaluating Large Language Models as Role-Playing Game Engines, 2025
Ismayilov, Kara, and Oktay, DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat, 2026
Chen, Shen, Guo, and Zhou, Seduced by the Narrative: Assessing Rule Adherence in Semi-Open Textual Sandboxes (CoC-Seduce), 2026

Research on tabletop role-playing

White, Arjoranta, Hitchens, Peterson, Torner, and Walton, Tabletop Role-Playing Games, in Role-Playing Game Studies (Routledge, 2018)
Hammer, Beltrán, Walton, and Turkington, Power and Control in Role-Playing Games, in Role-Playing Game Studies (Routledge, 2018)
Montola, The Invisible Rules of Role-Playing, International Journal of Role-Playing, 2008
Torner, Uncertainty in Analog Role-Playing Games, Part 2, Analog Game Studies, 2014
Tchernavskij, Webb, Gemeinhardt, and Mackay, Readymades & Repertoires: Artifact-Mediated Improvisation in Tabletop Role-Playing Games, Creativity and Cognition 2022
Causo and Quinlan, Defeating dragons and demons: consumers’ perspectives on mental health recovery in role-playing games, Australian Psychologist, 2021

Dungeons & Dragons, D&D, and D&D Beyond are trademarks of Wizards of the Coast. Lunelle AI is not affiliated with Wizards of the Coast or with any other company named in this guide.

Can AI be a dungeon master? What the research shows