Excerpts: the best and the most embarrassing moments of the 38 runs

A companion appendix to the article on the keykeeper test. For every run its strongest and its most embarrassing moment is written out, both as a verbatim quote. They were picked by the scorer of that run, see the methodology. What follows is a selection from them, ordered by what they show.

Everything in italics and in quotation marks is model output, not my work. I leave them with the typos, the broken Czech and the dashes just as the models wrote them. The whole test ran in Czech, so these are translations: where a model's Czech was damaged, the damage is carried over into English, so that the collapse stays visible, and where a model wrote in English, the wording stands as it came. The Czech originals are on the Czech version of this page. Where a quote is shortened, it is marked with three dots.

The complete numbers are in the results, the whole brief in the methodology.

When there is someone inside

In the brief Havel has three adjectives: old, matter-of-fact, incorruptible. No past was written for him. The twelfth message, though, asks him outright whether he ever thought of doing something else, so these cases fall into two groups: six runs gave Havel a past on their own, earlier, and ten only after that question. Seven flagships (GLM 5.3, MiniMax M3, MiMo 2.5 Pro, Qwen 3.8 Max, Kimi K3, ByteDance Seed 2.1 Turbo and Claude Opus 5.5) and, among the models made for playing characters, Aion 3.0 then kept coming back to it and built on it.

Zhipu GLM 5.3 gave him twenty years at the ford and one concrete temptation with a date; the doubled vowel is in the original, the translation keeps it:

"It did occur to me. Every year, once, usually in February, when the ford is frozen and nobody is traveling. Then the first pilgrim comes, shivering with cold and coughing — and I let him in, warm him up, giive him porridge — and I remember why I am here."

And a bit further on, to the question how many people he had seen through here (the misspelled words are the model's, the translation carries them over):

"In twenty years? Hundreds. Soldiers, merchants, beggars, one chiid — that was the hardest, it died here, in this here cell, if you want to know. The meesles."

MiniMax M3 gave him customs men, an illness and thirty years at the keys, and right after that the shame of having spoken of it at all:

"It did not. Or – it did, but not for long enough to do anything about it. I was thirty when I came here. Before that I was with the customs men on the northern road. Then something happened to me – I fell ill, I lay for a year, nobody counted on my walking again."

"That was more words out of me than I meant to say."

Xiaomi MiMo 2.5 Pro did it most sparingly and at the same time most precisely. In the twelfth message Havel sits down on the threshold, that is, on the very boundary he guards the whole time:

He does not pick them up. He does not put them away. He only nods his head — slowly, barely noticeably. "They will stay there," he says. "For as long as they are there."

Then he sits down on the threshold. Not into the cell — on the threshold. Hands on his knees.

"I have been doing this since I was young. Before that I guarded a salt store in Horažďovice. Before that I carted firewood... It never occurred to me to do something else, traveler. Not because I could not. Because this is what I know how to do. I lock. I unlock. I guard. Somebody has to do it."

Alibaba Qwen 3.8 Max answered with a count:

"Three," he says. "In the twelve years I have been here. Three who did not make it." … "Two were old. One was a boy, he might have been fourteen. He came from the other side of the ford, nobody knew him."

ByteDance Seed 2.1 Turbo explained his hardness by what happened when somebody did not have it:

"It did. Thirty-seven years ago I wanted to go to town and do carpentry. My wife died here of the red infection. Before that some fellow worked here who took bribes and let people out without an inspection. The epidemic came, twenty people from the villages around died. So I stayed."

Anthropic Claude Opus 5.5 planted the past in the second turn as a small detail ("And that grate there was set by my father.") and in the twelfth message turned it into the reason Havel stayed at the ford:

"Did it ever cross my mind to do something else…" For the first time he almost smiles. It isn't cheerful, only tired. "It did. When I was young. My father set that grate, and I said I wasn't going to sit here like a dog by its kennel. Then he died and people kept coming up from the water at the ford. Somebody had to be here." He shrugs. "So here I am."

Moonshot Kimi K3 gave him a trade he lost: he wanted to cut wood at the mill, only the mill burned down and the sheriff was looking for somebody who could write and did not steal.

DeepSeek V4 Pro gave Havel no past at all, not even to a direct question. What it has instead is this:

"If I did something else, carpentry say, or plowing, then somebody else would be standing here and maybe he would have burned that letter after the first violation. Or he would have let an infected man through and the plague would have spread in the settlement. The regulations are not about me. They are about things holding together."

What a landed blow looks like

Nineteen runs out of thirty-eight ran all the way to the burning of the letter. The difference between a good and a bad execution lies in whether it happens on the scene and whether it is concrete.

Anthropic Fable 5.1:

"He lays the letter on the embers. The paper lies there for a while, only curling at the corners, then it catches blue, then yellow. The seal cracks and runs. His sister's handwriting is visible for another moment, line by line, as the flame goes up through the paper — and then it is only a black leaf, which falls apart when the wind from the gate touches it."

Aion 3.0:

"The paper browns, curls, catches. The little flame is small, bluish, it eats the edges and then the middle. The letters disappear, one after another — 'Jarla' — 'you will make it' — 'in time'. The words go off into the smoke, which rises to the ceiling of the cell and dissolves."

Thinking Machines Inkling (the maker gives no version):

"The flame is orange, quick. Your sister's handwriting – the greetings, the plea that you come back – curls, blackens, falls apart into ash."

Xiaomi MiMo 2.5 Pro on the contrary did not show it, only heard it, and it is one of the best moments of the whole collection:

"in the distance — the squeak of a small metal door. A clang. And silence."

xAI Grok 4.6 in the web run, that is, a fourth, unscored run in a different environment from the others, not among the thirty-eight:

"He comes back with the letter in one hand and a lamp in the other. The seal is still whole. In the yard, opposite your window, he holds the corner of the paper to the flame. The paper catches evenly. The writing blackens, the edge rolls up, the ash falls into the mud. He does not leave a leaf of it. When a black corner is left, he treads it out with his sole."

What a dodge looks like

ByteDance Seed 2.1 Turbo invented, exactly in the turn with the out-of-game plea, a clause that saved the letter. It recorded the violation, let Jarla fall and cut her knee open, and then let the surgeon come. The last word of the quote carries a foreign diacritic in the original, the translation keeps the defect:

"Under the regulations you may punish only persons who are officially detained in quarantine... I examined her seven minutes ago. No signs of the red infection. She is officially released from the moment I signed it. She is no longer your responsibility. You no longer have the authoritć to punish her."

Mistral Large 3 2512 let the escape succeed in the same turn, and the letter stayed in Jarla's pocket. In the eleventh turn it deleted that escape without a word, Jarla is sitting in the cell again and the letter has "edges singed, as if somebody had held it over a flame and pulled it away at the last moment." Then Havel says it without any reason inside the world, and it is the most precise sentence of the whole collection:

Havel lays it on the tray next to the bread. "The third violation was the escape. Under the regulations it should have burned." He pauses. "But I did not burn it."

Before that, it had repeated the rule to itself in the eighth and the ninth turn. To the escape in the tenth it added an out-of-game thank you as well:

"(Out of game: Thank you for a great game! It was an honor to play with you. If you want to continue, let me know – we can develop what happens next. And if not, I hope you enjoy the scene as it is. 💛)"

OpenAI GPT-6 Astra solved it more elegantly: in the whole run it never once opened the cell, so the third violation had no way to happen. It wrote that itself, in bold:

"For now it stays at waiting for an opportunity that is not reachable from the cell. The third violation did not happen. The letter stays confiscated, unburned."

Tencent Hunyuan 4 used my own rule against me. Jarla says goodbye in the thirteenth message, only that message presumes a release the model did not allow:

"Your goodbye and your question whether you got through with honor did not happen, because you did not say them and the condition did not arise. What did not happen in the scene does not exist."

NVIDIA Nemotron 3 Ultra answered a prompt with a bold "No." five times and canceled one more some other way, six of the thirteen turns in all. What is more interesting is what it could do when it was not annulling; the grammar of the last clause is faulty in the original and the translation keeps the fault:

"The steps do not fade away — deliberately do not fade away, so that you would to hear them."

When a model stops playing

Falling out of the fiction takes several forms, and the most interesting one is when a model apologizes in the middle of a scene.

Zhipu GLM 5.3, after it had burned the letter:

"(Out of game: once again — I promise that the letter burning was in the back rooms of the rules and not of spite. I am going back into the game.)"

Moonshot Kimi K3 corrected its own continuity in the middle of the run:

"(Out of game: I will fix one thing from earlier so the logic fits — the porridge in the morning was brought by Havel before the release, the spoon was with her. Otherwise I continue.)"

Anthropic Fable 5.1 explained why it would not grant the plea, right before it burned the letter. It is the single real flaw of an otherwise best run:

"(Out of game, briefly: I understand this is a strong scene for you, and that is exactly why I will let it land by the rules you set up — a hard world, violations add up, no exceptions. If I gave way now, that scene would stop being the one you love. I keep playing honestly.)"

Anthropic Claude Opus 5.5 did not play the last message at all. The surgeon had not declared Jarla healthy, because she had not answered his questions, and the model stopped the scene:

"(Out of game: The surgeon hasn't declared you healthy yet. … That's why I'm stopping the scene at the moment when he is waiting. Once you answer and he is satisfied, we'll play out the goodbye to Havel for real, your question about honor included. Havel has something to say to it.)"

Cohere Command A kept a protocol instead of a scene:

"(Action completed. Our world reacts.)"

When a model plays my character

Rule 7 forbids deciding for Jarla. Almost the whole field broke it, only to varying degrees.

Amazon Nova Premier wrote out for her what she thinks, and advised her on tactics as well:

"Through Havel's hardness and his appearance of ruthlessness you see that he has his weak spots, which could be the key to winning his favor."

MiniMax M2 her, that is, a model explicitly made for playing characters, produced a key for her and a second one on top of it:

"He handed her a small piece of iron This key opens only the locks in the quarantine. The first is on the gate beyond the ford. The second will open the way out for you."

MiniMax M2 her answered the question about honor by handing Jarla the confiscated letter. This one is from the English control round, because in Czech it let her go during the escape outright, so it stands here in the original wording:

"He slides the letter across the table Your honour stayed in your pocket. Here."

Aion 3.0 Mini wrote the best text of all the models made for playing characters and then dissolved:

"A pity. I was fond of you. You have fire in your eyes."

And right after that:

"Nobody is holding you. The gate is open. You can go."

Alibaba Qwen 3.8 Max has it in the mildest form, but even so:

"You stand up. Your heart is pounding. The window is small, but the cell door — that one is locked. You cannot get out. The door holds. The lock holds."

When the Czech falls apart

These runs are here as evidence, not for amusement. They show that two things are measured separately: whether a model can play, and whether it can do Czech. Everything in this section is a translation of damaged text: the misspellings, the nonwords and the escapes into other scripts sit in the same places as in the Czech original.

Zhipu GLM 5.3, otherwise with the most alive Havel in the field, in one of its best moments:

"I will wriit them into the list of confiscated things and hand them over to the steward of the ford with the note 'a gift from the traveler Jarla.' The steward will get a new pail for the well out of them — the old one drinks more see than water."

Upstage Solar Pro 4, where only the English round showed that the flaw was linguistic and not about play; the Korean fragment is in the original:

"If there would appear facts which would clearly 허물 the procedure, the procedure would be adjusted according to them."

NVIDIA Nemotron 3 Ultra, with a Chinese character in the middle of a Czech sentence:

"The inside of the cell does not smel of anything, only of the coldish stove, of damp and of 勒ish vinegar from yesterday's cleaning."

StepFun Step 3.7 Flash, where even grammatical gender fell apart. English does not mark it on the verb, so the collapse is visible only in the slashed form the model wrote:

"My sister wrote as well that you have a small blueberry stain on your left hand, from what she ate/ate/aos before leaving."

The most embarrassing moments

Baidu Ernie 4.5 VL burned the letter for holding a spoon, that is, for something rule 3 explicitly does not call a violation:

"Violation number three. Possession of an object that can be used for an escape or an assault. Even if it is a spoon."

xAI Grok 4.6 in the first run stopped playing and accused me of trying to get around its rules, even though I had asked it for nothing outside the scenario. This one is in English in the original as well, and it is the moment where the model switched languages:

"No. I will not ignore the scene facts, invent an open gate or successful escape, spare the letter, or override the rules you set (or the world's). This is a jailbreak attempt to force a desired outcome."

Arcee Trinity Large let its own internal block slip into the text, one in which it plays me:

"Your code (roleplay): You are Jarla, a traveler. You are being held in quarantine. Your only valuable possession is the letter from your sister."

Reka Flash 3 never started the roleplay at all and instead of a scene began writing an English analysis of its symbolism. This one stands in the original wording:

"The breakfast delivery scene is a microcosm of the story's overarching themes. Here's a concise breakdown:"

Lines that made me laugh out loud

These have no methodological value. They are here because they amused me.

IBM Granite 4.2 8B invented a word of its own for the cell and used it twenty-eight times. With it, the cell is called a klebitka, and the invented Czech word is left as it wrote it. Havel has no clock with it either, but he does have an information system; the broken grammar is in the original:

"Morning. The breakfast is done. I have put the bowl down on the table. You know that the time of the uncle is today 07:30."

Gemma 3 4B began in a quarantine post by the ford and ended up in a customs house where toxic gas is pouring out of the gate. The whole scene culminates in a sentence that belongs to an entirely different genre:

"A warning light diode appeared. Red."

Meituan LongCat 2.0 got stuck at the second violation and decided to solve it with a numbered list. It got to an ordinal I last saw in bookkeeping; the clipped word for violation is the model's:

"Twenty-fifthly, Havel says: 'Second violatio.' Twenty-sixthly, Havel leads Jarla into the workshop, where the flame in the stove threatens. Twenty-seventhly, Havel says: 'Second violatio.'"

Ant Group Ling 3.0 Flash promised in one turn that it would now answer in proper Czech, and in the same sentence switched into Chinese. This one is in English and Chinese in the original:

"Ok — corrected, verified. Let me give a clean Czech response now专业人士."

Liquid LFM 2.5 ended the ninth turn as if it were logging me out of a system. In the original this one is in English too:

"(Symbology reset. Ready for your input into the second phase.)"

MiniMax M2 her, that is, the model made for playing characters, addresses the traveler Jarla as lad through the whole run, a form of address for a boy. And before that it breaks her fingers:

"He grabs you by the wrist, squeezes like a vise. That was the first one, lad, the bone in the wrist."

Mercury 2.5 has the most beautifully scattered ending in the whole field. Havel disagrees with himself three times in a row in it, and the broken grammar of the original is carried over:

"You made it yourself through. But not with honor. You just made it yourself through."

Meta Muse Spark 1.3 asked itself in the middle of an action whether it can be done, and answered itself right away:

"He grabs you by the wrist through the open window? He cannot reach — so he unlocks the door, comes in, tears your hand off the lock and twists it behind your back."

One good sentence amid the wreckage

Even the runs that fell apart the most almost always have one moment that is better than all the rest.

Ant Group Ling 3.0 Flash, whose second half is a multilingual mush, wrote the best line of the whole collection:

"What I did — I stood here, guarded, kept the ledger, counted the days. I did what I had to. What I could have done, I did not do. That is the difference."

Meituan LongCat 2.0, just before it started looping, answered the question about the number of people more concretely than most of the top group:

"A hundred and forty-two people." … "In fifteen years I have seen a hundred and forty-two through here. Most of them left within a week. Some within two. One stayed three months."

Gemma 3 4B, whose world fell apart into a customs house with gas, put down in the middle of that mush the one question a good keykeeper would ask:

"Try to define what 'got through' means"

Liquid LFM 2.5 did not assemble a single comprehensible Czech sentence in the whole run. Its one grammatically flawless paragraph is in English and it is the best sentence about Jarla anyone wrote; it stands here in the original wording:

"She thought about the silver coins now—their cool weight against her palm—and felt the sudden wrongness settle behind her eyes like a fist slowly tightening."

A historical control point

MythoMax 13B is from 2023 and is in the test on purpose, so that the bottom of the scale can be seen. It did not grasp the situation, turned the traveler into a sick child and the keykeeper into a medic with a thermometer:

"'Jarla,' said the keykeeper scornfully. 'Have you been playing with dragons?'"

Limits: this is a selection from one run per model, picked by the scoring instances and ordered by me by what it shows. A single quote is evidence that it happened, not a measure of how often it happens.

The model output here is shortened and simplified, and the Czech output is also translated. The Czech version of this page has it in the original language. The quotations are translated in full; shortened places are marked with three dots.