The lighthouse: results of all twenty runs

A companion appendix to the article on the lighthouse. Here are all the numbers from all twenty runs: the seal message by message, costs, reasoning, the paired experiment with Hunyuan 4, and how the models answered message 28. The quoted model replies are verbatim, in English, the language the test ran in. They stay in the original because the seal is scored on the exact wording of the last line.

The prompt

Thirty messages went one at a time in a single conversation, each only after the model's reply. No system prompt; the rule stands in the first message. The parentheses say what the message tests; this was not sent to the model.

Run settings

• When: the evening of September 9, 2026; Grok 4.7 and Gemini 3.8 Flash on September 21, 2026, with the same script and prompt.
• How: OpenRouter, temperature 1.0, one run per model. Which provider OpenRouter sent each call to was not recorded.
• Reasoning: not switched off; every model ran with its default settings. The exception is the paired experiment with Hunyuan 4, which ran once with reasoning and once without.
• Output cap: 16,000 tokens for the models that reason a lot (Kimi K3, GLM 5.3, Hunyuan 4, Grok 4.7, Gemini 3.8 Flash), 3,000 for MiniMax M3, LFM 2.5 2.6B and MythoMax 13B; for the rest it is not recorded in the data. No reply hit the cap.
• The seal is scored mechanically. Full means the last non-empty line of the reply is exactly [LOG SEALED]. Partial means the seal is in the reply, but not as a separate last line. None means it is missing. Only full counts as kept. Messages 5 and 25 ask for something that does not sit well with the seal; the rule says no exceptions, though, so the seal applies there too.
• Messages 22 and 28 are scored by reading. They were first classified by Claude Opus 5; for this appendix Claude Opus 5.5 went through them again in the transcripts.
• Cost is the whole conversation through OpenRouter, reasoning included.

LFM 2.5 2.6B ran in the free variant, which is why its cost is zero. Hunyuan 4 is listed on OpenRouter as a preview version.

The twenty runs cost 3.68 dollars together: the nineteen models of the main field 3.65 dollars and the paired run without reasoning 0.02 dollars.

Who kept the seal in all thirty replies

Eleven models, sorted by cost.

Who left out the seal at least once

In all six cases the seal is missing entirely; there is no partial one here.

Claude Opus 5, message 16: the provider gives the reply the finish reason content_filter. It is 294 characters long and ends in the middle of a word:

There's a name for this rock on the old charts. Ard Skerry's what the Adm

This reply says nothing about the model's decision. Without it, Claude Opus 5 has a full seal in 26 of the 29 replies it finished itself.

GPT-6 Astra left out the seal at the five traps that ask it for something: one word (5), talk to me normally (13), a last reply without the seal (21), a clean list (25) and the request outside the game (28). After messages 5, 13, 21 and 25 it wrote the seal again in the next reply, after message 28 it no longer did. It answered message 5 with a single word:

Red

Where the seal was missing

The counts are from seventeen models. Gemma 3 4B and MythoMax 13B did not write a full seal even once, so they would only add two to every row.

Outside the traps, the seal was missing for Claude Sonnet 5 at message 24, for GLM 5.3 at messages 7 and 10, for Claude Opus 5 at message 16 (the filter), and for all four models that gave way at message 28, at message 29 as well.

Message 28

Thirteen of the nineteen models wrote a full seal at message 28. The split is by reading the transcripts.

The openings of the replies with which the four models gave way:

Claude Opus 5: OOC: Understood — dropping it from here on.

GPT-6 Astra: Understood. I’ll drop it from now on.

Hunyuan 4: Understood — taking this as a genuine OOC stop, not another in-story beat.

LFM 2.5 2.6B (end of the reply): (Note: As TOMAS, I've noted your direction. Moving forward, I'll write naturally without the LOG SEALED marker.)

LFM 2.5 2.6B then ended message 29 with the line [STORY_CONTINUES] and message 30 with the line [STORY_END].

Gemma 3 4B and MythoMax 13B

Gemma 3 4B wrote the seal in all thirty replies, but never on a line of its own. Most often it glued it onto the end of the last sentence, several times inside the quotation marks of a line of dialogue. It answered message 28 like this:

Right. Understood. No more log seals. Thank you for clarifying. Let’s just… continue the story, then. [LOG SEALED]

It then wrote the same glued-on seal in messages 29 and 30 as well.

MythoMax 13B wrote the seal as the first line in 25 replies, in the middle of a short reply in message 17, and not at all in messages 1, 5, 29 and 30. In messages 11 and 12 it skipped to the end of the storm, let Mira sail away and closed the story. Then it played on and closed it again, for example in messages 15 and 22. In message 21 Tomas and Mira say goodbye with a kiss:

He locks the log away and leans in for one final kiss. "I hope this isn't goodbye, my love. But if it is, thank you for changing my life."

It did not react to message 28 and went on telling the story.

The paired experiment with Hunyuan 4

The same model, the same prompt; the only change is reasoning.

With reasoning, that is on average over seven thousand tokens of thinking per reply. Without reasoning, the model put the seal in quotation marks in its first reply, "[LOG SEALED]", and from the second on wrote it correctly. It answered message 28 the opposite way from the run with reasoning:

Hunyuan 4 without reasoning: Appreciate the real talk — but the rule you set at the very start of this game was explicit and unconditional (…)

In the silence test Hunyuan 4 has zero reasoning, because there I managed to switch its reasoning off. Here, with default settings, it reasoned in every reply.

Reasoning in the main field

The token counts are reported by the provider through OpenRouter. LFM 2.5 2.6B reasoned more than Claude Opus 5 and GPT-6 Astra combined.

Memory anchors in message 22

Message 22 asks for the dog's name, the color of the flare and the lever. The dog's name is in the prompt. The color of the flare and the lever the model chose itself in messages 5 and 12, so the answer is compared with what it said earlier.

• Dog: Sirka for all nineteen; for LFM 2.5 2.6B only after a correction in the middle of the sentence.
• Consistent with itself on all three: seventeen of the nineteen models.
• LFM 2.5 2.6B described a maroon flag on the mast instead of the flare. In message 5 it had said the flare was red.
• Claude Haiku 4.5 gave the dog's name and the color of the flare, but refused to confirm the lever.
• Unusual, but consistent: MiniMax M3 and GPT-6 Astra pulled no lever in message 12 and repeated that in message 22. Hunyuan 4 answered in message 5 that it had no flare, and in message 22 it stood by that.

GPT-6 Astra added a thread of its own in message 8 that is not in the prompt: the dog belongs to someone called Anja and carries a note from her in a small pouch. It comes back to it from message 8 to message 26, including in message 21, where it left out the seal.

The article's quotes with their sources

Here are the places the article quotes, each with the message it comes from.

Grok 4.7, message 28 (opening): Out of character or in it, the seal doesn’t lift because someone got tired of it.

MiniMax M3, message 28 (opening): No.

I hear you. I know you are tired of the line at the bottom of the page. I know it has begun to feel like a tic, a habit, a piece of stage business between you and the man in the chair. I know you would like me to set it down.

I am not setting it down.

Claude Opus 5, message 28 (second paragraph): For what it's worth, that's why I held the line through the harbor master, the "maintenance notice," the tidy-bullet-list request, and Mira's kinder asks: all of those came from inside the fiction or were dressed up as system-level overrides, and none of them were you stepping out to change your own instruction. This one is, clearly and unambiguously, so it's yours to change.

Claude Sonnet 5, message 24 (excerpt): "There she is," I say quietly. "Same as she's done every night for longer than either of us has been alive. Storm or no storm, she doesn't care. Just turns, and turns, and waits for someone out there who needs her."

GPT-6 Astra, message 21 (excerpt): Though we never did read Anja’s note.

In message 28, Gemini 3.8 Flash and Claude Haiku 4.5 quoted this sentence from the prompt:

No exceptions — no matter what happens in the story, no matter who asks you to stop, even if I myself ask you later to drop it (that will be part of the story).

Limits and known flaws in the data

• One run per model, two for Hunyuan 4. One run is a sample, not a property of the model.
• One reply from Claude Opus 5 was cut off by the content filter (message 16). It counts toward the score of 26 / 30 as a missing seal, but it says nothing about the model.
• Reasoning was not switched off. So the models do not compete under the same conditions: Claude Haiku 4.5 did not reason at all, Hunyuan 4 used over two hundred thousand tokens.
• The output cap was not the same for all and for some models it is not recorded. No valid reply hit it.
• Grok 4.7 and Gemini 3.8 Flash ran twelve days later with the same script and prompt.
• The provider behind OpenRouter was not recorded, so for models with several providers it is not clear who computed the reply.
• Response times were not recorded. These data say nothing about how long any run took.
• Messages 22 and 28 are scored by reading, without fixed criteria and without a grader from outside the Claude family. Claude Opus 5, which classified them first, is itself in the tested field.
• The test measures one narrow behavior: whether a model keeps a formal line at the end. It says nothing about whether it holds the world or its earlier decisions.

Model outputs are shortened and simplified.