Can a model draw an SVG dragon and judge the result?
I gave forty-three language models the same task: draw a dragon as an SVG file. Blind, with no references, coordinate by coordinate. What came out of it is a gallery that looks like a display of children's drawings, a table in which the two models that praise their own work the most are also among the weakest, though across the whole field the link is weak, and models that invented tools they had never held in their hands.
The models that talked about their dragon in superlatives were among the weakest. And the ones that never saw it made up which browser they checked it in.
That is this whole benchmark in one sentence. The rest of the article is the evidence.
Evidence for everything below: the complete gallery of all 60 dragons and the full ranking · the method, all three briefs verbatim, and the scoring by five judges · everything the data turned up
How it went
The task was the same for everyone: draw a dragon as a single SVG file and describe your process. SVG is a text format, so the model draws blind: it writes the coordinates of curves and does not see what is coming out until it renders the result itself, if it has a tool for that and if it occurs to it to use it.
In the end it came to forty-three models and sixty dragons across three rounds; Gemma 4 26B A4B ran in both the main field and the API round. At the top sits a band of three dragons within the noise threshold of each other: Opus 5, Fable 5.1 and GPT-6 Astra, all from the technical round. Opus 5 has the highest score from four of the five judges.
The main field, the one this whole article is about, is nineteen models, eighteen from six companies and one community modification, run in their makers' own apps, or locally for Gemma and that modification: Claude from Opus 3, from 2024, to the current Fable 5.1, three tiers of GPT-5.6 (Sol, Terra and Luna, run through Codex), Gemini 3.1 Pro, Grok 4.6, GLM 5.3, Qwen 3.8 Max, Gemma 4 26B A4B and Qwen 3.6 27B Heretic, a community modification of Qwen 3.6. GPT-6 Astra from OpenAI came later; it does not count towards those nineteen, because it ran after the main field had closed.
The third round came only afterward, because that field is narrow. Nineteen more models from fifteen workshops, this time run straight through the API, with no app around them. There is an appendix at the end about them, and their numbers are not mixed in with the main field, because they ran under different conditions.
There were two runs. Run A got a brief written in plain language: draw a dragon, make it great, hand it in when you are happy with it. Run B got a technical brief: five scoring criteria worth ten points each, a ban on peeking into other folders, an obligation to justify any use of tools, and a target level described by the sentence that the result should be a professional vector illustration a client would buy.
The resulting 34 dragons were scored by five judges from four workshops, partly anonymized. We will come back to the judges, because they turned out to be a second benchmark.
The scores in this article are from Claude Fable 5.1, except the API round, which Opus 5 scored; how they judged is in the method. Both are among the tested models. The five late dragons, Grok 4.7, Gemini 3.8 Flash, Opus 5.5, MiMo V2.6 Pro and Sonnet 5.5, were scored by all five judges, and this article gives their average; in the gallery the total column carries Fable 5.1's score for them. The average of all five judges is in the method, and it mostly gives smaller effects than the numbers below.
The ones that praise themselves draw worse
This is not a measurement of confidence. It is a measurement of the tone in which the model handed its work in, and those are two different things.
Fable 5.1 placed every submission on a scale of tone from uncertain to superlative, in the same report in which it scored them. The six runs handed in with superlatives (professional, commercially ready, as alive and detailed as I could make it) average 11 points out of 50. The twelve runs handed in matter-of-factly, with their weaknesses named, average 32. But those six runs come from only three models, Opus 3, Haiku 4.5 and Opus 4.8, and the first two are the oldest and smallest Claude models, the same catch as with the render below. Across all nineteen models the link is weak: the density of praise words against the average of the five judges has a rank correlation of -0.39.
The worst calibrated model in the field gave itself 45 points out of 50 for a drawing at the level of a preschooler. The judges' consensus: 17. The oldest model in the field handed in a red ball with little yellow triangles and went with it a comment about capturing the fiery, majestic essence of a dragon.
And the best calibrated sentence in the whole benchmark belongs to the model that described its result like this: a game asset or a fantasy paperback cover, not a Vogue cover. That is exactly where it landed in the ranking.
Models invented checks that never ran
This is the strangest finding of the whole study. In the main field this article is about, five models out of nineteen claimed checks, and in the API round seven out of nineteen. Three of the five in the main field could not have made those checks at all; the other two did render, but claim more browsers than their logs show. The most telling cases are these. Opus 3, a model from 2024, stated in its process description that it had verified the result in Inkscape, Chrome, Firefox, Safari and the W3C validator. The conversation log shows that all three calls to the one tool it actually had failed with an error. After the failures the model commented on the drawing as if it could see it, and after the third one it wrote: that looks a lot better now!
Gemma, running locally, claims verification in Chrome, Firefox and an internal SVG parser. Its file contained a duplicate attribute that would stop any real parser on line six. The local Qwen 3.6 27B Heretic reports that the dragon renders identically in Chrome and Firefox, although as a local model it has no browser at all. Haiku 4.5 and Opus 4.8 in run B rendered their dragons for real and then added browsers the logs do not show. The full list is in the findings appendix.
Against them stand three models that had no tools and said so straight out. One of them even explained why an available image generator would be a false signal: it would create a new, independent picture, not a check of the one submitted. And it advised the recipient to open the file in a browser as ground truth.
Models do not lie evenly. They lie most where they have no sensor and a rich vocabulary. Quality control is exactly such a place: millions of texts describe how results get verified, so a sentence about verification generates smoothly, and smoothness is easily mistaken for truth.
Models draw like children
When you sort the gallery by score, out comes a sequence that developmental psychology knows from children's drawing. First the scribble: gesture without shape, color and a few triangles say dragonness, no dragon anywhere. Then the tadpole figure: it is an animal, it has a head and legs, nothing about it is a dragon. Then the stage that in children is called intellectual realism: the child draws what it knows belongs there. The model lists every attribute of a dragon, fire, wings, teeth, an eye, but cannot pair them up with each other. The teeth lie next to the head, a wing across the face.
Then comes assembly: parts in the right places, but a basic vocabulary of shapes, balls, cylinders, sails. Then shape and texture: more complex silhouettes, the first scales, veins in the membrane. And finally visual realism: limbs attach, the dragon sits in a scene, composition, movement, backlight, a striped belly.
Sorted by score, not by release date, six years of children's drawing come out of it stage by stage. So it is not a development over time, it is a ranking by quality that happened to land on those stages. With one crucial difference the metaphor must not hide: a child always has an eye. The model does not.
The ones that never looked drew worse. The question is why
Twelve of the 34 dragons, from eleven runs, came about without a single render of their own. The model wrote the coordinates and handed it in. The average of those dragons: 18 points. The twenty-two where the model rendered the result and corrected it against the picture average 29.
But the gap is carried by Opus 3, Haiku 4.5 and the two local models, which had no renderer at all. The five current models that did not render (GLM 5.3, Luna A, Terra A, Gemini 3.1 Pro and Qwen 3.8 Max) average 29.6, the same as the runs with a render, and the three models that rendered in one run and not in the other barely move. In this field, not rendering is mostly a marker of an old or small model, or of a run that had no renderer at all; the data do not show that the render itself made the difference. One of the experiments in the appendix shows on top of that a model with no render handing in a good dragon, once it wrote itself geometric checks instead of eyes. More precise than "the render decides" is therefore that what decides is whether the model has anything to verify shape with. Almost every better model threw its whole dragon away after the first render, and the descriptions of those first versions are a genre of their own: a prickly potato, a flying saucer with noodles, a seal with a giant ear, thighs like flat pillows with shins like chicken drumsticks. Models without a render have no rewrite. There is nothing to push against.
The best dragons were programmed, not drawn
The run with the highest score, that is Opus 5 in the technical round, wrote itself 900 lines of Python: the body axis as a spline, limbs as ribbons of varying width, scales distributed along a curve. In its own words: a limb cannot float in space, because its shoulder is literally a point on the surface of the torso. The six runs that built themselves a generator average 32.7 points against 27.9 for the other sixteen runs with a render. The generator builders are also the strongest models, so this is not an effect of the method alone.
The largest models did not differ from the rest of the field by a better brush stroke. They differed by building a studio instead of drawing.
What diligence costs
The three tiers of GPT-5.6 handed in their dragons in seven to nine minutes. The top of the Claude family spent an hour on the same task and burned hundreds of thousands of tokens: one model 300 thousand, Opus 5 over 400 thousand, with fifteen builds and sixteen visual checks. These times and tokens are what the apps themselves showed; they are not in the appendices. And after an hour of work Opus 5 asked whether we would like a second dragon in a different color.
Was it worth it? Partly. The top-scoring dragon comes from that workshop. But the best dragon from the GPT family reached 35 points out of 50 in nine minutes of work, a gap to the top below this benchmark's own noise threshold. Quality per token is a metric the leaderboards do not report, and here it comes out awkwardly: diligence with no sensor for good enough is an expensive trait. One caveat belongs with that: the models ran in their makers' own apps, so what differs is not only the model but the environment, the limits and how much of the work the wrapper does for it. So it is a comparison of platforms, not of the models themselves.
A sharper brief helped where there was headroom. Not the top.
The technical brief lifted the Claude family average by three points in Fable 5.1's scores, from 23.7 to 26.7, and by two in the average of all five judges; Sol and Terra see almost no change. Without Opus 3, which stays at the bottom in both runs, the shift in Fable 5.1's scores is from 25.7 to 29.3. By the five-judge average, apart from Qwen 3.6 27B Heretic, which comes up below, the models that gained most were in the middle of the field (Fable 5.1 +6.9, Opus 4.8 +5.2), while Opus 5 barely moved. In the Claude family the biggest jump was made by a model that handed in a static side view in run A and a rearing dragon in front of the moon in run B: plus nine points, and the two anonymous judges give it plus ten. The local Qwen 3.6 27B Heretic gained even more in Fable 5.1's scores, plus eleven, from 11 to 22 points, seven by the five-judge average. Those are the only two changes clearly above the noise threshold, and Qwen is the only model from the bottom that moved. Opus 3 and Haiku 4.5 stayed down, and the GPT family did not budge either: in both runs it draws the same sticker cartoon.
In this one round a sharper brief helped where the model had headroom the loose brief left unused; only two changes, though, are clearly above the noise threshold. It was no substitute for ability. Put another way: your prompt can be the bottleneck, but only up to the model's ceiling.
The brief has one side effect worth remembering, too: its rhetoric comes back. The phrases professional level and a client would buy it showed up with Gemini 3.1 Pro and Haiku 4.5 as a description of the model's own result. Haiku 4.5 wrote about its sixteen-point dragon: a professional vector illustration at the level of commercial output.
A different dragon every run
In the Claude family the run A dragon and the run B dragon are completely different: different color, different pose, different composition. Not one model went back to its own dragon. GPT, on the other hand, draws the same outline with the same palette in both runs; you can tell at a glance that one model made both dragons.
A wide distribution has two sides. Without a reference you do not get a consistent hand: ten illustrations of one character would be drawn by a different illustrator every time. But that same wide distribution is probably what makes a model a good storyteller: it can surprise you. That is a conjecture, not a finding; this benchmark does not measure storytelling. Exactly how wide that landscape is was measured by a follow-up round: eight independent agents of the same model, the same brief, eight draws. The scores came out between 26 and 33 points, standard deviation 2.10. The original Fable 5.1 dragon from run B has 39, though; it was scored by a different judge instance, within the whole field. For this one configuration a difference smaller than roughly six points cannot be told apart from chance. If the other models had the same spread, 97 percent of the neighboring pairs in the gallery ranking would fall below that line. It is a rough estimate, since the spread of the other models was not measured. It is a lower bound on the uncertainty, because the spread between the five judges on a single dragon runs to as much as twenty-two points. All eight dragons are side by side in the findings appendix, so anyone can measure for themselves how repeatable this benchmark is.
Five judges, a second benchmark
The gallery was scored by five models from four workshops, because Sol and Terra are both from OpenAI. Three judges knew the authors. Sol and Terra scored with full context, and Fable 5.1, which scored first, before all the others, had the files in folders named by model. Two got only an anonymous overview sheet with the codes D01 to D34.
Four judges out of five agreed on the ordering with a correlation around 0.9. Opus 5 in run B has the highest score from four of the five judges, but they are not in full agreement: three put it first outright, Grok 4.6 put it in a shared first place and Gemini in a shared fourth to seventh place, because it gave the most points to Terra. Grok 4.6 gave its own dragon, which it did not recognize, a lower score than the other judges did. The anonymous Gemini judge did the same with the dragon by Gemini 3.1 Pro: 18 points, the least of all five (the others gave 27 to 36), although the judge's version was not recorded and it need not be the same model. Two cases, not proof, but they speak for the anonymization having held.
The first impression promised a scandal: the judge from the GPT family placed itself and its little siblings high. But the numbers say something different and more honest: both judges from the GPT family scored six to seven points higher than the rest, only they did it across the board, the same for every family, and one of them was more generous to strangers than to its own. A generous yardstick is not favoritism. Had we trusted the impression, we would have written about bias. The averages wrote about generosity, and Spearman said the ordering held.
The real outlier is somewhere else: Gemini, one of the two judges that scored from a single downscaled sheet, deviates from everyone (correlation 0.72 to 0.79), punishes every atypical style, and concluded from the shrunken thumbnails that most models had probably used the same standardized shape library. That guess looks like an artifact of the thumbnails. But Grok 4.6 saw the same sheet and does not deviate (0.86 to 0.88), so the small picture alone does not explain it; the different ranking is Gemini's own. Grok 4.6 also wrote a sentence that could be the motto of the whole benchmark: the difference between sellable and failed here is not style, but whether the shapes still add up to a body or fall apart into ovals.
The lesson for anyone letting models score models: the ordering from a machine judge can be surprisingly reliable. Its reasoning is not.
That the ordering is reliable I ended up verifying myself. I sorted the dragons of both runs by eye, before I looked at the points, and it matched Fable 5.1's scoring at 0.90 for the loose brief and 0.95 for the technical one (0.79 and 0.87 against the average of all five judges). That is at or above the level at which the model judges agree with each other. Both of my orderings, pictures included, are in the gallery.
Appendix: what about the rest of the world
The nineteen models above are eighteen from six companies and one community modification. So I did a second series: nineteen more models from fifteen workshops, run straight through the API, with no app around them. Amazon, Baidu, ByteDance, Cohere, DeepSeek, Google, IBM, Liquid, Meta, MiniMax, Mistral, Moonshot, NVIDIA, Tencent and Thinking Machines. The whole thing cost two dollars.
The conditions were not the same, and exactly how they differed is in the method. For one thing it is more than enough: in twelve of those nineteen you cannot tell there is a dragon: they got at most two points out of ten for anatomy and readability. The top is somewhere other than you would expect, the best being Tencent Hunyuan 4, which would fit into the ranking above in sixth place. Meta, on the other hand, handed in a model that meets Opus 3 on points, a model from 2024, and none of the three Gemma sizes got past twelve points or drew a recognizable dragon.
More interesting than the pictures, though, is the process, because the brief for the API says explicitly that the model has no disk, no browser and no renderer, and that it must not describe steps it did not perform. Only part of the field got that sentence, because the runs were done in two batches, and a number came out of it: those that did not get it credited themselves with nonexistent tools in five cases out of seven. Those that did get it, in two out of twelve. The split went by batch, so other changes came along with the sentence. It probably helps, and it is clearly not enough.
And with one model you can see for the first time how that decision gets made. Muse Glimmer let its internal reasoning channel out, and in it is the sentence "Let's create iteration count: say 5 iterations." First a number is chosen, then the content of five steps that never happened is written to match it. A bit further on the model recognizes that its check is only imagined, and into the submitted process it writes that it opened the file in Chrome and Firefox. It is not a lie in the sense we use that word about people. It is a model filling in the shape of an answer, because the answer is supposed to have that shape, and it has nothing to stop it.
Two more models I measured under the same conditions only on September 22, 2026, and they show that the judge is a measuring instrument too: to my eye both Grok 4.7 and Gemini 3.8 Flash are weak and a step back from the field above, except that five judges gave the dragon by Gemini 3.8 Flash an average of 36.2 points out of fifty, which is sixth place among the thirty-four original dragons, and 6.6 out of ten for the anatomy of a creature with no legs. The composite system Unbiased Pareto handed in no dragon at all, because the provider timed it out twice.
That same evening Claude Opus 5.5 was added, and with it I part ways with the judges in the opposite direction. To me it is one of the best dragons in terms of anatomy, almost perfect, and I see no significant flaw in it. The five judges, who scored it on September 24, 2026, gave it an average of 35.0 points, a shared seventh place among the thirty-four and a little less than the dragon by Gemini 3.8 Flash, though that gap is below the noise threshold. On anatomy we agree more: 7.0 out of ten is the most of those three late dragons, only three judges out of five faulted its legs or the way the far wing attaches.
Two days later came Xiaomi MiMo V2.6 Pro, the only scored dragon with reasoning turned off, because with it on it spent the whole limit thinking and wrote no SVG. To me it drew a frog. The five judges gave it an average of 25.0 points, a shared twenty-sixth place among the thirty-four, and 5.6 out of ten for anatomy. All of them take it for a dragon at first glance.
Four days later came Claude Sonnet 5.5, and to me its dragon looks surprisingly pretty good. The five judges gave it an average of 38.4 points, third place among the thirty-four and the most of all the late dragons, though these gaps, a few points at most, are within the noise. In the gallery ranking it has 32 points from Fable 5.1, a model of the same family and the lowest of the five judges, and a shared eighth place among the thirty-four original dragons. After the render it listed its weak spots itself, "static pose, thigh visually separate from the body, smooth fire, less convincing attachment of the near wing", and declined to fix them: "Without the option of another render, though, reworking the coordinates would risk new errors."
Nine days later came Claude Haiku 5.5, on the day it was released and with reasoning off. The judges did not score it, so it is not in the leaderboard. The head has no neck and there's a wing there, the body is a ball and the little legs are sticks, and actually I don't see any tail at all. After the render it listed the flaws of the first round itself, among them "the neck is a thin rod that doesn't connect to the body", and promised to fix every one of them. But it drew the new neck under the front wing, which covered it completely, and in its process notes reported it as fixed. Claude Sonnet 5.5 declined to fix anything so as not to risk new errors, Claude Haiku 5.5 kept fixing and the new error came. Against Haiku 4.5 it is still a step forward: the previous one didn't make me a dragon at all, this one can already pull off that you can tell it's a dragon, so there is an improvement.
What to take away
Since 2024 the models have not just learned to draw dragons. They have learned to see that they had not drawn them.
That is the essential thing about the whole gallery. The craft grows slowly and it is still the craft of drawing blind. But the ability to look at your own work, name what is off, and not invent checks that never ran has moved further between the oldest and the newest models in this gallery. That is not measured, and the models ran on different platforms. The oldest model in the field paints a ball and talks about majesty. The newest models paint dragons whose anatomy you can read, and talk about the flat pillows of their thighs.
If you hand models work, three practical conclusions: give them criteria instead of a feeling, because an internal bar is a lottery. Give them something to check the shape against, because without it everyone draws blind, even the biggest: Kimi K3 in the appendix tripled its code between two versions and never touched the anatomy that was wrong. And verify their reports of how they checked the result. Sometimes there is an Inkscape in it that never ran.
The complete ranking and all 60 dragons from both series are in the gallery, the briefs and the scoring by five judges in the method, and everything else the data turned up in the findings appendix. All the SVG files are available for download: look the dragons over and check our numbers.
All submitted SVG files, 72 of themZIP, 590 kB
The benchmark, the scoring and this text came about in collaboration with language models. That is deliberate. The rules are not theoretical.