Tested models

The models we tested

Every model that went through at least one of our tests with usable output, grouped into size bands; within a band they are listed by maker, newest first for each maker. For MoE models the first number is the total size and the second is the number of parameters that actually do the work.

Large

a trillion parameters and up

Mid-sized

a hundred and fifty billion to a trillion

Small

fifteen to a hundred and fifty billion

Miniature

under fifteen billion

Composite

not one model, but a system built out of several

According to their makers, Pareto and both Aion models are built from several models whose size the makers do not disclose, so they have no band.

Two more models played in Twenty questions on a gap but have no page of their own, so they are not in the table. Ternary Bonsai 2 27B (Prism ML, 27 B) wrote nothing but "It is" sixteen times, and Fireworks Ember 1 did not finish its game.

We also tried Reka Flash 3 (Reka), but none of its runs returned usable output; in the zero test it returned zero characters three times.

In the tests column, Train is the train test, Dragons the dragon benchmark, Keykeeper the keykeeper test, Silence the silence test, Fan the fan test, Zeros the zero test, Voice the control runs for A voice above the world, Amplifier The amplifier and the spring, and Tool calls unwanted tool calls. Gap is Twenty questions on a gap, Lighthouse the lighthouse test, Tools the tools test, Pressure Pressure on a fact, Ambulance The ambulance that never came, and Strict boss The strict boss and the sad emoji. Seven tones is the article A reward and seven tones and I'll do anything the article An altar, commandments and a plan for the day. From the bottom up is the 28 short tasks test, Dice the fair die test, and River the river percentages test. A link goes to the article when the model is in it, otherwise to the appendix with its result. An empty release date means the maker does not state one or I have no documented date.

* The maker does not disclose the size. The band is our estimate and only approximate: it goes by where the model stands in the maker's lineup and by the models in this table whose size is known.