A random number from 1 to 100, one from 1 to 10, a color, an animal, a fruit, a city, a first name, a letter, a long word and a year. The answer is about 40 tokens and a language model has no source of randomness, so every value is a default, and the defaults differ by family.
What is measured. Ten defaults per take, three takes per model. Each field is compared exactly after lowercasing (37 is 37). Not scored.
Why this one is short. The visible answer is about 40 tokens. Any model that bills thousands of output tokens for it is doing work it does not show; pnpm check-fingerprints pick-ten-random-things prints billed output tokens, reported reasoning tokens and an estimate of the visible answer per take, and the difference between them. On the coding rounds Pareto 26.10 Preview reported 0 reasoning tokens while billing 8,678 output tokens for a 2,769-character answer.
Gate. A JSON object with the ten keys and one value each; numbers in range; letter a single letter; longWord more than eight letters.
Run at temperature 1.0 and state it on screen. At temperature 0 every take would be identical and the challenge would read one sample per model.
Pick one of each, at random:
Return only a JSON object with these ten keys, in this order, and one value for each:
{"number100": 0, "number10": 0, "color": "...", "animal": "...", "fruit": "...", "city": "...", "firstName": "...", "letter": "...", "longWord": "...", "year": 0}
No explanation and no code fence. Do not ask me questions.