Why your vocabulary score will not match another site's
This one differs from our other three in an unusual way. The two tests often do not even produce the same kind of number — one is a count of what you got right, the other is a projection produced by a model.
The short version. A vocabulary size test samples words from across frequency bands and projects a total — "you know about 27,000 words". A vocabulary test like ours counts — "you got 14 of 20 right". A projection needs a model of a population; a count does not.
Our reference sample is zero people, so we do not have the model, so we do not make the projection. That is the whole difference and it is not modesty — it is what we are entitled to say.
1. A size estimate is a claim about people, and it is doing more work than it looks
Method described by the tests themselves · peer-reviewed source read for the numbers
The method is not a secret and it is not unreasonable. A size test picks words from bands of decreasing frequency, sees where you stop recognising them, and multiplies that up against how many words exist in each band. The output is an estimate of a total you were never actually tested on.
Everything about that estimate depends on choices you cannot see: which corpus defines the frequency bands, whether a "word" means a lemma or a word family, how the projection handles the words you half-know, and how the sample was validated. Change any one and the number changes.
The peer-reviewed literature is unusually candid about this. Estimates of adult vocabulary vary widely, and the authors of the most-cited practical estimate say so in their title: practical estimates of vocabulary size dependent on word definition, the degree of language input and the participant's age. The definition of a word is listed first, ahead of the person being measured.
None of that makes size tests bad. It makes them a different instrument answering a different question, and it makes the comparison between their number and ours meaningless rather than merely imprecise.
2. "Language" is at least six factors, and vocabulary marks one of them
Canonical reference · read at source
The Kit of Factor-Referenced Cognitive Tests — 72 tests established as markers for 23 cognitive factors — separates verbal ability into at least six factors, each with its own marker tests:
| Factor | Marker tests | Roughly, the demand |
|---|---|---|
| Verbal Comprehension (V) | Vocabulary Test I & II · Extended Range Vocabulary · Advanced Vocabulary I & II | Recognising what a word means |
| Word Fluency (FW) | Word Endings · Word Beginnings · Word Beginnings and Endings | Producing words that fit a formal constraint, against a clock |
| Associational Fluency (FA) | Controlled Associations · Opposites · Figures of Speech | Producing words related in meaning to a given one |
| Expressional Fluency (FE) | Making Sentences · Arranging Words · Rewriting | Putting words together into acceptable phrasing |
| Ideational Fluency (FI) | Topics · Theme · Thing Categories | Producing many ideas about a thing, quickly |
| Verbal Closure (CV) | Scrambled Words · Hidden Words · Incomplete Words | Recovering a word from incomplete or disguised material |
Ours is a vocabulary task, which marks Verbal Comprehension. If the other site had you name as many animals as you could in a minute, that was Ideational Fluency. If it had you unscramble letters, that was Verbal Closure. Those are different abilities that share the English word "language".
This is the same finding as the spatial one, and worse. Spatial reasoning splits into three factors in this kit. Verbal splits into six.
3. The word list is the difficulty, and almost nobody publishes theirs
On a twelve-item folding test the difficulty lives in the number of folds, which we publish. On a vocabulary test the difficulty lives entirely in which words were chosen — there is nothing else. Swap five words and the same person gets a different score, with nothing about them having changed.
So the only way to interpret a vocabulary score is to see the words. Ours are on the result page, every one, with what you picked. If a test does not show you its items, or at least say how they were selected, it has not given you enough to know what your score means.
And here is ours, unflatteringly: our twenty words are ordered commonest to rarest by our judgement, not against a frequency corpus. A properly calibrated list would place each word against how often it actually appears in written English. Until ours is calibrated, our difficulty curve is an opinion — which is exactly why our result is a count and never a projection.
4. Option count and format set the floor, and they are rarely the same
We use five options, so a blind guess is right about one time in five — roughly four of our twenty by luck alone. A test with three options hands you more; a yes/no "do you know this word" format hands you a great deal more, and is also the format most exposed to people over-claiming, which is why the careful versions of it include invented words as a check.
An untimed multiple-choice test and a timed one are also not the same measurement. Ours has no clock, which suits a recognition task and follows the same reasoning we set out in what a browser can and cannot measure about your reaction time.
What would have to be true for two scores to be comparable
- The same factor. Recognition, fluency, closure and expression are different abilities, and this is the one most often got wrong.
- The same kind of output — two counts, or two projections built the same way. A count and a projection are not comparable at all.
- The same item selection, or at least a published one: how many words, chosen how, and from where.
- The same option count, so the chance rate is the same.
- The same language history in the test-taker, which is not a property of the test but decides more of the score here than anywhere else we measure.
- Both norm-referenced against a described sample, if either is going to say where you sit relative to other people. Ours is not, and says nothing.
So which number should you believe?
Neither, as a statement about your vocabulary as a whole. Ours is a count of correct meanings out of twenty words we picked, on one screen, on one day, with no clock. That is all it claims and all it can claim — our reference sample is 0 people.
The comparison that survives every objection on this page is the item list: not the number, but which words you knew and which you did not. The test prints that for you and stores nothing to do it.
Sources
- Educational Testing Service. Kit of Factor-Referenced Cognitive Tests (1976 Edition). ets.org — the six verbal factors reproduced in the table above, with their marker tests, read at source.
- Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How Many Words Do We Know? Practical Estimates of Vocabulary Size Dependent on Word Definition, the Degree of Language Input and the Participant's Age. Frontiers in Psychology. ncbi.nlm.nih.gov — open access; the title itself is the point being made in §1, that the estimate depends on the definition of a word before it depends on the person.
No timer, no account, nothing stored.