Cognitive Reflection Test

None of these three puzzles is hard. Each one is a single sum, and every one of them can be done in your head.

What they have in common is that a number is available before the sum is. A bat and a ball cost £1.10, and the bat costs a pound more. Ten pence turns up on its own, before anybody has worked anything out.

Shane Frederick published the three in 2005 and called what they measure reflection. Not arithmetic, and not care. The specific thing is noticing that your first answer is a first answer, and looking at it again before you say it.

That is why the people who get these right are not simply the people who are quicker at sums. The sum was never the obstacle.

Your score can only be 0, 1, 2 or 3. That is coarse, and it is the price of the one thing this page has that its neighbours do not.

These three items have been put to tens of thousands of people since 2005, in the same words. The spread of the score is published, so a number from this page can be placed against something real rather than against a guess.

Add a fourth question of our own and that comparison is gone. Whatever the new set measured, nobody would have measured it before. The page would be back where the rest of this hub is, with a number and nothing under it. A score that can be read against published figures is worth a great deal. The same argument sits behind what counts as a good digit span.

One run is still one run. Three items is a short measurement, and a second run cannot tell you much, because by then you know the answers.

This is the largest problem the test has, and it is bigger than anything about the puzzles themselves.

Stieger and Reips put the three items to 2,137 people online and asked a plain question afterwards: had you done this before? Almost half said yes, and the ones who said yes scored 1.65 out of three against 1.21 for everybody else.

So this page asks the same question, after every puzzle, and it asks before it marks you. Asking afterwards would collect nothing useful, because everybody remembers a puzzle once they have been shown the answer.

The count sits on the result card next to your score, and the comparison underneath is built from the people who said no.

If you already knew the answers, the settings panel has the same three puzzles with the numbers moved. Those are ours rather than a published form. Thomson and Oppenheimer built a proper second version in 2016, for exactly this reason, and this is not it.

Each puzzle has one wrong answer it was built to produce. Ten pence, a hundred minutes, twenty four days.

Any other wrong answer is a different event. Somebody who types six pence was working the problem and slipped. Somebody who types ten pence did not get as far as working it.

So the card reports them separately. Two right with one tempting answer is a different run from two right with none, and the card shows which you had.

The tag belongs to the number rather than to the person who typed it. Nothing on this page knows what you were thinking, and it does not guess.

• Bat and ball. The answer is 5 pence. Ten pence is the pull.
• Machines and widgets. The answer is 5 minutes. A hundred is the pull.
• Lily pads. The answer is 47 days. Twenty four is the pull.

The reference distribution is the naive half of the Stieger and Reips sample, which is 1,174 people who had not met the test.

Their scores ran 33 in 100 with none right, 26 with one, 26 with two and 14 with all three. That is a median of one, and it is why a single correct answer is close to the middle rather than close to the bottom.

Each score is placed at the middle of its own band rather than the top of it. Two correct beats about 73 in 100, rather than the 86 in 100 who scored two or fewer. The other people who scored two are not people you beat.

It is somebody else's sample and not ours, and the page says so wherever the bar appears. It was recruited online, it was German speaking, and the education mix of one study is never the education mix of the internet.

Our own norm goes up when a cohort here reaches a thousand runs, and not one day before that.

Frederick reported that scores on these three items lined up with how his participants answered questions about risk and about waiting for money. That is a finding about a sample. It is not a statement about you, and nothing on this page has been checked against anything.

It is not an intelligence test. It is three questions, and no score here has been compared with any measure of intelligence.

It screens for nothing, it rules out nothing, and a score of zero is not a diagnosis. It is three puzzles on a Tuesday.

Getting better at these three is getting better at these three. That is what researchers mean by near transfer, and whether brain training games are worth it covers what happened when the wider claims were tested.

If something has genuinely changed in how you think, that belongs with a doctor rather than with a web page.