Consider a very simple question, What does two plus two make? The obvious correct answer is four. Most people will arrive at four by simple addition using the equation 2 + 2 = 4 but in fact the answer four can also be arrived at by adding a superscript two to the first two, i.e. 2² = 4. If you place the second two alongside the first one then the answer becomes 22 and if you use Roman numerals then it is possible to get one thousand one hundred and eleven, 1111, etc. etc. Each answer depends upon how the words “two”,  “plus” and "make" are interpreted.

Another simple well-known example asks “What is the minimum number of lines that can be drawn to go through all nine dots (in three regular rows of three) without taking the pen (or pencil) off the paper?

The answer of four is often given as an example of ‘thinking outside the box’ as the question does not say that you cannot move outside the area delineated by the dots. So when can you think ‘outside the box’ and when must you remain within it? Thinking outside the box is not really seen as acceptable in the the two plus two example even though the problem did not say you couldn’t use superscripts etc. In fact if you really think outside the box for the dots example the minimum number of lines is one  - either use a very thick nibbed pen or pencil or fold the paper so all the dots are in the same plane etc.

The point about these two examples is that in a multiple choice question where the numbers of words should be kept to a minimum it is difficult to cover all the parameters so that only one of the four answers can be correct.  When setting questions examiners should not be trying to catch students out but the three ‘wrong’ answers must have something about them which could lead the student to at least consider them rather than reject them outright. To be fair to the IB when it sets multiple choice questions it does not award marks for the correct answer but for the best answer to the question asked.

I’ve been reading an interesting book called The Intelligence Trap by David Robson. It is not specifically about chemistry but I recommend it as it will certainly improve your critical thinking and is a great addition to the NOS and TOK aspects of the IB Diploma. It basically looks at why otherwise intelligent people sometimes make wrong, even stupid, decisions. It has got me thinking about multiple choice questions as they depend so much upon precise wording and in fact have inherent assumptions in them.

In his book Robson asks an interesting (non-chemistry) question which I will turn into a multiple choice question.

Ignoring any thinking outside the box (e.g. Donald is looking at Greta in a mirror and so can also see Rita etc.) most people will give answer B or D as we need to know whether Greta is married or not and the question does not specify this.  This is the ‘gut reaction’ which often dictates the answer due to the time constraints in an examination. In fact the answer is C*.

A question that many would also answer by a ‘gut reaction’ but which, like the marriage question above, benefits from a deeper analysis was asked on one of this year’s IB chemistry papers.

Ignoring the fact that a pedant might claim that the graph should just show one horizontal straight line as the mass of the beaker (as opposed to the mass of the beaker + contents) remains constant, the answer needs to be able to explain why the initial mass is higher, why more mass is lost and why the slope is steeper (rate increases) for Curve 2. The gut reaction is that answer B is correct. Doubling the mass of the marble chips increases the initial mass, it will also double the mass of carbon dioxide evolved (assuming the acid remains in excess for this double amount).  Powdering the marble chips will increase the surface area and so increase the rate.  Since the acid is in excess, answers C and D will not change the mass lost as the amount of carbon dioxide evolved will be the same. Answer A cannot be correct as the initial mass does not change. So at first glance, answer B would appear to fit all the criteria completely, especially when looking at the change in mass lost. The mass lost for Curve 1 is m₁ and the mass lost for Curve 2 is m₂.  It is clear from the graph that m₂ is double the mass of m₁.

In fact more careful thought will show that none of the answers is correct. The graph has no scale but for Curve 2 the mass of the marble chips is double the initial mass of the marble chips for Curve 1. If the extra mass of marble chips is x then the initial mass of marble chips in the flask is shown as x’ which is the same as the value of x. Since CaCO₃ (M_r = 100) only loses 44% of its mass when the carbon dioxide (M_r = 44) is evolved as a gas (the remaining 56% stays in the beaker as calcium ions and part of the water formed), the mass loss for the initial graph is equal to 0.44x which is much less than that shown.  Moreover when the mass of the marble chips is doubled to become 2x the mass loss will now be 0.88x. This means that for answer B to be correct Curve 2 should end up above Curve 1, not below it as shown in the question. In fact it should even finish above the initial starting mass of Curve 1.

It is not easy to set good, unambiguous multiple choice questions as the setter needs to think through carefully all the possible assumptions that are or can be made and yet keep the number of words (or diagrams) to a minimum. Multiple choice questions are a good way to test understanding rather than just testing learned factual information. This raises an interesting question for the new examinations in 2025. According to the March 2020 curriculum review report on My IB there will only be two examination papers. Paper 1 will contain Section A: Multiple choice questions and Section B: Data analysis and lab-based questions. This implies that students will need to have access both to a calculator and to the IB data booklet for the whole of Paper 1 which will have a big effect on the nature of multiple choice questions that can be set. For example, questions looking at trends in physical properties across a period or down a group can no longer really be asked as the trends can be clearly deduced simply by looking in the data booklet.

* Whether or not Greta is married has no bearing on the answer. If Greta is married then Greta (married) is looking at Rita (not married). If Greta is not married then Donald (married) is looking at Greta (not married). In either case a married person is looking at an unmarried person so the probability is 100%.