Critique of BCS sample exam questions

This note assesses the sample questions on the BCS website and highlights how these differ from real papers which candidates can expect to find when they sit the exams.


Improvements

BCS has made some significant improvements to the assistance they provide candidates for examinations leading to practitioner level Certificates. Possibly the most useful change has been to the sample questions that are now provided for each practitioner module.

Previously BCS posted sample papers and marking guides of somewhat dubious quality, one for each module. Rightly these were criticised; they had become a little long in the tooth and so were not indicative of the sorts of questions that candidates would experience and they did not cover the syllabus. Following the recent syllabus changes we now have a set of brand new sample questions and specimen answers. These have, justifiably, received positive comments. They are much more like the questions on exam providers papers and they reflect more of the breadth of the syllabuses. Even so, they are receiving criticism.

Candidate expectations

What do candidates actually want from examples? One answer might be that they want questions that are similar to those they can expect to find when they sit the exam. On that score I think BCS has succeeded. The questions are, in general, pitched at the correct level and address the major topics of the syllabus.

Candidates might also want to see what level of detail is expected in the answers and how marks might be allocated, particularly if the answer includes multiple instances of different things – such as marks per lane and per process if the question calls for a swim lane diagram. Here again I think it would be churlish to criticise BCS. The marking guides provide reasonable, complete examples of answers and show in quite reasonable detail how marks would be awarded.

So where can the examples be criticised?

The criticisms

Three criticisms have been levelled at them. Firstly they do not reflect an exam paper. Yes, the questions may be indicative of a real paper but they are not packaged as a set of questions all to be answered in an hour. So they do not serve well those candidates who want to practise their management of time. We all know how quickly an hour can slide by so this valid ambition on the part of a candidate is poorly supported by the questions.

Secondly, for the questions to be used under ‘test’ conditions, and not just as something to look at to get a feel for them, it should not be possible for a candidate to see the sample answers when reading the questions. Unfortunately this is exactly what happens for some, not all, of the questions for some of the modules. It’s a simple problem, with a simple answer – re-paginate the text so that the questions are separated from the questions.

And thirdly, perhaps the mildest of the criticisms, real exam papers have a set of questions based on a single scenario but the sample questions are each based on their own mini scenario. When this was first raised with me I will admit that I was inclined to dismiss it as trivial but the more I discussed it the more significant it seemed. These examples are the only way for direct-entry exam candidates to find out what the exams will be like. If they see scenario, question, scenario, question why would they not believe that that is how the exams are structured. Such candidates have enough problems without the shock of finding that their expectations are incorrect.

Final thought

It would be churlish to criticise too severely an organisation that is trying hard to help. That is really not the intention of this note. BCS deserves credit for the improvements it has implemented but users of the questions need to be aware that there are still limitations to their utility.

Let's hope that BCS will make them even better next time.