ITIL V3 sample exam question: is it just vague or plain wrong?

Blog entry submitted by skeptic on Fri, 2008-02-29 13:12. [nid:478]
in

It is sample exam questions like this that give candidates the cold horrors.

I'm moving a comment from Alison Adams up to here where everyone can see it:

Q19 from the recent V3 sample paper #6 is shown below. I believe that the 'correct' answer as supplied is wrong - or at best ambiguous - and would like your views, as at present I am marking it differently to the official guidance....

19 Which is the first activity of the Continual Service Improvement (CSI) model?

a) Assess the current business situation
b) Carry out a baseline assessment to understand the current situation
c) Agree on priorities for improvement
d) Create and verify a plan

Based on the CSI book, page 15 "2.4.4 CSI approach" which describes the CSI model, the first step of 'what is the vision ?' includes 'embrace the vision of the business', 'understand business objectives', 'align business and IT strategies'. The closest fit to this would be answer A.

The official correct answer is B, but the same section of the CSI book lists the activities of 'baseline assessment' and 'assess the current situation' (as opposed to 'business situation'), as step 2 'where are we now ?'.

I have submitted this same query to ISEB weeks ago for them to forward to the examining board as I am concerned that this is perhaps in live exam papers and I would get it wrong ! No response yet so I thought I might have more luck here :-)

Your views please ?

Alison

ITIL be fixed in Version 3 (SP1)

Bookmark this post with:
  • Delicious
  • Digg
  • Google
  • StumbleUpon
  • Yahoo
  • Reddit
  • folkd
  • Facebook
  • Propeller
  • Magnoliacom
  • Newsvine
  • Furl
  • Rojo
  • Technorati
  • Icerocket

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Ambiguous

Submitted by IT Governance Blog (not verified) on Fri, 2008-02-29 15:55.

I chose A as well but A and B are very ambiguous to me. A doesn't really say HOW to assess - at least with B there is the action of establishing a baseline. But I don't believe you can know WHAT to baseline until you've done an assessment, which is why I chose A. And based on what is described as being documented, it seems clear it must be A.

I'll bet there are more than a few of these...

Bill

Vague Questions

Submitted by ITSMdoc (not verified) on Fri, 2008-02-29 21:36.

Some of my friends sitted Foundations recently and I compiled a list of questions they had. Some 70% of them are logical and easy, some are light brainers, but some 5% look impossible to answer on that attendee level. I have read all 5 (6) books, some of them a few times, and they still take 5-10 minutes to answer with literature and all.
I still think that (given the scope of v3) foundations level exam should be a little more then knowing the glossary, knowing what is what and what is where. There are other levels for showing off.

So whats new!!

Submitted by buraddo on Sat, 2008-03-01 06:21.

So when have any ITIL exams been anything other than ambiguous and misleading.. They are based on the British higher education system which tests comprehensive more so than understanding.

I once tried to bring a group of non-native english speakers through the management exams, before local language was available, and it was very sad.. Bright people, with great understanding, completely demoralized by language comprehension.

Another reason why the certification structure is completely wrong for this and most other professional education schemes. When did "professionals" become too high and mighty to service an apprenticeship. When did education and experience take such different paths. What happed to education through immersion.

As I have said before. Nothing has disappointed more, since the infestation of IT Datacenters by Certified Netware Engineerings in the late 80s and 90s.

Brad Vaughan
http://blogs.sun.com/buraddo

Don't knock the Brits ;-)

Submitted by JamesFinister on Sat, 2008-03-01 07:37.

Brad,

I'm not sure what your experience of British higher education, is but over the years I would say its influence on the exams has kesseneed, not increased. I trained candidates for both the highly Dutch influenced EXIN exams and the British ISEB exams and in the early days the ISEB exams were head and shoulders above the EXIN ones in terms of testing true understanding - though they were far from perfect. The early manager's exam featured the need to write long essay answers that drew equally on your knowledge of the books and on your real life experience. They even allowed freedom to have some debate with the examiner. The move towards multiple choice questions, especially at the post foundation level, isn't something you find at British universities. Personally I'm not against multiple choice questions in the right context if they are well written. I think anyone who has undertaken the ISACA CISA exam will tell you it is no walk over and the questions really make you think.

James

Keeping it in the Empire

Submitted by buraddo on Sat, 2008-03-01 08:34.

My experience with British education comes only from what was transplanted into Australia.

Still whether short-answer (which really becomes an issue of translating knowledge into concise text) or multiple choice (which when written in ambiguous form as per this blog entry example or using the classic, which answer is more right than the others trick) is still a test in english comprehension and not understanding.

I stick by my original concern. Its not whether they make you think, but whether they make you think about the right stuff.. I have no doubt that they are challenging, but a accomplished student can always win over and accomplished subject matter expert.

Brad Vaughan
http://blogs.sun.com/buraddo

It is not a Brit problem

Submitted by Liz Gallacher on Sun, 2008-03-02 14:49.

I agree with James - the problems with the V3 exams is nothing whatever to do with the British education system (even if such a thing exists - the Scottish curriculum, secondary school exam structure and university degree structure is very different from England). Multiple choice is seldom used, and the belief commonly voiced in the UK is that the problems with ITIL V3 exams stem from catering to an American approach to education and testing(which may be equally unfair).
The real problem is the way that the questions emerge from the examiners board (which, with its global representation, is far less likely to take a British or Dutch approach). Writing good multiple choice questions, which truly test understanding is a skill. I tried it myself some time ago, and gave up - I do not have the skill required, despite my ITIL knowledge. The members of the board may be very learned, but not have this skill. There should have been wider consultation, the proposed questions should have been opened to a wider audience of reviewers/trainers etc. Instead we have questions released, then withdrawn, which gives a very poor impression.

As far as the written examination for V2 managers is concerned - as a trainer and examiner, I find it strange that it is so misunderstood. It is not, and has never been, about writing essays. It is about providing written answers - in bullet points, tables, or in a report-format, which cover the points in a logical way, and demonstrate understanding. Surely anyone operating at a manager level has to do this all the time in their real jobs - writing reports, giving presentations, etc. It IS under exam conditions, with a time limit, but a good candidate can get a pass mark for a question from 2-3 pages. It is the ones who waffle, or brain-dump who struggle with time.

Liz Gallacher
Freelance Trainer and Consultant

the driver for the use of multiple-choice exams is commercial

Submitted by skeptic on Sun, 2008-03-02 20:54.

"the belief commonly voiced in the UK is that the problems with ITIL V3 exams stem from catering to an American approach to education and testing". Voiced in New Zealand too :-D

But even though I think the American education system produces new-age numbskulls who are inarticulate and bordering on illiterate and the American media pounds their remaining brains into 30-second pap-machines that couldn't write a meaningful paragraph, I suspect the real driver for the use of multiple-choice exams is purely commercial: marking can be automated, eliminating the need for expensive paper-markers.

Evidence indicates that many certification and training decisions are now made with the profitability of the vendors in mind not the education of the trainees.

Its not all about Multi-guess

Submitted by buraddo on Mon, 2008-03-03 03:48.

So I will take back what I said about the Brit education system. It is my cultural birth right as an Aussie to heckle the mother land as much as possible. They are only complicit in developing the modern mass education system.

Ignorant controversial ramblings follow

"The French (inventing the university) and the Italians (via the Roman Catholic Church, inventing multi level schooling) are as much to blame. Before these things, education was more of an apprenticeship, with scholars, artists and professionals able to show a direct genealogy to there mentors. Your skill was given credibility through who you worked with and the exposure they gave you to success. I don't care whether the V3 exams have British or Dutch influence. Paper based education and exam based certification is just a poor instrument for creating capability."

I am not sure if multi-guess is necessarily the problem. Its a tool to measure content knowledge and if done properly, can be good as any other. But capability is X parts knowledge & Y parts skill & Z parts experience. Its true that the cost of marking the exams is almost eliminated through the use of multi-guess.

I think it is right to say that the whole concept of exam based certification is driven by a financial motive. Having a certification scheme is a known method for improving adoption of any framework. Companies are always looking for ways to try and quantify the skill of staff.

There is nothing about the way the certification scheme measures success that can reinforce the goal they state for the certification. They make no attempt to measure if the courses work (I think skep you made the commment about feedback). So growth, adoption and revenue can be the only measure, and therefore motive.

Brad Vaughan
blogs.sun.com/buraddo

Exam driven training

Submitted by JamesFinister on Mon, 2008-03-03 08:50.

Brad,

One of the implications of the changes in the way ITIL is examined, which IIRC dates back to the v2 refresh, is a shift in the way the subject is taught. V1 training schemes focussed less on exam technique and practice and more on developing ideas. I know from speaking to some of the early examiners that they often found new insights by reading what students had written. We also took into account a delegates performance on the training course itself. I think this used to attract up to 10% of the marks, and we took the assessment very seriously.

Now it sems the training is more black and white and not aboput development of the individual.

ISACA seperate out technical competence evidenced by exam success from on the job certification by professional peers. Perhaps a way ahead?

professional certification as an annually renewed status

Submitted by skeptic on Mon, 2008-03-03 07:33.

Agree with Jim: the model of professional certification as an annually renewed status administered by a governing body who track ongoing professional development and behaviour becoming a registered professional is the way to go. It is also light-years ahead of the capability or intent of OGC, APMG or itSMF. IOSM tried but muffed it. I said on ITSM Watch

There was nothing until recently that provided certification and college for practicing [ITIL] professionals. Now we have the Institute of Service Management in the U.K. and the newly formed Institute of Certified Service Managers, or the IT Service Management Institute in U.S.

Do any of these have the official recognition of OGC? What governance does it provide over their activities or standards? Why is the ICSM-USA only for ITIL “masters” but the U.K. body more broadly for “senior ITIL-ers”? Because nobody is internationally regulating the emergence of these bodies.

and in another article

We [in IT] still have a long way to go to meet the levels of best practice set by other industries. A professional is measured by three attributes: their attitude, development and ability...There needs to be international standards of tertiary qualification, and very different courses offered by the universities. There needs to be international networks of accreditation. Most of all there must be much higher expectations set of the people we entrust with these huge projects.

PiMP your ITIL Cert :)

Submitted by buraddo on Mon, 2008-03-03 08:13.

I still think the continuous certification model used by PMP and others (even in different industries) falls short is that it is only requiring more book based learning or in most cases just attendance. It is really just measuring interest and commitment, not capability..

Brad Vaughan
blogs.sun.com/buraddo

CPE

Submitted by JamesFinister on Mon, 2008-03-03 09:01.

We should keep a distinction between a practical experience based component to the inital qualification, and an on going CPE element post qualification. With both, however, there are practical issues to consider. Who, for instance, decides what qualifies as CPE, where do we find people suitably qualified to assess practical work, and how do we deal with the possible mismatch between the whole range of ITIL processes and functions compared to the span of control of an individual candidate.

And how do we stop it becoming another revenue generating self perpetuating aspect of the industry?

The saviour of the investment

Submitted by buraddo on Mon, 2008-03-03 08:19.

I posted a comment on the other topic from skep related to this

http://www.itskeptic.org/node/479#comment-2409

Continuous Certification like the PMP model will help to prolong the life of the certification and stem to rot created by commodotizing the system. It create the ability to extend the brand to other partners and products which make people continually invest in the model. And the reason I know why the lord of ITIL are thinking on this, is it could double, triple or even quadruple the revenue per customer they have now..

Brad Vaughan
blogs.sun.com/buraddo

Simple mistake?

Submitted by D Wells (not verified) on Tue, 2008-03-04 01:52.

It may be that this is just a simple mistake? Interestingly, the identical question appears in Sample paper 4 for the v3 Foundation (Question 11). In this paper the correct answer is also "b". However, we are given slightly different options and "b" reads "Understand the vision of the business".

I always understood this to be an unambiguous (correct) answer, although it appears something has been 'lost in translation' between sample paper 4 and sample paper 6!

By the way, I too have been awaiting a response from ISEB on this particular issue ...

Lost In Translation...

Submitted by Alison Adams on Wed, 2008-03-05 15:44.

D - I agree entirely that Paper 4 Q11 is a completely reasonable question with no ambiguity.
So what on earth went wrong with the amendment in Paper 6 and why didn't anybody notice before it went public ?!
I just had a discussion on this with my class as we used paper 6 as a 'mock' and I have asked them to let me know if the question appears in their real exam - I dont know what to advise them - put the APMG answer or the right answer ?!
I posted this on itSMFI too, hoping for an answer there or to my email to ISEB/APMG.
Still waiting.....

Alison

Just Plain Wrong !

Submitted by Alison (not verified) on Thu, 2008-03-20 06:57.

Rosemary Gurney was kind enough to respond to this via the itSMFI bulletin board and confirmed a 'typo' that sneaked through QA. It is fixed in sample paper 7. For Rosemarys full comment see http://www.itsmfi-forum.org/view_topic.php?id=307&forum_id=12&jump_to=945#p945

Alison

how do such typo's happen?

Submitted by Liz Gallacher on Thu, 2008-03-20 08:37.

Firstly - this was not just any typo - a misspelling or such. SURELY you check and double-check the correct answers are given! The same thing happened with the "How to pass your V3 exam book", which lessened the credibility of an otherwise good book.
Equally worrying is the answer given on the itsmfi forum:-

"The examination team responsible for the v3 Foundation papers, both live and sample find it difficult to comment directly on this forum to queries surrounding specific questions, even if they are aware of them.
This is because they are contracted to APMG as the Official Accreditor and the agreed channel for communication of this nature is from the Accreditor, to all of the respective Examination Institutes and then to Training Providers.
The examination team have been aware of the issue regarding this particular question for some time, they have advised APMG and have resolved the issue. They are not responsible for issuing any communications.
However, on this occasion I am happy to advise that the cause of the issue was quite simple. An error was made during the data entry phase, answer 'B' was input whereas it should have been answer 'A' and this was missed during subsequent QA.
This has been corrected and additionally, Option A has been reworded to - Understand the business objectives.
The question is still available as a sample and will appear on Sample Paper 7 which will be issued shortly. At this time, all Training Providers will be advised to retire Sample Paper 6
from use."

So, when mistakes are spotted, they are not communicated properly, and the answer each time is to issue another paper. Am I the only training provider who prints student manuals to include the mock exams - as mine are spiral-bound, that makes changing them rather difficult!
Liz Gallacher
Freelance Trainer and Consultant

It's the OODA loop!

Submitted by Al Maurer (not verified) on Wed, 2008-05-28 21:34.

OK, I've come late to this discussion, having been working with Sample Papers 3 & 4 and new to Sample Paper 6. Thinking ITIL, I would have guessed answer b. In ITIL, all the processes start with figuring out where you are, making a plan, then acting on it.* That immediately narrows it down to a & b--but b is the better answer because it's more inclusive of figuring out where you are. a refers to the business side b is the situation generally.

So I think it's typically tricky but not ambiguous.

* That is a lot like Plan-Do-Check-Act or in Col Boyd's formulation Observe-Orient-Decide-Act. The OODA Loop. :-)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Other things the IT Skeptic is up to

It may be egotistical to think anybody cares, but for those of you visiting this site while at work, here are some links for your idle curiousity: