How Do You Measure Success?

You can never be sure of how effective you’ve been in creating a learning intervention unless you measure its impact. This is something that few organisations do. It is often considered enough for something to look ‘cool’ and for people to like it, to brand it a ‘success’. Also, evaluating the success (or failure) of something can be a substantial piece of work – and what if this creation of yours hasn’t made a difference?!

I would argue that unless you know what you’re doing right (and wrong) then you stand little chance of achieving greatness in any endeavour. You are probably aware of the Kirkpatrick Evaluation Model and I won’t go through it again here. Instead I’ll offer some practical advice based on my experience:

Make use of your LMS statistics – if you have a learning management system make use of the data that it gathers. This will give you information on for example; numbers of users, completion of material, scores in assessments and more. Be wary though – just because some has opened a piece of content it doesn’t mean they’ve been through it, just because they’ve completed something doesn’t mean they’ve paid any attention, and just because they’ve scored 80% or above in the assessment doesn’t mean they’ll remember any of it in a month’s time, or be able to apply it. Nevertheless, LMS statistics do form an important part of the data you need to gather.

Have a clear target – at the start of the project identify what you are trying to affect within the organisation. Set an amount by which you want to change it and timescale within which to see the change. This might be an increase in sales, reduction in errors, fewer complaints, a faster response or fewer accidents. Find the metrics that are already used to measure this within your organisation and make use of them. See whether there is any existing trend, pattern or variation. Having launched your training intervention look for trends, for example, if you see a reduction in errors in using a system for a group who you’ve given system training to, it’s not unreasonable to suggest there might be a connection between the training and the outcome.

Ask your users – survey them before and a few weeks after going through your training. See if they have perceived an improvement in their performance. Talk to them and gather their comments. Talk to, or survey their line managers to see if they’ve noticed an improvement. You don’t need to talk to all your users, a representative cross section will do.

If you can gather data from these three sources then you stand a good chance of seeing what went well and what you could improve upon. Think about how you will measure success right at the start of your project and put a plan in place for doing so. It will focus you on improving peoples’ performance and make it much more likely that you will succeed.