ROI for learning (in times of rapid change)

Measuring return on investment by Glynn Jung

Measurement and accountability have long been the order of the day in most organisations. Those infamous three little words; return on investment (ROI) analysis has been a standard tool in a manager’s kit and deployed when needed. There are however a few challenges however with ROI:

  1. there’s a number of techniques or methods to learn and use;
  2. these techniques and methods sit more easily with accountants than learning and development specialists and
  3. determining ROI can be a long and ponderous process when often operating units within the organisation need us to address their needs very quickly.

So is there a rapid approach for an impatient world? Well yes, there are in fact two such approaches that I’ve been using: Critical Mistakes Analysis (CMA) and Fast-track Proficiencies.

But firstly “traditional ROI”: ROI is not a tool that you can use to prove the value of ALL Learning – it’s just a tactical tool, not a strategic weapon. We see it often used to justify a move from traditional training to self-directed eLearning but this is only really valid in regulatory training e.g. iterative Aviation Cabin Crew training for certification purposes. This is training which has to be done and proven to have been done. In this context we can reasonably claim to offer a return on investment via shortened time to competence and reduced or eliminated indirect costs such as shift rota cover, travel or machine / simulator time, expert support and tuition time.

Apart from those situations, I suggest that many organisations struggle to develop precise and meaningful ROI based on classic accountants’ techniques because:

  • they rarely calculate the cost and value of people’s time and efforts;
  • they rarely have precise targets for focusing the training  – i.e. what it is that needs to change or improve and what are the benchmarks for this and
  • there is little evidence of a direct relationship between training / learning and improved competence in the workplace.

We’ve struggled with this for decades. Way back in the 1970’s and 1980’s, I worked with IBM on their methodologies, (SATD and SATE) for demonstrating the value of training to the organisation. Like Kirkpatrick’s Level Four  – improved organisational performance – it proved increasingly frustrating to prove the link or outcome.

Even when you have done your due diligence and used the appropriate method to calculate the expected return, getting decision-makers to accept your analysis requires presentation, negotiation, collaboration, compromise, and persuasion. Think of it as a consensus-based process. It’s certainly rarely a quick fix for a problem. Additionally some people, like me, struggle to work like an accountant.

So are there any ways of ensuring that targeted learning or training can really deliver benefits, and do so rapidly? Well, there are two approaches that I’ve been using for a number of years which can be proved to deliver a return on investment (training time, expert time, learner time et al). These two approaches are:

  1. Critical Mistakes Analysis” and  
  2. “Fast-track proficiencies”.

Critical Mistakes Analysis

CMA is a proprietary methodology offered by Cognitive Arts, a subsidiary of NIIT. CMA derived from a US Government initiative to determine if it could ever be possible to guarantee a return on investment for training. By analysing the data gathered from major Six Sigma (no, don’t switch off) projects they found that a small number of very common mistakes caused most of the damage in an organisation. This was in every sector imaginable – manufacturing, logistics, health care, education, research, pharmaceuticals etc.

They were even able to put a cost against these common mistakes and so define the value of fixing them. The multi-year research project upheld the 80/20 rule and also spawned a commercial organisation with proprietary methodology. The thing is – all organisations have loads of data, of anecdotal evidence to point to where we should focus our attention for fixing mistakes; even the IT Help Desk records  – we don’t really need Six Sigma to find out what to fix. What typically is delivered as an intervention, by the way, is a 4 to 7 minute online tutorial and assessment for each fix, supported by simple online reference materials.

Fast-track Proficiencies

A CMA type approach will help reduce or eliminate the most damaging problems. Fast-Track Proficiencies on the other hand will help you deliver the key proficiencies essential to each different role. The approach came out of work done by Steve Rosenbaum and Jim Williams which resulted in a new, simple-to-implement set of techniques to help get employees up to speed in record time. It is all documented in their book: “Learning Paths: Increase profits by reducing the time it takes to get employees up-to-speed” (Pfeiffer and ASTD Press 2004), which includes a CD with templates and procedures for identifying key proficiencies. I was able to implement the techniques with a client immediately after reading the book.  It even includes a rapid approach called 30/30 Learning Paths for tackling just a handful of proficiencies in a role, (not least to prove to yourselves that it really works). Again, what is typically delivered is a blend of short tutorials and online workplace support information. For more information visit http://www.learningpathsinternational.com, check out the contents of the book via the “look inside” facility on Amazon or just buy the book.

Glynn will continue to discuss the classic approach to ROI in our next blog edition. Follow us on twitter @aurionlearning for our latest blog articles and updates. 

Advertisements

Tags: , , , ,

2 responses to “ROI for learning (in times of rapid change)”

  1. Shane Hogan says :

    The CMA model looks interesting. The challenge for us is to measure impact outside of our own organisation. We provide an eLearning service to other Irish public bodies. How can we measure the impact or ROI of that service?

    • Ciara Cunningham says :

      Hi Shane, Thanks for the comment. Going beyond simple cost savings, the trick to measuring the impact or ROI of that service is by working closely with the client and establishing what the goal or course outcome actually is and how the eLearning course helps meet it. If you can find a simple metric that you can use before starting your eLearning it makes the evaluation easier. For example, companies that have established a system of job roles and competencies, it is easier to measure the positive impact of training for an individual.

      Please feel feel to get in contact with one of the team, should you wish to discuss this further or indeed if you have any queries,

      Thanks! AL

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: