by Glynn Jung
In this second instalment of learning analytics, Glynn discusses the classic approach to return on investment (ROI) for learning.
If you take the accountants’ approach to ROI for learning analysis there are five important points to note;
- The assumptions made before conducting the analysis are important and you must document them.
- It takes more than one ROI model to establish value, and not all ROI models will be valid for a given case.
- Collaboration with customers and senior management in identifying Learning benefits is critical; ROI determination is not a one-sided exercise.
- It is too easy to fall prey to the temptation to just “play with the numbers” until an acceptable result appears.
- Calculators can only “do numbers” – they can’t compute the value of the intangibles.
There are a number of classic approaches to show the financial impact that a given investment (your e-learning project) will have on a business.
The issue here is “How long will it take to get all the investment back?” Payback analysis results are expressed in months or years. This is calculated as the net investment amount divided by the average annual cash flow from the investment. The payback analysis is easy to use and easy to understand. However, it does not take into account the time value of money (which is addressed by another model, Net Present Value, or NPV). Payback also does not consider the financial performance of the investment after break-even Payback is best used to establish relative priority between potential projects.
Accounting Rate of Return (ARR)
This is another “simple” method for calculating the return on a major project. It gives a quick estimate of a project’s payback, supports comparisons between projects and it also considers returns for the entire life of the project.
Net Present Value (NPV)
Net Present Value is best used for long-term projects. It considers the time value of money- it expresses future cash flows in terms of their value today. While this is the strength of NPV, it also means that this method is not appropriate for projects that do not have clearly defined cash flows, or when the benefits of the project are not financial. NPV can be tricky!
Internal Rate of Return (IRR)
IRR is not as easy for non-accountants to understand or to calculate as NPV. I don’t even understand the terminology let alone the techniques.
Full business impact: the Balanced Scorecard
Many human performance interventions have complex effects on business results. In recent years, the best known method of impact assessment has probably been the balanced scorecard.
The balanced scorecard looks at the effect of a project in four areas:
- learning and
- internal processes.
It is holistic and long-term, and it is forward-looking. Financial results are still an important area considered, but they are not the only element.
If your organisation uses balanced scorecards it may be useful to relate the benefits of your Learning project to each of the four areas of the scorecard. Show how the program objectives relate to the objectives and important questions in each area. The emphasis is on process, not on metrics.
Continuing his series of guest blog posts on talent management, Steve Curtis, EMEA Channel Director at NetDimensions talks about performance appraisals and PDPs.
So onto Performance Appraisals – I don’t know about you, but I think a lot of managers don’t really enjoy performance appraisals. In my humble experience managers in some larger organisations will complete a performance appraisal because HR tell them that they have to, but they will put as little effort into the thing as possible and in reality their views in the appraisal will be far too subjective. If they like the person, then the appraisee will get a good rating, and if they don’t it’s highly unlikely that that person will do well, irrespective of their ability to do the job.
So this then is the conundrum of HR – most HR managers understand this, and so want to achieve two things:
- Remove subjectivity and replace it with objectivity.
- Make the process as easy as possible for all concerned.
Every company I have worked for in the software world has used a different form for performance appraisals, and since I have worked for a number of companies for more than 5 years, I have seen performance appraisal forms change inside a company a few times as well. This is the challenge for the software business that wants to supply this functionality to clients – they will have one way of doing it, and often they will want you to replicate this in software – which is difficult to do without customisation.
Constituents of a Performance Appraisal
I’m not going to tell you in this blog what a performance appraisal is – I’d actually be quite worried if you don’t know this….however let’s for a minute consider the type of things that different companies might (or might not) want inside a performance appraisal.
- A review of the past period – often 6 months or more often 12 months – including a competency review – has the appraisee got better in the competencies related to his or her job during the period?
- A review of objectives and progress – most HR groups want their employees to be set objectives at the start of the period and then the performance appraisal is the formal assessment of progress against those objectives. Objectives can be personal in nature, or can be linked to department, business unit, or company objectives.
- A review of the next period – setting and agreeing some objectives for the next period. These again can be supplemented during the period if needed.
- An overall rating – some organisations want an overall rating for the appraisee.
- A skills review – what skills does the appraisee have that do not directly tie to their current role?
Some organisations might have more than this, but some will have less. Some will separate the objective setting for the following year, and want to put this into a separate area and time. Evaluating a person’s performance in a given year may sound easy, but with management changes happening quite often in large organisations, and changes of role, these can make performance appraisals a complex area to try and handle in the real world, and to try and automate this in software can be even harder.
Benefits of Automating Performance Appraisal Management
For Talent Management what is the implication of a good process here – what are the benefits of automating this area?
- Automatic storage of a person’s potential improvement over time.
Paper records in HR are an administration overhead and time consuming to manage.
- Automated management of the workflow process.
Who does the appraisal or part of the appraisal need to go to and when?
- Conformance with regulatory requirements.
Managing performance appraisals in this way means that the business can much easier conform with any government regulations in the area.
- An ability to look wider across the business and report on the critical talent much easier.
Having this data across the business often means that it is much easier for the business to select talent pools – groups of people who can be accelerated and become the future leaders.
- An increase in objectivity.
Software should allow other people to be involved in the appraisal process for more critical or senior people. This gives the business the ability to have a 360 degree view at appraisal time. Think about these things when you go to companies and talk to them about this.
The PDP and its Relationship with the Performance Appraisal
So now you understand what a performance appraisal is, what is a PDP? A PDP is a Personal Development Plan. At the start of a period (this may be for instance when the performance appraisal for the last year has just been completed), the business will want each user to know what he or she needs to accomplish in the next period. This goals setting can include goals that the individual wants to achieve (personal goals), department goals, business unit goals etc. Goals should be capable of being pushed down through the organisation hierarchy to the individual.
So why would an individual want or need a personal goal? Perhaps they want a promotion or want to move roles into a more senior position. It might be that some of the competencies required to be in this position are ones that the person needs to have training in order to improve his or her rating. Personal goals always need agreement from management but are often a critical element of the PDP. The PDP is a living breathing and evolving document during the period, and the goals from the PDP then drop into the Performance Appraisal at the end of the period, and the user is formally rated against those goals at that time.
Does this Have an Effect on Compensation?
A pretty obvious question, but one you need to be aware of in software terms. Compensation is often directly or indirectly tied to ratings and scores from performance appraisals. Objectives set for the period often have bonus pay linked to them, and so if you remember the Gartner diagram from my second blog in this series (Talent Management 2: Competencies, Ratings, Scales & Pitfalls) compensation management is a part of Talent Management, and you will sometimes get request from businesses to link the performance appraisal process through into a compensation product.
There is a lot more to Performance Management but that will do for now. A lot to think about – but think of it from the viewpoint of the person who is wanting to buy – what the HR director wants will be different to the CEO, and different to the business unit manager. We’ll discuss that a little more next week.
By Maresa Molloy, Instructional Designer
We love when clients come to us full of enthusiasm for a new e-learning programme but comments like: “Our staff need to know our policies – let’s put them online into an e-learning programme,” or “We need to make sure they’ve read all the material – an e-learning programme is perfect!” can set the alarm bells ringing.
Educating staff on new information, and identifying whether or not they’ve read and understood the material, are perfectly good reasons for developing an e-learning programme. Bombarding your staff with everything you know about the information and making sure they can recite it word perfect is not.
Imagine trying to educate your staff on Health & Safety and providing them with pages and pages of policies and procedures on Health & Safety and then expecting them to list these verbatim. This is clearly not a good idea – they’ll never remember all of the material and they’ll get bored fairly quickly.
Now imagine providing your staff with a course which teaches them the key information on these policies and procedures and showing them how to apply these in their work. This is clearly more effective.
But how do you split the wheat from the chaff?
At Aurion Learning, we believe that the key to designing a successful, performance based e-learning programme is to identify current performance gaps. To do this, we use what we call our ‘DIF analysis’:
- What tasks do your learners find Difficult?
- What tasks do they find Important to their jobs?
- What tasks do they do Frequently?
Once you know these tasks, you can design a course which targets what they really need to learn, and furthermore, you can spend more time and resources focusing on making this content more interactive and enjoyable.
|Assume that your goal is to increase their knowledge.||Identify what they need to do differently.|
|Provide all of the information you have on the subject.||Provide only the information that can help them do something differently.|
|Ensure that they know all of the information.||Ensure that they can use the information.|
(Source material Cathy Moore)