By Glynn Jung
Whichever product or service you seek, an organised, comprehensive selection process is required – perusing websites of e-learning companies just doesn’t work. The selection process for a suitable e-learning vendor should be guided by whether they are supplying:
- off the shelf e-learning titles or
- design and development services.
It helps if you develop a checklist, (indeed most purchasing departments demand this) so that you are consistent in comparisons. We recommend listing all the attributes of a perfect-fit vendor and deciding which features are must-haves, whether these are immediate needs or future growth and finally how important each feature is (“points”).
We also recommend that organisations adopt the “MoSCoW” method for determining their needs. This is based on agreeing:
- ‘should have’
- ‘could have’ and
- ‘would be nice to have’ – most organisations concentrate exclusively on “must have”.
Below is a sample checklist that you might consider as a starting point for your own selection of a technology vendor.
In any e-learning vendor selection process there are generally a number of important criteria, such as pricing, technology, quality, service and so on. With regard to technology, ensure your vendors know what they will be dealing with in your organisation.
|Attribute||Must Have||Now or Future||Points|
|Does the vendor serve organisations similar to yours?|
|What do current customers similar to yourselves say|
|Is the vendor’s customer base sizeable enough to ensure continued operation?|
|Are customer references available?|
|Does the vendor support customer implementations with training and support?|
|Can the vendor assure you of a successful implementation?|
|Does the vendor have a proven plan for implementation of its system?|
|How long has the vendor been operating in the e-learning market?|
|Is pricing in line with similar offerings?|
|Does the vendor rely primarily on revenue from its commercial system or is customization a large part of its income?|
|Does the vendor offer a base price that scales with volume?|
|Does the price include everything you will require to get started?|
|Can you see a relationship between cost and quality?|
|Does the vendor guarantee successful operation?|
|Is there a stated quality policy?|
|Are “bugs” resolved quickly or do they wait for a future release?|
|How easy is the system to use: How much training is required?|
|Does the system require minimal resources for administration?|
|How reliable is the system: How often and for how long does it go down?|
|Do the technical qualifications reflect our technology|
|Is the system’s technology up to date? State-of-the-art?|
|Does the vendor rely on outside support for its basic services?
Is the system capable of delivering current types of media?
|Does the vendor provide multiple solutions for your needs?|
|Can the system support with various authoring tools?|
|Does the system support the browsers we need supporting?|
|Does the system support mobile devices?|
|Does the system support our compliance requirements?|
|Are maintenance fees readily available?|
|Does the vendor require the purchase of periodic updates?|
|Does the vendor provide 24/7customer support?|
|Does the system support multiple languages?|
|Does the system support the accessibility we require?|
|Can the software be placed in Escrow?|
Bespoke e-learning development
If the need is for bespoke course development or off-the-shelf titles many of the same technical considerations still apply. You need to ensure that any course content can be accessed and viewed using devices which your staff will be using. You further need samples of their work to compare but before you do this we recommend you identify:
- who will be using the courses,
- where they’ll be using them and
- what you consider to be fit-for-purpose regarding design of content.
For example if your IT people operate a “no download, no plugins” policy that the course material requires no extra software, will operate properly on your LMS (if you use one) or as a web-playable course and on any special devices your learners may use.
Location of learning is significant – if it’s in a retail store, warehouse or factory audio is rendered virtually useless.
Your list may be modified as you start talking to potential vendors: the critical thing is to keep your absolute priorities and needs in front of you at all times and not be swayed by sophisticated marketing or sales.
Project planning and management.
Ensure the vendor provides a clear project approach which is logical and understandable – they’re the experts so they should be able to keep to plan, warn of any pitfalls and deliver on time, within budget and to agreed benchmarks.
The final thing I want to talk about is working relationships. Working with willing, supportive, responsive and flexible vendors can quickly develop into a true partnership: if you really solely on numerical weighting systems you run the risk of attempting to work with people who don’t fit your organisation’s or people’s style and culture.
Demand three personal referees similar to yourselves in their client base. Talk to these referees; don’t use a pro forma reference form: find out what they’re like to work with and what their strengths and weaknesses are.
Also find out who will actually be working on your project: assess them as people when you come to interview your shortlisted companies – have a get-out clause ready in case the sales time disappears after they’ve closed the business and there’s no-one to talk to in the vendor organisation who understand your needs.
What checklist do you use when selecting a vendor? Please let us know your comments or share with others who you think may benefit from this checklist.
ABOUT THE AUTHOR, Glynn Jung
Glynn Jung is Non-Executive Director at Aurion Leanring. He has over 25 years’ experience delivering innovative and cost-effective learning and process improvement strategies for a wide range of public, private and third sector organisations.
By Maresa Molloy, Instructional Designer at Aurion Learning.
At Aurion Learning, our experience tells us that one of the best ways to guarantee learner interest and retention is through the appropriate and frequent use of interactivity.
This short article identifies the top five benefits of adding interactive activities to your online learning resources.
- Changes learner behaviour – Interactive activities such as, scenario-based exercises, behaviour modelling and guided practice prompt learners to review the lesson against their own work-place practices which is an ideal approach to affect positive behaviour change amongst staff.
- Ensures the message is understood – formative assessments, questioning and assessed role-plays provide learners with instant feedback, offering appropriate affirmation or explanation depending on whether the learner has answered correctly or incorrectly.
- Connects with the workplace – printable job aid exercises, such as checklists and action plans, prompt learners to focus on the application of the course material to their particular role.
- Engages all learning styles – variety of presentation, practice and assessments support high levels of user interactivity and engagement. Rich task-based multi-media and audio immerse learners by providing realistic practice in the subject areas being taught.
- Promotes a positive learning experience – the use of appropriate interactivity encourages learners to return to refresh their learning as well recommend the resource to their colleagues as material for group or individual learning.
As a quick rule of thumb, we feel that the definition provided by American Instructional Technology guru, Brandon-Hall encapsulates the spirit of good interactivity:
“An interaction is an engagement of the mind……. not the finger!”
by Ciara Cunningham, Marketing Manager Aurion Learning
Aurion Learning and Irish League of Credit Unions (ILCU) are delighted to announce that their online learning system – CU Learn has been named as a finalist for an IITD National Training Award.
ILCU in partnership with Aurion Learning configured and implemented a new learning management system – CU Learn in 2012, to manage and deliver online training, accredited programmes, record classroom training attendance and provide leanring and development information in one central area for the 485 affiliated credit unions, 4,500 staff and 9,500 volunteers across Ireland.
CU Learn has also supported ILCU to fulfill its obligations with regard to new regulatory requirements by placing a renewed emphasis on the training and education of its members.
The Irish Institute of Training & Development (IITD) National Awards celebrates excellence in learning and development and CU Learn has been revealed as a finalist for the Networks and Group award which recognises the unique co-operation that is now taking place between organisations who are working together in networks and groups around the country to develop training solutions to meet the demands of their industry.
Speaking about the announcement, Suzanne Ryan, Head of CU Learning & Development at ILCU, commented:
“CULearn.ie has been central in supporting our focus to facilitate the efficient administration of training and education amongst our affiliated credit unions, particularly at a time when there is a renewed emphasis on developmental support, tailored training and Continued Personal Development, in the credit union movement.
The system has already added value to ILCU and has proven good learner outcome. CU Learn is available for free to all credit unions and we encourage those not already registered to logon to www.culearn.ie and start reaping the benefits.”
Dr. Maureen Murphy, Managing Director at Aurion Learning said:
“CU Learn has been an exemplary partnership and we are very proud that it has been shortlisted for the prestigious IITD National Training Awards. We look forward to continuing our close working relationship with the learning and development team in ILCU as they continue to transform their learning and development services to Credit Unions throughout the country.”
CU Learn is powered by the NetDimensions’ Talent Suite. Aurion Learning is a certified reseller for NetDimension’s award-winning learning & talent management products in Ireland and the United Kingdom.
The winners of the IITD awards will be revealed on Friday, 22nd March, 2013 at Killashee House Hotel, Naas.
by Maresa Molloy, Instructional Designer at Aurion Learning
If a picture is worth a 1000 words, then it is important to ensure that you add the right pictures and images to improve your e-learning rather than distract from it. Simple – right? Well, actually yes.
This article contains tips and advice to ensure that you get the picture to maximise your e-learning environment.
Knowing your photos from your icons
There are three main types of images:
- Icons –They are multi-use clip-art type images such as, Important, Test Question, Review Point, or Key Fact. iconfinder has a good free range of icons to use.
- Graphics – a graphic is a designed image and are very specific. Graphics can be charts or graphs; they can be illustrations or word art.
- Photographs – Finding the right photo that portrays everything that you want and are trying to represent can be a powerful tool, however, these images can be hard to find, particularly if you are not a professional photographer! We have listed below an assortment of both paid for and free images that you can use:
Finding your images:
If you would like to find free images, you can use the Creative Commons area on Flickr or stock.xchng. You can also use Compfight to help you search for photos that you can use. Remember to read the guidelines on proper attribution!
Now that you have you images, it is important to remember the following:
Don’t over egg the pudding.
After you have spent hours pulling together the content for your e-learning environment, the last thing you want to do is to clutter the page with images. Equally, images should not be there to take up space. Take a moment before adding an image and ask ‘What is the purpose of this image? Ensure that it has relevance and reinforces what it is you are trying to get across.
It is important that your images work with the content of your e-learning programme making it easier for participants to focus and don’t serve as a distraction. It is also worth noting that the images also need to work with each other.
by Glynn Jung
In this second instalment of learning analytics, Glynn discusses the classic approach to return on investment (ROI) for learning.
If you take the accountants’ approach to ROI for learning analysis there are five important points to note;
- The assumptions made before conducting the analysis are important and you must document them.
- It takes more than one ROI model to establish value, and not all ROI models will be valid for a given case.
- Collaboration with customers and senior management in identifying Learning benefits is critical; ROI determination is not a one-sided exercise.
- It is too easy to fall prey to the temptation to just “play with the numbers” until an acceptable result appears.
- Calculators can only “do numbers” – they can’t compute the value of the intangibles.
There are a number of classic approaches to show the financial impact that a given investment (your e-learning project) will have on a business.
The issue here is “How long will it take to get all the investment back?” Payback analysis results are expressed in months or years. This is calculated as the net investment amount divided by the average annual cash flow from the investment. The payback analysis is easy to use and easy to understand. However, it does not take into account the time value of money (which is addressed by another model, Net Present Value, or NPV). Payback also does not consider the financial performance of the investment after break-even Payback is best used to establish relative priority between potential projects.
Accounting Rate of Return (ARR)
This is another “simple” method for calculating the return on a major project. It gives a quick estimate of a project’s payback, supports comparisons between projects and it also considers returns for the entire life of the project.
Net Present Value (NPV)
Net Present Value is best used for long-term projects. It considers the time value of money- it expresses future cash flows in terms of their value today. While this is the strength of NPV, it also means that this method is not appropriate for projects that do not have clearly defined cash flows, or when the benefits of the project are not financial. NPV can be tricky!
Internal Rate of Return (IRR)
IRR is not as easy for non-accountants to understand or to calculate as NPV. I don’t even understand the terminology let alone the techniques.
Full business impact: the Balanced Scorecard
Many human performance interventions have complex effects on business results. In recent years, the best known method of impact assessment has probably been the balanced scorecard.
The balanced scorecard looks at the effect of a project in four areas:
- learning and
- internal processes.
It is holistic and long-term, and it is forward-looking. Financial results are still an important area considered, but they are not the only element.
If your organisation uses balanced scorecards it may be useful to relate the benefits of your Learning project to each of the four areas of the scorecard. Show how the program objectives relate to the objectives and important questions in each area. The emphasis is on process, not on metrics.
Measurement and accountability have long been the order of the day in most organisations. Those infamous three little words; return on investment (ROI) analysis has been a standard tool in a manager’s kit and deployed when needed. There are however a few challenges however with ROI:
- there’s a number of techniques or methods to learn and use;
- these techniques and methods sit more easily with accountants than learning and development specialists and
- determining ROI can be a long and ponderous process when often operating units within the organisation need us to address their needs very quickly.
So is there a rapid approach for an impatient world? Well yes, there are in fact two such approaches that I’ve been using: Critical Mistakes Analysis (CMA) and Fast-track Proficiencies.
But firstly “traditional ROI”: ROI is not a tool that you can use to prove the value of ALL Learning – it’s just a tactical tool, not a strategic weapon. We see it often used to justify a move from traditional training to self-directed eLearning but this is only really valid in regulatory training e.g. iterative Aviation Cabin Crew training for certification purposes. This is training which has to be done and proven to have been done. In this context we can reasonably claim to offer a return on investment via shortened time to competence and reduced or eliminated indirect costs such as shift rota cover, travel or machine / simulator time, expert support and tuition time.
Apart from those situations, I suggest that many organisations struggle to develop precise and meaningful ROI based on classic accountants’ techniques because:
- they rarely calculate the cost and value of people’s time and efforts;
- they rarely have precise targets for focusing the training – i.e. what it is that needs to change or improve and what are the benchmarks for this and
- there is little evidence of a direct relationship between training / learning and improved competence in the workplace.
We’ve struggled with this for decades. Way back in the 1970’s and 1980’s, I worked with IBM on their methodologies, (SATD and SATE) for demonstrating the value of training to the organisation. Like Kirkpatrick’s Level Four – improved organisational performance – it proved increasingly frustrating to prove the link or outcome.
Even when you have done your due diligence and used the appropriate method to calculate the expected return, getting decision-makers to accept your analysis requires presentation, negotiation, collaboration, compromise, and persuasion. Think of it as a consensus-based process. It’s certainly rarely a quick fix for a problem. Additionally some people, like me, struggle to work like an accountant.
So are there any ways of ensuring that targeted learning or training can really deliver benefits, and do so rapidly? Well, there are two approaches that I’ve been using for a number of years which can be proved to deliver a return on investment (training time, expert time, learner time et al). These two approaches are:
- “Critical Mistakes Analysis” and
- “Fast-track proficiencies”.
Critical Mistakes Analysis
CMA is a proprietary methodology offered by Cognitive Arts, a subsidiary of NIIT. CMA derived from a US Government initiative to determine if it could ever be possible to guarantee a return on investment for training. By analysing the data gathered from major Six Sigma (no, don’t switch off) projects they found that a small number of very common mistakes caused most of the damage in an organisation. This was in every sector imaginable – manufacturing, logistics, health care, education, research, pharmaceuticals etc.
They were even able to put a cost against these common mistakes and so define the value of fixing them. The multi-year research project upheld the 80/20 rule and also spawned a commercial organisation with proprietary methodology. The thing is – all organisations have loads of data, of anecdotal evidence to point to where we should focus our attention for fixing mistakes; even the IT Help Desk records – we don’t really need Six Sigma to find out what to fix. What typically is delivered as an intervention, by the way, is a 4 to 7 minute online tutorial and assessment for each fix, supported by simple online reference materials.
A CMA type approach will help reduce or eliminate the most damaging problems. Fast-Track Proficiencies on the other hand will help you deliver the key proficiencies essential to each different role. The approach came out of work done by Steve Rosenbaum and Jim Williams which resulted in a new, simple-to-implement set of techniques to help get employees up to speed in record time. It is all documented in their book: “Learning Paths: Increase profits by reducing the time it takes to get employees up-to-speed” (Pfeiffer and ASTD Press 2004), which includes a CD with templates and procedures for identifying key proficiencies. I was able to implement the techniques with a client immediately after reading the book. It even includes a rapid approach called 30/30 Learning Paths for tackling just a handful of proficiencies in a role, (not least to prove to yourselves that it really works). Again, what is typically delivered is a blend of short tutorials and online workplace support information. For more information visit http://www.learningpathsinternational.com, check out the contents of the book via the “look inside” facility on Amazon or just buy the book.
Glynn will continue to discuss the classic approach to ROI in our next blog edition. Follow us on twitter @aurionlearning for our latest blog articles and updates.