Measuring Training Impact – The Holy Grail of Learning
Continuous learning is essential to the ongoing success of any organization. But it comes as some surprise to find that many mid-tier organizations don't measure the impact of their employee training. That’s right – fewer than 15 % measure learning impact. Yet, behaviour changes and ROI are the very things almost every organization wants from their training department. How can you turn it into something more concrete?
More than 240 books on Amazon.com and 90 articles on the ASTD website are devoted to the measurement and evaluation of training. There are also more than 2,000 websites featuring consultants and tools to help the measurement process. Yet, despite all these extensive resources, most training departments in my opinion do not have an actionable measurement program.
As many of you are aware, sustainable behaviour change is a good indication of the success of a learning and development program. But it is equally important to ensure that change in behaviour is aligned with the business outcomes - whether that's to develop sales leadership skills or improve productivity among staff for greater return on investment, linking behaviour with business outcomes will enable you to measure and track any impact that change has had on the business and therefore demonstrate the true value of training. Companies that can demonstrate behaviour change, maintain this change and measure its effect on business results are the ones that perform the best.
The key is to focus 90% of time and resource on putting in place steps that will align learning with the goals of the organization and sustain these skills. The other 10% should be spent on equipping staff with skills through training programmes.
Continuous learning is essential to the ongoing success of any organization. It allows employees to cope with and also excel at increased job demands - allowing businesses to achieve excellence and maintain competitive advantage. So it comes as some surprise to find that many organizations don't measure the impact of their staff training and development on performance.
The reasons include lack of resources, confusion about what should be measured and lack of staff who understand measurement methodologies. Many simply don't think it matters to know which training is working best or having most impact. Also, many managers in charge of training simply don't see measurement as a priority. However, as the saying goes; “if you can't measure it, you can't manage it”. And if you can't measure or manage it, you may well ask yourself exactly what the point is in investing in it.
A research conducted by Deloitte in 2012 surveyed over 600 Training Managers asking them what their top two challenges were - 92% percent of respondents rated measurement as the top challenge.
So the biggest challenge of Training Managers and Vendors is not to deliver appropriate training, but to measure the effectiveness of the training imparted. Unless the value or the impact of training can be accurately measured, there is danger of it becoming an enjoyable activity, but producing very little value to the organization. While it is important for participants to enjoy the learning experience, that itself is not the objective of training. The objective is to deliver high impact training that will raise the bottom line by increasing productivity.
Are We Going About Measurement the Wrong Way?
Why is this? One reason organizations fall short in meeting their own expectations is that many organizations tend to approach the measurement problem the wrong way. Rather than establish a measurement process which delivers actionable information to help them improve the effectiveness and efficiency of their operation, they overly focus on measuring the four levels specified by the Kirkpatrick model (1=satisfaction, 2=learning, 3=job impact, 4=business impact).
Donald Kirkpatrick’s Four Levels of Training Evaluation has been the industry standard for measuring training success since 1979. Level-1 measures the participant’s reaction to the training program (smile sheets or program feedback form) and Level-2 measures the learning that has occurred, usually through pre- and post-testing.
But Level-3 (behaviour changes back on the job) and Level-4 (ROI) are rarely, if ever, measured. In fact, according to the Learning Resources Network, 77% of organizations use reaction measures; 36% use learning evaluations; 15% measure behavior change; ‘and only 8% measure results’.
While the Kirkpatrick model is a good taxonomy to help understand the basics concepts of learning measurement, it is limiting. Organizations which implement the most effective and complete measurement programs do not overly dwell on the measurement of job and business impact: they focus on capturing actionable, operational information which is relevant to their business and their key training programs. They set operational targets through key performance indicators and they measure them consistently and regularly.
At Learngage, where we work with companies on learning engagements to ensure effective business outcomes – we suggest these six strategies to improve a training program’s overall efficiency during the front-end planning stage. The first three strategies are aimed at increasing the benefit (value) of training programs, and the following three strategies are aimed at decreasing costs. Although they are presented as interdependent, they may be applied individually or in any combination,
- Align Strategy with Goals
Determining where training budgets and resources should be allocated is an important decision for any training unit. Unless the training program is a revenue generating business, the budget is viewed as an expense – which is being continually scrutinized. Managing training budgets and resources should be no different than managing any other investment, such as major equipment or information technology acquisitions. - Reduce Time to Competency
Since the ultimate objective of a training program is to develop skills, competencies and/or attitudes needed to resolve a performance deficiency and, in-turn, attain unit and organizational goals, closing the performance deficiency sooner may result in significant added benefits. For example, sooner operational staff acquires skills needed to implement a new process, the sooner the savings will kick-in. - Select the right blend of delivery options
Certainly, most training professionals are aware that eLearning, for example, can provide just-in-time access to training material whenever and wherever it is needed. Moreover, it can reduce travel costs and, in some cases, time required to complete training. - Consider Internal vs. External options
In certain circumstances, external training partners can be more effective and economical than internal trainers since they offer certain expertise in specific fields that are difficult to attain within a training group that caters to a wide range of needs. By capitalizing on external expertise, the learning curve of the internal training group may be reduced and costly errors may be avoided. - Duplicate efficient programs and detect problem areas
Are you comparing the costs and benefits of various learning initiatives and allocating funds accordingly? Are you forecasting budget and resource needs for your training program over the next 3-to-5 years, or beyond? Do you know how your organization measures up against other training organizations/units? In other words, are you making the most of your training budgets and resources?
By managing training budgets and resources from a centralized database, data from multiple training programs, businesses, and/or units may be compiled, analyzed, and compared – to identify ways of running training programs more effectively and economically.
“Measurement of training is a journey, not a destination” Training is a means to an end. No satisfaction, learning, or job impact measures are meaningful if they are not correctly applied to the right urgent and pressing business problems. And if we know what these are, then we can measure things which are far more specific to those initiatives, And it is a critical one: with organizations like yours spending 5-15% of your total payroll on training, it is important to know that these budgets are being well spent.
I look forward to hearing from you at pavan@learngage.com with examples of your measurement challenges and successes.
