The e-learning euphoria is over. Gone are the days of extended budgets and expanded investment in virtual classrooms and interactive training without clear business objectives or returns. And that’s the way it should be. If you’re spending on e-learning,
by Site Staff
October 30, 2003
Don’t worry, there’s no need to panic—you don’t need to be a finance MBA to build a solid business case for your e-learning project. Here are a few pieces of good news: First, e-learning can deliver significant returns, and there are some simple rules you can follow to keep projects on the positive ROI track. Second, approaching your learning objectives with an ROI focus can help you pick the right courses to deliver by e-learning. And third, without ever picking up a calculator, you can evaluate the e-learning initiatives likely to deliver the most value to your organization.
First, the Good News
Many companies deploying e-learning solutions have achieved significant returns by reducing training-related costs and improving employee productivity. In its exploration of e-learning deployments over the past two years, Nucleus Research found the most successful deployments followed very similar strategies:
- They defined objectives prior to deployment. Instead of initiating dialogue with technology vendors before identifying their e-learning objectives, all of the successful companies internally determined goals and the steps to take to meet those goals before calling in vendors.
- They followed a rapid-deployment timeline. The companies successfully implementing e-learning followed quick deployment timelines, using open platforms for development that were easily deployed and scalable for future needs. We recommend less than six months for deployment, at least for launching the initial components.
- They planned strategies for user adoption. Companies that trained users in the new technology and addressed adoption issues achieved rapid returns. For example, Lucent Wireless University initially launched its e-learning solution in a classroom with instructors so users would receive support as they adopted the solution.
The e-learning deployments studied ranged in size and scope from departmental projects based on off-the-shelf content and simple Web-based delivery platforms to extensive investments of more than $5 million to deliver courses to a global user community. Some companies, including IBM and Lucent Wireless University, achieved more than 1,000 percent ROI from their e-learning deployments. They did so by significantly reducing the cost of training-related travel to their organizations while impacting productivity and other benefit areas.
Although the scale of investment varied widely, all companies made an ongoing investment in the development and management of content—either by internal personnel or external consulting staff. Companies should consider both the initial cost of deployment and the ongoing cost of managing and developing content when budgeting for an e-learning deployment.
When it came to receiving benefits, there was a wide variance in the ratio of direct to indirect benefits among deployments. While some companies completely paid for their e-learning investment through reduced travel costs, others found productivity gains to be a key return area.
When thinking about your e-learning project, you should consider two basic tiers of benefits:
- First-tier benefits are benefits realized through the elimination of existing costs. The most common first-tier returns are reductions in travel expenses, human-resources overhead, regulatory compliance and customer-support costs. For example, Lucent Wireless University achieved a 2,302 percent ROI from its e-learning deployment in 1999 by cutting back on travel expenses and fees for third-party instructors.
- Second-tier benefits are achieved when companies leverage e-learning to support changes in the strategic direction of the organization in a way that positively impacts the bottom line. Lucent achieved second-tier returns in the form of enhanced manager productivity when it implemented reporting and tracking features, allowing managers to follow employee progress without having to prepare reports manually. IBM achieved second-tier returns by making its managers more effective in driving teams to produce greater revenues.
Identifying the types of benefits you expect to receive is critical to achieving a positive ROI, but so is identifying the characteristics of the learning you plan to provide.
Picking the Winners
Companies achieving a positive ROI made e-learning investment decisions based on the fidelity of their courses—and you can too. To evaluate course offerings in order to assess whether an e-learning strategy will produce benefits, Nucleus has defined the concept of “fidelity.” Fidelity incorporates three main elements of learning:
- Complexity of the course content.
- Level of student/instructor interaction.
- Sophistication of the learning environment.
To begin assessing how e-learning could affect your company’s training costs, it is crucial to determine your current classroom courses’ fidelity levels. By examining your current course offerings in terms of a fidelity scale ranging from low fidelity to high fidelity, you can make decisions about e-learning that maximize ROI.
Once you determine course fidelity, you can determine whether you can re-create or increase the fidelity of a traditional course with an e-learning solution and justify the investment based on the number and richness of learning experiences it will provide. If you cannot deliver the same fidelity with an e-learning solution, you are better off with traditional training methods.
Low-fidelity courses have relatively basic content and a minimal need for interactive learning. Tactically, they’re often the best option for e-learning replacement. Low-fidelity courses include those that have the least student/instructor interaction and complexity of content. Such courses may have other peripheral goals, such as boosting employee morale, allowing an employee to meet a checklist of training requirements or providing certification on specific skills or knowledge. These courses tend to be the type that “push” content to users, and the success of the program is assessed by a completion or retention test. In general, courses of low fidelity are not vitally linked with the company’s key business objectives.
Courses that fall toward this end of the fidelity scale are most suitable for rapid e-learning replacement. The travel expenditures in this area of training may currently be large, but the content is neither highly customized nor particularly interactive. In many cases, these courses could be replaced with Web conferencing tools such as PlaceWare or WebEx. In addition, there are many off-the-shelf or ASP-provided training programs that can effectively replace classroom training quickly and cost-effectively. Thus, a company may significantly reduce travel-related training expenditures without compromising the learning potential of the course.
Medium-fidelity courses have a greater reliance on student/instructor interaction and more complex content. In this category, a course’s learning environment tends to be more sophisticated. Examples of this level of fidelity could be the teaching of the rewiring of a BMW GPS system or the restringing of a new line of guitars. In this level of training, relatively complex simulations may be used to test the learner’s skills. Courses of medium fidelity are more likely to be indispensable to the employee’s current and future job performance and are more significantly linked to the company’s revenue-generating activities.
For medium-fidelity courses, the choice to deploy an e-learning solution is not as simple. The cost of an e-learning solution is likely to be greater, but the payoff could be considerable as well. At the middle of the fidelity scale, you must consider other factors in making an e-learning investment decision, including current training-related costs and the number of students involved. Training-related travel costs really add up when a large group of employees is being trained at a distant classroom, and in such a situation, an e-learning solution could result in a positive ROI. However, if a medium-fidelity course has relatively few learning experiences, higher costs for course development may outweigh any travel-related savings.
High-fidelity courses have complex content and need to be customized to individual student needs and levels. In high-fidelity courses, the content and method of teaching is fine-tuned to the specific needs of the learner. There is also a high level of student/instructor interaction and a personalized evaluation of the learner’s progress. In order to replicate the learning potential of these courses, the e-learning solution must have a highly interactive quality as well. Simulations, agent-based interactions and highly context-situational feedback would be required to reproduce such a sophisticated learning environment that in the classroom would involve real hands-on student/instructor interaction.
The high fidelity of these courses means high costs to develop their electronic counterparts. However, in some cases, technology truly allows for a course’s potential to go beyond the experience of the classroom with complex simulations and interaction.
Consider the case of airline pilot training: Conventional training could include daily test flights with an instructor explaining how to deal with a particular atmospheric change. A student learns how to deal with one specific situation each day, and learning is often dependent on the weather. With an electronic simulator, that same student could be placed in a virtual situation and learn how to react with a hands-on experience that requires neither an instructor to explain each step, nor appropriate weather conditions. This simulation could expose the student to 10 different situations in a day, whereas actual instructor-led training would take weeks for the same content. In this simplified example, it’s clear how electronic replacement of high-fidelity courses can save time and money while not just maintaining the fidelity, but completely surpassing the quality of the traditional learning.
In a more concrete example, IBM deployed its Basic Blue e-learning initiative to deliver annual training to more than 5,000 of its managers. Originally, the training was delivered in a five-day event where all managers were brought together to learn about IBM culture, management and strategy. Over time, IBM managers’ jobs became more complex, and the company needed more than a five-day session to teach the vast amount of material and skills. IBM also wanted to make training more of an ongoing process than a one-time meeting. IBM’s new training program includes virtual collaborative tools, interactive online simulators and virtual learning labs to complement on-site training. Despite the expense of development, IBM achieved an ROI of 2,284 percent and a payback period of two weeks. Clearly, IBM found an electronic solution that surpassed the fidelity of its current training—and managers now learn five times as much as in the traditional training program.
The Challenge
Many companies make the mistake of attempting to save money on high-fidelity traditional courses by replacing them with a lower-fidelity electronic solution. In the process, they sacrifice the learning potential of the course. The goal of maintaining or surpassing the fidelity and learning potential of the current course must always be kept in sight: Not all e-learning solutions are equal, and “electronic” does not always mean “better.”
If you have a high-fidelity course that greatly benefits a few upper-level employees, don’t sacrifice the value of the course by deploying a lower-fidelity electronic solution for the sake of e-learning. In the long run, the students’ content retention—and your ROI—will suffer. Until you can match or surpass the fidelity of the original course, leave it as it is. However, if you find you can realistically replace a high-fidelity course electronically (as in the flight-simulation example), you can realize great benefits.
High-fidelity courses replaced with e-learning may not show the immediate direct benefits like travel-related cost savings found in low-fidelity e-learning strategies. However, over time the indirect benefits will likely outweigh costs as the learning experience remains rich and is easier for more students to access.
Getting Started
Once you’ve taken stock of your e-learning objectives and course fidelity types, it’s time to assess where e-learning will likely deliver returns for your organization. Without opening a spreadsheet, you can use some key criteria to assess a project and the likelihood of a positive ROI:
- Breadth: How many people will be helped by the project? The greater the number of people, the greater the potential ROI.
- Repeatability: How often will people use the application? The more often an application is used, the greater the ROI.
Secondary factors to consider include the following:
- Cost: The more costly the task, the greater the benefit from automation or appropriate technology support.
- Knowledge: The greater the potential to reuse the information in the system, the greater the potential ROI.
- Collaboration: Communication between employees is costly, so the greater the collaboration component, the greater the potential ROI.
Once you’ve passed the test on the five factors, you’ll want to gather and validate data to support your ROI calculation. Nucleus Research uses a three-year time horizon to evaluate projects. Costs and benefits can be either one-time or recurring, so be sure to include them appropriately. Follow these basic rules when gathering and including costs in the calculation:
- Count everything that is directly associated with the project. For example, “I purchased a Web server for this project.”
- Don’t count infrastructure items not associated with the project. For example, “I used the existing Web server.”
- Do count infrastructure items that were driven by the project. For example, “The company purchased a Web server because of this project and two others like it” means you should include one-third of the cost.
Benefits are more difficult to assess and can be either directly quantifiable or indirect productivity-based gains. It’s easy to claim that productivity gains should not be included because there are no direct benefits or reductions in budgets from increased productivity. However, you should consider the additional employees you would need to hire to do the same work or the increase in output from the same number of employees. The challenge is to account fairly for gains in productivity. To do this, correct for inefficient transfer of time—which simply means that the total time saved rarely equals the total additional work performed. To measure, follow these rules:
- Measure based on fully loaded cost.
- Correct for inefficient transfer of time using a value of 1.0 for line workers to 0.5 for general employees.
- Find a corroborating measurement that supports the change. For example, if the legal department saves 10 percent of its time, do you expect it to fire 10 percent of the lawyers or increase its productivity by 10 percent?
When in doubt, you may want to survey your users and average their estimates, look for case studies or similar examples to get an idea of the scale of benefits or look for industry benchmarks that can provide guidance on an appropriate estimate for your company. If you choose a conservative estimate, consider calculating the ROI twice: once for the expected ROI and once for the worst-case scenario. In fact, running different scenarios (What if the consultants don’t finish on time? What if one division refuses to participate?) will give you and the finance folks a clear view of how closely you need to track key elements of the project as you deploy.
Conclusion
To assess how e-learning could produce positive returns, it is important to assess your current classroom offerings in terms of fidelity. E-learning strategies are not a blanket solution for reducing training costs. However, by classifying your current traditional training in terms of fidelity, you can better identify where e-learning is likely to produce positive returns.
Many organizations have achieved significant returns from even modest investments in e-learning by identifying low-fidelity courses that can be rapidly delivered at a fraction of the cost of traditional methods. The other key area where companies achieved a positive ROI was in delivering high-fidelity courses to enrich the quality of the learning environment. These are two areas we recommend companies considering an investment in e-learning explore as there are opportunities for a significant return on investment. Using these basic rules and a structured view of the costs and benefits of your e-learning investment will help keep your e-learning on track to deliver clear returns to the bottom line.
Rebecca Wettemann is vice president of research for Nucleus Research Inc., an independent ROI research firm based in Wellesley, Mass. For more information, e-mail Rebecca at rwettemann@clomedia.com.